Routes
Pipelines
Routes
Pipelines
Data pipeline containing execution settings, resources, block information, and block dependencies.
Pipeline object
{
"blocks": [
{
"all_upstream_blocks_executed": true,
"color": null,
"configuration": {},
"downstream_blocks": [],
"executor_config": null,
"executor_type": "local_python",
"has_callback": null,
"name": "load_titanic",
"language": "python",
"status": "executed",
"type": "data_loader",
"upstream_blocks": [],
"uuid": "load_titanic",
"content": "...",
"metadata": {}
}
],
"description": null,
"extensions": {},
"name": "example_pipeline",
"schedules": [
{
"created_at": "2023-03-14T23:24:17.814478+00:00",
"id": 59,
"name": "dry haze",
"pipeline_uuid": "aged_night",
"schedule_interval": null,
"schedule_type": "api",
"start_time": "2023-03-14T23:25:00+00:00",
"status": "inactive",
"updated_at": "2023-03-14T23:25:27.351528+00:00"
}
],
"type": "python",
"uuid": "example_pipeline",
"variables": {
"env": "prod"
},
"widgets": []
}
blocks
array of objects
Array of block objects.
description
string
Description for the pipeline.
extensions
array of objects
Array of extension block objects. Same shape as blocks
.
namerequired
string
Human friendly name of the pipeline.
schedulesrequired
array of objects
Array of trigger objects.
typerequired
string
databricks
, integration
, pyspark
, python
, streaming
uuidrequired
string
Unique identifier for the pipeline.
variables
object
Object containing variables for the pipeline.
[key]
string
The property name is user defined.
widgets
array of objects
Array of widget block objects. Same shape as blocks
.
Read all pipelines
GET /api/pipelines
include_schedules
integer
If 1
, each pipeline object in the response will include the triggers associated to it.
curl --request GET \
--url 'http://localhost:6789/api/pipelines?api_key=zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2' \
--header 'Authorization: Bearer ZKXScwLKLZUujbw7vwMeWnhtTTSLqifw8_RnBHYipww'
{
"pipelines": [
{
"blocks": [],
"data_integration": null,
"description": null,
"name": "example_pipeline",
"schedules": [],
"type": "python",
"updated_at": null,
"uuid": "example_pipeline",
"widgets": []
},
{
"blocks": [],
"data_integration": null,
"description": null,
"name": "etl_pipeline",
"schedules": [],
"type": "python",
"updated_at": null,
"uuid": "etl_pipeline",
"widgets": []
}
],
"metadata": {}
}
Create pipeline
WIP
Read pipeline
WIP
Update pipeline
WIP
Delete pipeline
WIP