Pipelines
Create pipeline
POST /api/pipelines
namerequired
string
Human readable name of the pipeline.
typerequired
string
The type of the pipeline:
integration
, pyspark
, python
, streaming
Note that python
is a standard (batch) pipeline with a python backend, while pyspark
is a batch pipeline with a spark backend.
clone_pipeline_uuid
string
If supplied, the uuid of a pipeline to clone.
blocks
array of objects
Array of block objects. See the blocks section for more details.
extensions
array of objects
Array of extension block objects. Same shape as blocks
.
callbacks
array of objects
Array of callback block objects. Same shape as blocks
.
conditionals
array of objects
Array of conditional block objects. Same shape as blocks
.
curl --request POST
-H 'Content-Type: application/json'
-H 'Cookie: oauth_token=some_really_long_string'
-H 'X-API-KEY: zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2'
-d '{"pipeline": {"name": "arwen-starlight", "type": "python"}}'
--url http://localhost:6789/api/pipelines
{
"pipeline": {
"data_integration": null,
"description": null,
"executor_config": {},
"executor_count": 1,
"executor_type": null,
"name": "arwen-starlight",
"notification_config": {},
"type": "python",
"updated_at": null,
"uuid": "arwen_starlight",
"spark_config": {},
"blocks": [],
"callbacks": [],
"conditionals": [],
"widgets": []
}
}
curl --request POST
-H 'Content-Type: application/json'
-H 'Cookie: oauth_token=some_really_long_string'
-H 'X-API-KEY: zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2'
-d '{"pipeline": {"name": "arwen-starlight", "type": "python"}}'
--url http://localhost:6789/api/pipelines
{
"pipeline": {
"data_integration": null,
"description": null,
"executor_config": {},
"executor_count": 1,
"executor_type": null,
"name": "arwen-starlight",
"notification_config": {},
"type": "python",
"updated_at": null,
"uuid": "arwen_starlight",
"spark_config": {},
"blocks": [],
"callbacks": [],
"conditionals": [],
"widgets": []
}
}