Create pipeline

  curl --request POST 
  -H 'Content-Type: application/json' 
  -H 'Cookie: oauth_token=some_really_long_string'
  -H 'X-API-KEY: zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2'
  -d '{"pipeline": {"name": "arwen-starlight", "type": "python"}}' 
  --url http://localhost:6789/api/pipelines

{
  "pipeline": {
    "data_integration": null,
    "description": null,
    "executor_config": {},
    "executor_count": 1,
    "executor_type": null,
    "name": "arwen-starlight",
    "notification_config": {},
    "type": "python",
    "updated_at": null,
    "uuid": "arwen_starlight",
    "spark_config": {},
    "blocks": [],
    "callbacks": [],
    "pipeline_callbacks": [],
    "conditionals": [],
    "widgets": []
  }
}

POST /api/pipelines

string

required

Human readable name of the pipeline.

string

required

The type of the pipeline: integration, pyspark, python, streaming Note that python is a standard (batch) pipeline with a python backend, while pyspark is a batch pipeline with a spark backend.

array of strings

Non-empty tag names to associate with the pipeline. Tag names are trimmed and duplicate names are ignored before persistence. Each tag name cannot exceed 70 characters.

string

If supplied, the uuid of a pipeline to clone.

array of objects

Array of extension block objects. Same shape as blocks.

array of objects

Array of callback block objects. Same shape as blocks.

array of objects

Mage Pro only. Pipeline-level callback configuration for callback blocks that should run once after the pipeline run reaches a terminal status. Callback decorators inside the callback block determine the behavior for success, failure, and cancellation.

Show properties

string

required

UUID of a callback block in the pipeline.

boolean

If false, the pipeline-level callback is saved but not dispatched.

array of objects

Array of conditional block objects. Same shape as blocks.

  curl --request POST 
  -H 'Content-Type: application/json' 
  -H 'Cookie: oauth_token=some_really_long_string'
  -H 'X-API-KEY: zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2'
  -d '{"pipeline": {"name": "arwen-starlight", "type": "python"}}' 
  --url http://localhost:6789/api/pipelines

{
  "pipeline": {
    "data_integration": null,
    "description": null,
    "executor_config": {},
    "executor_count": 1,
    "executor_type": null,
    "name": "arwen-starlight",
    "notification_config": {},
    "type": "python",
    "updated_at": null,
    "uuid": "arwen_starlight",
    "spark_config": {},
    "blocks": [],
    "callbacks": [],
    "pipeline_callbacks": [],
    "conditionals": [],
    "widgets": []
  }
}

  curl --request POST 
  -H 'Content-Type: application/json' 
  -H 'Cookie: oauth_token=some_really_long_string'
  -H 'X-API-KEY: zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2'
  -d '{"pipeline": {"name": "arwen-starlight", "type": "python"}}' 
  --url http://localhost:6789/api/pipelines

{
  "pipeline": {
    "data_integration": null,
    "description": null,
    "executor_config": {},
    "executor_count": 1,
    "executor_type": null,
    "name": "arwen-starlight",
    "notification_config": {},
    "type": "python",
    "updated_at": null,
    "uuid": "arwen_starlight",
    "spark_config": {},
    "blocks": [],
    "callbacks": [],
    "pipeline_callbacks": [],
    "conditionals": [],
    "widgets": []
  }
}