Pipeline runs
Pipeline run object
{
"backfill_id": null,
"block_runs": [
{
"block_uuid": "load_titanic",
"completed_at": "2023-02-16 06:40:49.769087+00:00",
"created_at": "2023-02-16 06:40:43.575738+00:00",
"id": 33,
"metrics": null,
"pipeline_run_id": 23,
"started_at": "2023-02-16 06:40:48.816197+00:00",
"status": "completed",
"updated_at": "2023-02-16 06:40:48.873526+00:00"
}
],
"block_runs_count": 4,
"completed_at": "2023-02-16 06:41:19.249956+00:00",
"created_at": "2023-02-16 06:40:38.681910+00:00",
"event_variables": null,
"execution_date": "2023-02-16 04:41:03.300124+00:00",
"id": 23,
"metrics": null,
"passed_sla": false,
"pipeline_schedule_id": 1,
"pipeline_schedule_name": "late forest",
"pipeline_schedule_token": "260385a38d62466fb1c88fa8c5ca503f",
"pipeline_schedule_type": "time",
"pipeline_uuid": "example_pipeline",
"status": "completed",
"updated_at": "2023-02-16 06:41:09.154102+00:00",
"variables": null
}
Backfill that this run belongs to.
The block runs for this specific pipeline run.
Number of block runs for this pipeline run.
Date and time the pipeline run finished running successfully.
Date and time the pipeline run was created.
When pipeline run is triggered from an API request, the API request payload is saved in the
pipeline run’s event_variables
column.
Date and time the pipeline run started running.
ID of pipeline run.
Various metrics related to the pipeline run; e.g. number of rows processed in a data integration pipeline sync.
Whether or not the pipeline run has passed the configured SLA.
Pipeline schedule ID that this pipeline run belongs to.
Name of the pipeline schedule this pipeline run belongs to.
Unique token of the pipeline schedule that this pipeline run belongs to.
Pipeline schedule type that this pipeline run belongs to.
Pipeline UUID that this pipeline run belongs to.
initial
, running
, completed
, failed
, cancelled
Date and time the pipeline run record was most recently updated.
Object containing variables for the pipeline that is being overwritten by this pipeline run.
The property name is user defined.
Read all pipeline runs
GET /api/pipeline_schedules/:pipeline_schedule_id/pipeline_runs
Pipeline schedule ID that pipeline runs should all belong to.
Maximum number of logs to be returned in the API response.
Example: 20
Read logs after N number of logs, where N equals _offset
.
Example: 10
curl --request GET \
--url 'http://localhost:6789/api/pipeline_schedules/1/pipeline_runs?api_key=zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2' \
--header 'Authorization: Bearer ZKXScwLKLZUujbw7vwMeWnhtTTSLqifw8_RnBHYipww' \
{
"pipeline_runs": [
...
],
"metadata": {
"count": 22,
"next": false
}
}
Trigger pipeline
Run a pipeline using an API request. For more information, read this guide.
POST /api/pipeline_schedules/:pipeline_schedule_id/pipeline_runs/:token
Pipeline schedule ID that pipeline runs should all belong to.
Unique token of the pipeline schedule that you want to trigger.
curl --request POST \
--url 'http://localhost:6789/api/pipeline_schedules/71/pipeline_runs/4506463a01944ef492f70005996dac1a' \
--data '{
"env": "dev",
"test": 1
}'
{
"pipeline_run": {
"id": 1249,
"created_at": "2023-04-01 00:36:23.507282+00:00",
"updated_at": "2023-04-01 00:36:39.962084+00:00",
"pipeline_schedule_id": 71,
"pipeline_uuid": "example_pipeline",
"execution_date": "2023-04-01 00:36:39.904371+00:00",
"status": "running",
"completed_at": null,
"variables": {},
"passed_sla": false,
"event_variables": {
"env": "dev",
"test": 1
},
"metrics": null,
"backfill_id": null
}
}