Mage home pagelight logo
Routes
Backfills

​
Backfill object

{
  "block_uuid": null,
  "completed_at": null,
  "created_at": "2023-03-23 18:27:58.848959+00:00",
  "end_datetime": "2023-03-05 00:00:00+00:00",
  "failed_at": null,
  "id": 9,
  "interval_type": "hour",
  "interval_units": 4,
  "metrics": null,
  "name": "sparkling glitter",
  "pipeline_schedule_id": null,
  "pipeline_uuid": "example_pipeline",
  "start_datetime": "2023-03-01 00:00:00+00:00",
  "started_at": null,
  "status": null,
  "total_run_count": 25,
  "updated_at": "2023-03-23 18:28:22.379725+00:00",
  "variables": null
}
block_uuid
string

If the backfill uses custom code to generate the pipeline runs, this block UUID contains the code that will be used.

completed_at
datetime

Date and time the backfill finished running successfully.

created_atrequired
datetime

Date and time the record was created.

end_datetime
datetime

The date and time the backfill should stop on.

failed_at
datetime

Date and time the backfill failed.

idrequired
integer

Unique ID for the record.

interval_type
string

second, minute, hour, day, week, month, year, custom

interval_units
integer

The number of interval_type to use in between each pipeline run that the backfill creates.

For example, if interval_type is day and interval_units is 3, then the backfill will create pipeline runs spanning 3 day increments between the start_datetime and end_datetime.

metrics
object

Various metrics related to the pipeline run; e.g. number of rows processed in a data integration pipeline sync.

namerequired
string

Name of the record.

pipeline_scheduled_id
integer

Pipeline schedule that this record belongs to.

pipeline_uuidrequired
string

Pipeline that this record belongs to.

start_datetime
datetime

The date and time the backfill should start on.

started_at
datetime

Date and time the backfill started running.

statusrequired
string

initial, running, completed, failed, cancelled

total_run_count
integer

Number of pipeline runs that will be created for this backfill.

updated_atrequired
datetime

Date and time the record was last updated.

variables
object

Object containing variables for the pipeline that is being overwritten by this backfill.

[key]
string

The property name is user defined.

​
Read all backfills

GET /api/backfills

_limit
integer

Maximum number of logs to be returned in the API response.

Example: 20

_offset
integer

Read logs after N number of logs, where N equals _offset.

Example: 10

curl --request GET \
  --url 'http://localhost:6789/api/backfills?_limit=20&_offset=0&include_run_count=true&pipeline_uuid=example_pipeline&api_key=zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2' \
  --header 'Authorization: Bearer ZKXScwLKLZUujbw7vwMeWnhtTTSLqifw8_RnBHYipww' \

{
  "backfills": [
    ...
  ],
  "metadata": {
    "count": 3,
    "next": false
  }
}

​
Create backfill

POST /api/pipelines/:pipeline_uuid/backfills

pipeline_uuidrequired
string

Pipeline UUID to create the backfill for.

Body

backfillrequired
object
namerequired
string
curl --request PUT \
  --url 'http://localhost:6789/api/pipelines/example_pipeline/backfills?api_key=zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2' \
  --header 'Authorization: Bearer ZKXScwLKLZUujbw7vwMeWnhtTTSLqifw8_RnBHYipww' \
  --data '{
    "backfill": {
      "name": "fire wind"
    }
  }'

{
  "backfill": {
    "id": 10,
    "block_uuid": null,
    "completed_at": null,
    "created_at": "2023-04-01 01:02:18.411997+00:00",
    "end_datetime": null,
    "failed_at": null,
    "interval_type": null,
    "interval_units": null,
    "metrics": null,
    "name": "fire wind",
    "pipeline_schedule_id": null,
    "pipeline_uuid": "example_pipeline",
    "start_datetime": null,
    "started_at": null,
    "status": null,
    "updated_at": "2023-04-01 01:02:18.411997+00:00",
    "variables": null
  }
}

​
Read backfill

WIP

​
Update backfill

WIP

​
Delete backfill

WIP