Pipeline runs
Retrying block runs from a pipeline run
Requires version
0.8.76
or greater.Automatic retry
Mage supports configuring automatic retry for block runs with the following ways
- Add
retry_config
to project’smetadata.yaml
. Thisretry_config
will be applied to all block runs. Here is an example:retry_config: # Number of retry times retries: 3 # Initial delay (in seconds) before retry. If exponential_backoff is true, # the delay time is multiplied by 2 for the next retry delay: 5 # Maximum time between the first attempt and the last retry max_delay: 60 # Whether to use exponential backoff retry exponential_backoff: true
- Add
retry_config
to the block config in pipeline’smetadata.yaml
. The block levelretry_config
will override the globalretry_config
. Here is an exampleblocks: - uuid: example_data_loader type: data_loader upstream_blocks: [] downstream_blocks: [] retry_config: retries: 2 delay: 5 max_delay: 10 exponential_backoff: false ...
The retry logs show in the same block run log page.
Retry incomplete blocks
- Supported pipeline types: Integration and Standard pipelines
- If a pipeline run fails, you can retry from the failed block
run (instead of retrying the entire pipeline run) by going to the individual
pipeline runs page (
/pipelines/[pipeline_uuid]/runs/[pipeline_run_id]
). - At the top of the individual pipeline runs page, there should be a red
Retry incomplete blocks
button. Click on it, and the block runs should retry from the failed block.
Retry from selected block
- Supported pipeline types: Integration and Standard pipelines
- If you want to retry block runs for a pipeline run starting from a specific block (regardless of the block run’s status), go to the same page for the individual pipeline run.
- Select a block run row, and a blue
Retry from selected block
button should appear. - Note that the
Retry from selected block
button will only appear if the pipeline run is not currently running. - If there is a failed block run and you try to retry from a selected block that is
downstream from that failed block, you may get
upstream_failed
block run errors.
Accessing the individual pipeline runs page
- Click on a pipeline from the main Pipelines Dashboard.
- Click on the
Runs
section or on an individual trigger from theTriggers
section. - Underneath the
Block runs
column of the pipeline runs table, there is a link that says something likeSee block runs (x)
. Click on that link to redirect to the individual pipeline runs page, which lists the blocks runs specific to a pipeline run.
Was this page helpful?