Retrying block runs from a pipeline run

​

Automatic retry

Mage supports configuring automatic retry for block runs with the following ways

​

1. Global retry config

Add retry_config to project’s metadata.yaml. This retry_config will be applied to all block runs. Here is an example:

retry_config:
  # Number of retry times
  retries: 3
  # Initial delay (in seconds) before retry. If exponential_backoff is true,
  # the delay time is multiplied by 2 for the next retry
  delay: 5
  # Maximum time between the first attempt and the last retry
  max_delay: 60
  # Whether to use exponential backoff retry
  exponential_backoff: true

​

2. Block level retry config

Add retry_config to the block config in pipeline’s metadata.yaml. The block level retry_config will override the global retry_config. Here is an example

blocks:
- uuid: example_data_loader
  type: data_loader
  upstream_blocks: []
  downstream_blocks: []
  retry_config:
    retries: 2
    delay: 5
    max_delay: 10
    exponential_backoff: false
  ...

​

3. Pipelne level retry_config

Add retry_config to the pipeline config in pipeline’s metadata.yaml. The pipeline level retry_config will override the global retry_config. Here is an example

blocks:
- ...
- ...
- ...
retry_config:
  retries: 2
  delay: 5
  max_delay: 10
  exponential_backoff: false
  ...

The retry logs show in the same block run log page.

​

Retry incomplete blocks

• Supported pipeline types: Integration and Standard pipelines
• If a pipeline run fails, you can retry from the failed block run (instead of retrying the entire pipeline run) by going to the individual pipeline runs page (/pipelines/[pipeline_uuid]/runs/[pipeline_run_id]).
• At the top of the individual pipeline runs page, there should be a red Retry incomplete blocks button. Click on it, and the block runs should retry from the failed block.

​

Retry from selected block

• Supported pipeline types: Integration and Standard pipelines
• If you want to retry block runs for a pipeline run starting from a specific block (regardless of the block run’s status), go to the same page for the individual pipeline run.
• Select a block run row, and a blue Retry from selected block button should appear.
• Note that the Retry from selected block button will only appear if the pipeline run is not currently running.
• If there is a failed block run and you try to retry from a selected block that is downstream from that failed block, you may get upstream_failed block run errors.

​

Accessing the individual pipeline runs page

1. Click on a pipeline from the main Pipelines Dashboard.
2. Click on the Runs section or on an individual trigger from the Triggers section.
3. Underneath the Block runs column of the pipeline runs table, there is a link that says something like See block runs (x). Click on that link to redirect to the individual pipeline runs page, which lists the blocks runs specific to a pipeline run.