A callback block is associated to another block. When the parent block succeeds or fails, the callback block functions are executed.
0.8.61
or greater.load_data_from_api
(data loader)clean_column_names
(transformer)save_data
(data exporter)load_data_from_api
(data loader)clean_column_names
(transformer)load_data_from_api
(data loader) block first.
It successfully completes. Then, the callback block will run and execute the function hello_world
.
clean_column_names
(transformer) block.
It fails due to some error. The callback block will run and execute the function alert_me
.
@callback
.
You can have as many functions in a callback block as you want.@callback
or @callback('success')
to decorate a function that should only run when the
parent block successfully completes.@callback('failure')
to decorate a function that should only run when the
parent block fails.success
failure
>= 0.8.81
.Name | Description | Sample value |
---|---|---|
block_uuid | Callback block UUID. | 'fireball_callback' |
parent_block_uuid | The block UUID of the parent block. | 'parent_block' |
ds | Date string when the parent block started executing. | '2023-12-25' |
event | A dictionary containing metadata from an event triggered pipeline. | {} |
execution_date | Python datetime object for when the parent block started executing. | datetime.datetime(2023, 4, 26, 20, 28, 17, 335254, tzinfo=datetime.timezone.utc) |
execution_partition | Partition used for the parent block when it was executed. | '207/20230426T202817' |
hr | Hour string when the parent block started executing. | '20' |
pipeline_run | Python pipeline run object associated to the current run. | PipelineRun(id=2357, pipeline_uuid=fire_etl, execution_date=2023-04-26 20:28:17.335254+00:00) |
pipeline_uuid | UUID of the current pipeline. | 'fire_etl' |
__input | For version >= 0.8.81 . See below for more details. | { 'some_block_uuid': {} } |
__input
keyword argument>= 0.8.81
.__input
in the callback function’s keyword arguments.
The value of the __input
key is a dictionary. The keys in that dictionary are named after the
callback block’s parent block’s UUID.
For example, if the callback block’s parent block’s UUID is load_api_data
, then this is how
you would access load_api_data
’s data output:
Name | Description | Sample value |
---|---|---|
destination_table | The table name that the source data is being exported to. | 'dim_users_v1' |
index | The order in which the parent block executed compared to other blocks of the same type. | 0 |
is_last_block_run | Boolean value for whether or not the parent block was the last block ran for the block type in the pipeline. | True or False |
stream | The name of the source stream (or table) that the parent block is syncing for. | 'stripe_transactions' |
< 0.8.61
.on_success
callback and/or a on_failure
callback. These callbacks
will run after your block run completes or fails. When you run your block in the
pipeline edit page, it will also run the callback depending on the status of the block.kwargs.get('<variable>')
syntax in your callback.