Extension block

Extension blocks are part of your pipeline but don’t run as individual steps in your pipeline like data loader blocks, transformer blocks, etc. However, extension blocks are associated to other blocks within the pipeline (e.g. data loaders, transformers, data exporters, etc).

After those other blocks are executed, any associated extension blocks will also be executed. Those extension blocks will perform some function on its associated blocks.

​

Example

1. Your pipeline has the following blocks:

   1. load_data_from_api (data loader)
   2. clean_column_names (transformer)
   3. save_data (data exporter)

2. Then, you add the following extension block to your pipeline:

   @extension('...')
   def some_powerful_function(*args, **kwargs):
       # some code...
   

3. You associate the above extension block to the following 2 blocks:

   • load_data_from_api (data loader)
   • clean_column_names (transformer)

4. When you run the pipeline, it’ll execute the load_data_from_api (data loader) block first. Once that is complete, the associated extension block will run.

5. Next, the pipeline will execute the clean_column_names (transformer) block. After that is done, the associated extension block will run.

6. Finally, the save_data (data exporter) block will run. Since it has no associated extension block, the pipeline will finish.

​

How to add extensions to your pipeline

1. Create a new pipeline or open an existing pipeline.
2. Edit the pipeline.
3. On the right side of the page, click the Power ups icon in the navigation. If you don’t see it, try expanding the right area of the page.
4. You’ll be shown a list of power ups you can add to your pipeline.
5. Click on a power up to configure it and add extension blocks to your pipeline.

​

Supported extensions

Here are the currently supported extensions:

• great_expectations

More coming soon!