Configure data integration pipelines

​

Interpolation

You can interpolate pipeline variables, environment variables, secrets, and more into the source config file, schema settings, and destination config file.

​

Pipeline variables

{{ variables('variable_name') }}

​

Variable names

You can add your own custom variables to the pipeline. For more information, read the documentation.

Here are the default variables available to interpolate:

​

Common prefixes

Mage supports table prefixes for integration sources. To add a prefix, add the following to the source configuration for a data integration pipeline:

_patterns:
  destination_table: MY_PREFIX_{{ variables('stream') }}

For example, configuring a PostgreSQL source:

​

Environment variables

{{ env_var('environment_variable_name') }}

​

Secrets

Use the following syntax to interpolate secrets:

{{ mage_secret_var('your_secret_name') }}

For more information on secrets, read the documentation.