Configure data integrations

​
Interpolation

You can interpolate pipeline variables, environment variables, secrets, and more into the source config file, schema settings, and destination config file.

​
Pipeline variables

{{ variables('variable_name') }}

​
Variable names

You can add your own custom variables to the pipeline. For more information, read the documentation. Here are the default variables available to interpolate:
NameDescriptionSample value
pipeline.nameName of the current pipeline.ETL demo
pipeline.uuidUUID of the current pipeline.etl_demo
dsExecution date string.2023-12-25
hrExecution hour string from 00 to 23.12
envEnvironment Mage is running in.dev, prod
execution_partitionThe current partition of the execution: datetime in this format %Y%m%dT%H%M%S.20231225T122520

​
Common prefixes

Mage supports table prefixes for integration sources. To add a prefix, add the following to the source configuration for a data integration pipeline: 
_patterns:
  destination_table: MY_PREFIX_{{ variables('stream') }}
For example, configuring a PostgreSQL source:
Configure data integrations

​
Environment variables

{{ env_var('environment_variable_name') }}

​
Secrets

Use the following syntax to interpolate secrets: 
{{ mage_secret_var('your_secret_name') }}
For more information on secrets, read the documentation.