Skip to main content

​
Location of pipeline configuration

Each pipeline in Mage has a metadata.yaml configuration file located in the pipeline uuid’s folder under the pipelines directory. For example, if you had a pipeline with uuid charismatic_inventor, the folder structure for that pipeline might look something like this:

​
Override pipeline config based on environment

In the pipeline’s metadata.yaml config file, add an overrides key at the top-level (no indentations) with the name of your environment (e.g. prod, dev, test) under the overrides key and indented once. Then under the environment name key, add the properties of your base pipeline config that you want to override. Make sure the indentations of the properties match those of the base config. Any environment-specific overrides will REPLACE the matching property in the base config, so be careful when overriding properties with nested values.
The environment name should match the environment defined in the ENV environment variable.

​
Example pipeline config file with environment overrides

In the example above when in the dev environment, the pipeline’s configuration property of concurrency_config will have its pipeline_run_limit and pipeline_run_limit_all_triggers properties overridden to be 1 (instead of 5 as defined in the base config). The description property will also be overridden to be Overridden description for charismatic inventor in dev instead of null. Similarly for the prod environment, pipeline_run_limit will be replaced with 10 and pipeline_run_limit_all_triggers with 20. The description property will be overridden to be Overridden description for charismatic inventor in prod. Other environments (not dev or prod) will not utilize the overrides section.