​

IO Config Setup Guide

​

File Location

your_mage_project/
└── io_config.yaml

​

Creating a New io_config File

​

1. Basic Structure

version: 0.1.1  # Optional but recommended
default:
  # Your default profile configuration here

​

2. Configuration Formats

​

Standard Format (Recommended)

version: 0.1.1
default:
  # PostgreSQL configuration
  POSTGRES_DBNAME: your_database
  POSTGRES_USER: your_username
  POSTGRES_PASSWORD: "{{ env_var('POSTGRES_PASSWORD') }}"
  POSTGRES_HOST: your_host
  POSTGRES_PORT: 5432

  # Snowflake configuration
  SNOWFLAKE_USER: your_username
  SNOWFLAKE_PASSWORD: "{{ env_var('SNOWFLAKE_PASSWORD') }}"
  SNOWFLAKE_ACCOUNT: your_account
  SNOWFLAKE_DEFAULT_WH: your_warehouse
  SNOWFLAKE_DEFAULT_DB: your_database
  SNOWFLAKE_DEFAULT_SCHEMA: your_schema

  # AWS configuration
  AWS_ACCESS_KEY_ID: "{{ env_var('AWS_ACCESS_KEY_ID') }}"
  AWS_SECRET_ACCESS_KEY: "{{ env_var('AWS_SECRET_ACCESS_KEY') }}"
  AWS_REGION: your_region

​

Legacy Format (Verbose)

default:
  PostgreSQL:
    database: your_database
    user: your_username
    password: your_password
    host: your_host
    port: 5432
  AWS:
    region: your_region
    Redshift:
      database: your_redshift_database
      port: 5439

​

3. Using Variables and Secrets

​

Variable Syntax

​

Example Configurations

version: 0.1.1
default:
  POSTGRES_PASSWORD: "{{ env_var('POSTGRES_PASSWORD') }}"
  SNOWFLAKE_PASSWORD: "{{ env_var('SNOWFLAKE_PASSWORD') }}"

version: 0.1.1
default:
  POSTGRES_PASSWORD: "{{ mage_secret_var('postgres_password') }}"
  SNOWFLAKE_PASSWORD: "{{ mage_secret_var('snowflake_password') }}"

version: 0.1.1
default:
  POSTGRES_PASSWORD: "{{ aws_secret_var('prod/postgres/password') }}"
  SNOWFLAKE_PASSWORD: "{{ aws_secret_var('prod/snowflake/password') }}"

version: 0.1.1
default:
  POSTGRES_PASSWORD: "{{ azure_secret_var('postgres-password') }}"
  SNOWFLAKE_PASSWORD: "{{ azure_secret_var('snowflake-password') }}"

version: 0.1.1
default:
  POSTGRES_PASSWORD: "{{ gcp_secret_var('postgres-password') }}"
  SNOWFLAKE_PASSWORD: "{{ gcp_secret_var('snowflake-password') }}"

version: 0.1.1
default:
  POSTGRES_PASSWORD: "{{ doppler_secret_var('POSTGRES_PASSWORD') }}"
  SNOWFLAKE_PASSWORD: "{{ doppler_secret_var('SNOWFLAKE_PASSWORD') }}"

version: 0.1.1
default:
  # Load from local file
  POSTGRES_PASSWORD: "{{ file('secrets/postgres_password.txt') }}"
  # Load JSON field from file
  SNOWFLAKE_PASSWORD: "{{ json_value(file('config/database.json'), 'snowflake_password') }}"

version: 0.1.1
default:
  # Extract specific field from JSON stored in environment variable
  POSTGRES_PASSWORD: "{{ json_value(env_var('DB_CONFIG_JSON'), 'postgres_password') }}"
  # Extract from JSON stored in Mage secret
  SNOWFLAKE_PASSWORD: "{{ json_value(mage_secret_var('database_config'), 'snowflake_password') }}"

​

4. Multiple Profiles

version: 0.1.1
default:
  # Default profile configuration
  POSTGRES_DBNAME: production_db

development:
  # Development environment configuration
  POSTGRES_DBNAME: development_db

staging:
  # Staging environment configuration
  POSTGRES_DBNAME: staging_db

​

Managing profiles in the UI

1. Open Settings.
2. In Workspace, select Data connections.
3. Select an existing profile or choose Add profile. You can also open /settings/workspace/io-config directly.

​

Create and save profiles

• Enter a profile name before choosing a provider or editing YAML. New profiles start with Full profile and an empty YAML mapping.
• Select Save, press Command+S on macOS, or press Ctrl+S on Windows and Linux.
• Profile names can contain up to 128 letters, numbers, periods, underscores, and hyphens. They must start with a letter or number, cannot contain spaces, and cannot be version.
• Mage confirms before deleting a profile or discarding an unsaved name or configuration change.
• If another user changes io_config.yaml, Mage rejects your stale save or delete. Reload before applying the edit again.

​

Edit provider configuration

• The backend supplies the provider list, YAML templates, and connection-test capabilities.
• Checked providers show their configured-value count. Blank template defaults are not saved or counted.
• Saving one provider preserves unrelated provider keys in the same profile. Removing a key removes it only from the selected provider scope.
• Clear provider keys resets that provider’s draft; select Save to persist the removal.
• Provider-scoped editing rejects mixed or custom keys. Use Full profile for those keys.
• Providers can share credential groups, such as AWS or DuckDB keys. Editing or clearing a shared group affects every provider that uses that group but preserves unrelated keys.

​

Work with masked secrets

• Leave the mask unchanged at the same YAML path to preserve the saved value.
• Replace it to update the secret, or remove the key to delete the value.
• Mage rejects a mask when no saved secret exists at that path.
• Lists containing masked secrets must remain unchanged. Replace the masks with explicit values before adding, removing, reordering, or editing list items.

​

YAML limitations and recovery

• For invalid YAML, fix the file and select Retry.
• For a symbolic link, edit the link target directly; the UI can read the file but cannot save it.

​

Test a connection

• Mage renders Jinja expressions such as {{ env_var('POSTGRES_PASSWORD') }} using current runtime values.
• Tests stop after 10 seconds, and only one test can run per project at a time.
• Most tests open and close a connection without previewing, extracting, or writing data. Success confirms only that lightweight check.
• Trino also runs SELECT 1 and requires catalog, host, and user configuration.
• Mage shows specific configuration errors after redacting known secrets. Provider and driver exceptions use the generic connection-failure message.

​

Workflow checklist

1. Add a profile, choose a provider, edit its YAML, test when available, and save.
2. Reload and confirm the profile persists, credentials are masked, and unchanged masks preserve saved values.
3. Save two providers in one profile and confirm each keeps its own configuration.
4. Clear one provider, save, and confirm unrelated provider keys remain.
5. Confirm invalid YAML, mixed provider keys, stale revisions, read-only access, and delete confirmation block unsafe writes.

​

Using io_config in Your Code

​

SQL Block Example

​

Python Block Examples

from mage_ai.settings.repo import get_repo_path
from mage_ai.io.config import ConfigFileLoader
from mage_ai.io.postgres import Postgres
from os import path

@data_loader
def load_data_from_postgres(**kwargs):
    query = 'SELECT * FROM your_table'
    config_path = path.join(get_repo_path(), 'io_config.yaml')
    config_profile = 'default'  # or any other profile name

    with Postgres.with_config(ConfigFileLoader(config_path, config_profile)) as loader:
        return loader.load(query)

from mage_ai.settings.repo import get_repo_path
from mage_ai.io.config import ConfigFileLoader
from mage_ai.io.postgres import Postgres
from os import path

@data_loader
def load_data_from_postgres(**kwargs):
    query = 'SELECT * FROM your_table'
    config_path = path.join(get_repo_path(), 'io_config.yaml')
    
    # Get profile from pipeline variables or use default
    config_profile = kwargs.get('profile', 'default')
    
    with Postgres.with_config(ConfigFileLoader(config_path, config_profile)) as loader:
        return loader.load(query)

import os
from mage_ai.settings.repo import get_repo_path
from mage_ai.io.config import ConfigFileLoader
from mage_ai.io.postgres import Postgres
from os import path

@data_loader
def load_data(**kwargs):
    query = 'SELECT * FROM your_table'
    config_path = path.join(get_repo_path(), 'io_config.yaml')

    # Automatically select profile based on environment
    environment = os.getenv('ENV', 'development')
    config_profile = environment  # 'development', 'staging', 'production'

    with Postgres.with_config(ConfigFileLoader(config_path, config_profile)) as loader:
        return loader.load(query)

​

Databases using io_config.yaml

• PostgreSQL
• MySQL
• Microsoft SQL Server
• SQLite
• Redshift
• Trino
• OracleDB
• ClickHouse
• DuckDB
• MotherDuck

• BigQuery
• Snowflake
• Databricks
• Athena
• Microsoft Fabric Warehouse

• S3
• Google Cloud Storage
• Azure Blob Storage
• Azure Data Lake Storage Gen2
• Iceberg

• MongoDB
• Druid
• Pinot
• Google Sheets
• Spark
• Power BI

• Chroma
• Qdrant
• Weaviate
• Algolia

​

Best Practices

1. Security:
   • Never commit sensitive credentials directly in the io_config file
   • Use environment variables or secret variables for sensitive information
   • Consider using a secrets management service for production environments
2. Organization:
   • Use meaningful profile names
   • Group related configurations together
   • Document any non-standard configurations
   • Use the standard format for new configurations
3. Version Control:
   • Add io_config.yaml to your .gitignore file
   • Provide a template file (io_config.yaml.template) with dummy values
   • Document the required environment variables

​

Troubleshooting

​

Common Issues & Solutions

• Make sure your io_config.yaml file exists in your project root (e.g., /home/src/your_mage_project/io_config.yaml).
• If running in a different environment (e.g., Docker), confirm the file is mounted and accessible.

• Double-check that the profile name you are referencing in your code (e.g., 'default', 'staging') exists in your io_config.yaml.
• Profile names are case-sensitive.

• Ensure all required fields for your data source/destination are present in the selected profile.

• If you use {{ env_var('VAR_NAME') }} in your config, make sure the environment variable is set in deployment environment.
• You can check this by running echo $VAR_NAME in Mage terminal.

• Use a YAML linter or validator to check for indentation or formatting issues.
• Strings containing special characters (like : or {}) should be quoted.

• Double-check your credentials and permissions.
• For cloud services, ensure your service account or IAM user has the necessary roles.

• Make sure you are importing from mage_ai.io.config (not mage_ai.io.io_config).
• Example:
  from mage_ai.io.config import ConfigFileLoader
  

• In the Mage UI, select the correct profile from the “Profile” dropdown before running your SQL block.

​

Additional Resources

• Variables Overview
• Data Integration Guide
• Pipeline Configuration