Overview

Use Amazon Redshift as a destination in Mage to export pipeline data into a highly scalable, cloud-based data warehouse. Mage supports both standard username/password authentication and IAM database authentication using AWS access keys.

Mage will automatically create and write to the specified table, with support for optional schema management and merge-based updates.


Configuration Parameters

You must provide the following credentials when configuring Redshift as a destination:

KeyDescriptionExample ValueRequired
databaseName of the Redshift database where data will be written.demo
hostHostname of the Redshift cluster endpoint.mage-prod.3.us-west-2.redshift.amazonaws.com
portRedshift port (default is 5439).5439
userRedshift username with access to the database and schema.awsuser✅ (if not using IAM)
passwordPassword for the Redshift user.abc123...✅ (if not using IAM)
schemaSchema within the database where the table will be created or written to.public
tableName of the destination table to store data from your pipeline.dim_users_v1
regionAWS region where the Redshift cluster is hosted.us-west-2
use_merge_loadIf true, Mage will attempt to merge incoming data with existing records (UPSERT-like behavior).false (default)

IAM-Based Authentication (Alternative to User/Password)

If you prefer to use IAM credentials instead of traditional authentication, you can use the following fields:

KeyDescriptionExample ValueRequired (if using IAM)
access_key_idAWS access key ID for an IAM user or role with Redshift database access.AKIA...
secret_access_keyAWS secret access key corresponding to the above access key ID.xyz123...
cluster_identifierIdentifier of the Amazon Redshift cluster.mage-prod
db_userIAM-authenticated Redshift user.admin

Optional Parameters

KeyDescriptionExample Value
skip_schema_creationIf true, Mage will skip running the CREATE SCHEMA command. Useful if the schema already exists. See issuetrue
lower_caseIf true, Mage will automatically convert column names to lowercase. Default is true.true

Grant Permissions

To allow Mage to create schemas and insert data, grant the following permissions to your Redshift user or IAM role:

GRANT CREATE ON DATABASE your_database TO your_user;
GRANT CREATE ON ALL TABLES IN SCHEMA your_schema TO your_user;

Refer to the Redshift GRANT command documentation for details.