How to configure Amazon Redshift as a destination in Mage to write data into Redshift tables using user credentials or IAM-based authentication.
Key | Description | Example Value | Required |
---|---|---|---|
database | Name of the Redshift database where data will be written. | demo | ✅ |
host | Hostname of the Redshift cluster endpoint. | mage-prod.3.us-west-2.redshift.amazonaws.com | ✅ |
port | Redshift port (default is 5439 ). | 5439 | ✅ |
user | Redshift username with access to the database and schema. | awsuser | ✅ (if not using IAM) |
password | Password for the Redshift user. | abc123... | ✅ (if not using IAM) |
schema | Schema within the database where the table will be created or written to. | public | ✅ |
table | Name of the destination table to store data from your pipeline. | dim_users_v1 | ✅ |
region | AWS region where the Redshift cluster is hosted. | us-west-2 | ✅ |
use_merge_load | If true , Mage will attempt to merge incoming data with existing records (UPSERT-like behavior). | false (default) | ❌ |
use_s3_copy | If true , Mage will upload data to S3 and load into Redshift using the COPY command (Pro feature). | false (default) | ❌ |
Note: use_s3_copy
is a Mage Pro-only feature. It enables faster bulk loads via S3 and supports IAM roles or access key authentication.
Key | Description | Example Value | Required (if using IAM) |
---|---|---|---|
access_key_id | AWS access key ID for an IAM user or role with Redshift database access. | AKIA... | ✅ |
secret_access_key | AWS secret access key corresponding to the above access key ID. | xyz123... | ✅ |
cluster_identifier | Identifier of the Amazon Redshift cluster. | mage-prod | ✅ |
db_user | IAM-authenticated Redshift user. | admin | ✅ |
use_s3_copy
is true)Key | Description | Required | Sample Value |
---|---|---|---|
aws_access_key_id | AWS access key for uploading data to S3. Ignored if using IAM role. | ✅ (unless using IAM role) | AKIA... |
aws_secret_access_key | AWS secret key for uploading data to S3. Ignored if using IAM role. | ✅ (unless using IAM role) | xyz... |
aws_region | Region where the S3 bucket is located. | ✅ | us-west-2 |
bucket | S3 bucket to temporarily stage Parquet files for Redshift COPY. | ✅ | mage-feature-sets |
redshift_iam_role | IAM role ARN with S3 read access attached to your Redshift cluster. | Optional | arn:aws:iam::123456789012:role/MyRedshiftRole |
s3_creds
config object:
Key | Description | Example Value |
---|---|---|
skip_schema_creation | If true , Mage will skip running the CREATE SCHEMA command. Useful if the schema already exists. See issue | true |
lower_case | If true , Mage will automatically convert column names to lowercase. Default is true . | true |