Configuration Parameters

To use Delta Lake on AWS S3 as a destination in Mage, provide the following parameters:

KeyDescriptionExample ValueRequired
aws_access_key_idYour AWS access key ID. Required to authenticate with S3.AKIAIOSFODNN7EXAMPLE
aws_secret_access_keyYour AWS secret access key. Used for writing to the S3 bucket.wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws_regionAWS region where your S3 bucket is located.us-west-2 (default)
bucketName of the S3 bucket where the Delta table is stored.mage-spark-data
tableLogical name of the Delta table that Mage will write to.dim_profiles_v1
modeWrite mode:
- append: Adds rows to the table. Fails if schema changes.
- overwrite: Replaces all existing rows and recreates the table.
append (default) or overwrite
object_key_pathPath inside the S3 bucket where the Delta table is written. Exclude s3://, bucket name, and table name from this path.users/ds/20250708

Notes

  • The object key path is relative to the S3 bucket and should not include the scheme (s3://), bucket name, or table name.
  • Mage uses the delta-rs engine for writing data to Delta Lake on S3.
  • Use overwrite mode with caution: it deletes all existing rows and recreates the table.