AWS Secrets Manager

​

Creating a New Secret in AWS Secrets Manager

Before you can use AWS Secrets Manager with Mage, you need to create a new secret in the AWS Secrets Manager console. Follow these steps:

1. Open the AWS Secrets Manager console
2. Click “Store a new secret.”
3. On the “Choose Secret Type” page, select the appropriate secret type for your use case, and then click “Next.”
4. On the “Configure Secret” page, provide the necessary details for your secret, such as the key-value pairs or the plaintext value, and then click “Next.”
5. (Optional) On the “Configure Rotation” page, you can enable automatic rotation for your secret. Automatic rotation periodically updates the secret value, ensuring that your sensitive data remains secure. Configure the rotation settings according to your requirements, and then click “Next.”
6. Review the secret details on the “Review” page, and if everything looks correct, click “Store.”

Your new secret is now stored in AWS Secrets Manager. Make a note of the secret’s ARN (Amazon Resource Name), as you’ll need it to access the secret from Mage.

For more detailed information on managing and storing secrets in AWS Secrets Manager, refer to the official AWS Documentation

​

Working with Secrets in Mage

Mage works with AWS secrets in two ways: directly accessing them through Python code, and referencing them in YAML configuration files using placeholders that Mage resolves at runtime.

Python:

To use secrets in Python code, read these instructions.

YAML:

"{{ aws_secret_var('some_name_for_secret') }}"

api_key: "{{ aws_secret_var('API_KEY') }}"
access_token: "{{ aws_secret_var('ACCESS_TOKEN') }}"

​

Working with AWS Secrets Manager

If you need a more secure environment for handling sensitive data, you can directly integrate your YAML files with AWS Secrets Manager. For example, to reference a secret stored in AWS Secrets Manager with the ARN, you would use the following syntax in your YAML file:

AWS_SECRET_ACCESS_KEY = arn:aws:secretsmanager:<AWS Region>:<AWS Account ID>:secret:<AWS_SECRET_ACCESS_KEY Name>

By integrating with AWS Secrets Manager, you can ensure that your sensitive data is securely stored and accessed throughout your Mage data pipelines.

1. AWS_SECRET_ACCESS_KEY: The secret value you want get from AWS Secrets Manager
2. arn: This is a prefix that identifies the resource as an ARN.
3. aws: This is the partition that the resource is in. In this case, it’s the AWS partition.
4. secretsmanager: This is the service namespace, which identifies the AWS service. In this case, it’s the AWS Secrets Manager service.
5. AWS Region: This is the AWS region where the secret is stored (i.e. us-west-2).
6. AWS Account ID: This is the 12-digit AWS account ID that the secret belongs to.
7. secret This is the resource type, which indicates that the resource is a secret.
8. AWS_SECRET_ACCESS_KEY Name: This is the friendly name or ID of the specific secret you want to reference. It can include alphanumeric characters, hyphens, and underscores.

In the context of the YAML file, this ARN is being used with the syntax to reference the secret value stored in AWS Secrets Manager. The YAML file itself does not contain the actual secret value; instead, it references the secret by its ARN.