> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mage.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Deploy to AWS with Terraform
## Pre-requisites
### Setup
[Download Mage maintained Terraform scripts](/production/deploying-to-cloud/using-terraform).
### Permissions
In order to create all the required resources in AWS, your IAM user account must
have the following
[permissions](/production/deploying-to-cloud/aws/terraform-apply-policy).
In order to terminate and remove all the resources created from Terraform, you
need the following
[permissions](/production/deploying-to-cloud/aws/terraform-destroy-policy).
***
## Deploy to ECS cluster
Use Terraform scripts in [aws](https://github.com/mage-ai/mage-ai-terraform-templates/tree/master/aws)
folder to provision resources for AWS ECS cluster.
***
### Terraform plan
You can run the following command to see all the resources that will be created
by Terraform:
```bash theme={"system"}
cd aws
terraform plan
```
By default, here are the **[resources](/production/deploying-to-cloud/aws/resources)** that will be
created.
***
### 1. Environment variables
If you don’t have the AWS CLI installed, you’ll need to create this file:
`~/.aws/credentials`.
In that file, add the following values:
```
[default]
aws_access_key_id = XXX
aws_secret_access_key = XXX
```
Alternatively, you can install the AWS CLI by following
[Amazon’s instructions](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).
***
### 2. Customize Terraform settings
**Application name (optional)**
In the file
[./aws/variables.tf](https://github.com/mage-ai/mage-ai-terraform-templates/blob/master/aws/variables.tf),
you can change the default application name that will appear in AWS ECS:
```
variable "app_name" {
type = string
description = "Application Name"
default = "mage-data-prep"
}
```
**Docker image (optional)**
In the file
[./aws/variables.tf](https://github.com/mage-ai/mage-ai-terraform-templates/blob/master/aws/variables.tf),
you can change the default Docker image:
```
variable "docker_image" {
description = "Docker image url used in ECS task."
default = "mageai/mageai:latest"
}
```
**Custom Docker images**
If you previously tagged a Docker image you built when following this
[CI/CD guide](/production/ci-cd), you must push that locally tagged Docker
image to Amazon Elastic Container Registry (ECR) before deploying using
Terraform. Read the
[AWS documentation](https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html)
to learn more.
1. `docker build --platform linux/amd64 -tag [image_name]:latest .`
2. `docker tag [image_name]:latest [registry_uuid].dkr.ecr.[region].amazonaws.com/[image_name]:latest`
3. `docker push [registry_uuid].dkr.ecr.[region].amazonaws.com/[image_name]:latest`
**Region (optional)**
In the file
[./aws/variables.tf](https://github.com/mage-ai/mage-ai-terraform-templates/blob/master/aws/variables.tf),
you can change the region:
```
variable "aws_region" {
type = string
description = "AWS Region"
default = "us-west-2"
}
```
**Availability zones (optional)**
In the file
[./aws/variables.tf](https://github.com/mage-ai/mage-ai-terraform-templates/blob/master/aws/variables.tf),
you must change the availability zones to match your region from above:
```
variable "availability_zones" {
description = "List of availability zones"
default = ["us-west-2a", "us-west-2b"]
}
```
**More**
Other variables defined in
[./aws/variables.tf](https://github.com/mage-ai/mage-ai-terraform-templates/blob/master/aws/variables.tf)
can also be customized to your needs.
***
### 3. Configurable environment variables
In the
[`./aws/env_vars.json`](https://github.com/mage-ai/mage-ai-terraform-templates/blob/master/aws/env_vars.json)
file, you can edit the following variables, which are used by the tool while
running in the cloud:
Change the value of the variables with the following names to match the actual
values you want the tool to use while running in the cloud:
* `AWS_ACCESS_KEY_ID`
* `AWS_SECRET_ACCESS_KEY`
These variable values are used by the tool to retrieve AWS resources like
CloudWatch events, etc.
**Other environment variables**
You can add any environment variable you want in this file. These will be set on
the running container.
> Values
>
> The values in your `env_vars.json` must be strings. Wrap each value in double
> quotes.
***
### 4. Deploy
**Using CLI**
1. Change directory into scripts folder:
```bash theme={"system"}
cd aws
```
2. Initialize Terraform:
```bash theme={"system"}
terraform init
```
* If you run into errors like the following:
```
│ Error: Failed to install provider
│
│ Error while installing hashicorp/template v2.2.0: the local package for registry.terraform.io/hashicorp/template 2.2.0 doesn't match any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for packages targeting different
│ platforms); for more information: https://www.terraform.io/language/provider-checksum-verification
```
then run the following commands to resolve this:
```
brew install kreuzwerker/taps/m1-terraform-provider-helper
m1-terraform-provider-helper activate
m1-terraform-provider-helper install hashicorp/template -v v2.2.0
rm .terraform.lock.hcl
terraform init --upgrade
```
3. Deploy:
```bash theme={"system"}
terraform apply
```
Once it’s finished deploying, you can access Mage in your browser.
1. Open your EC2 dashboard.
2. View all load balancers.
3. Click on the load balancer with the name `mage-data-prep` in it (if you
changed the app name, then find the load balancer with that app name).
4. Find the public DNS name, copy that, and paste it in your browser.
***
**Using Docker**
From the root directory of Mage, run the following commands:
1. Initialize Terraform:
```bash theme={"system"}
docker run -i -t -v $(pwd):/mage --workdir="/absolute_path_to/mage-ai-terraform-templates/aws" \
hashicorp/terraform:latest init
```
2. Deploy:
```bash theme={"system"}
docker run -i -t -v $(pwd):/mage --workdir="/absolute_path_to/mage-ai-terraform-templates/aws" \
--env AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
--env AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
hashicorp/terraform:latest apply
```
***
### Errors
**Page isn’t loading**
If you run into connection issues, check to see if your IP is whitelisted in the
appropriate [security group](#security).
**503 Forbidden**
Check to see if the service task in your ECS cluster is running or if it
stopped.
503 typically means that the service task isn’t running and that can be caused
by a variety of things.
Open the service task that stopped running and click on the "Logs" tab to see
what issues occurred.
**`exec /bin/sh: exec format error`**
If your cluster is running and the service launches a task that eventually
stops, check the logs for the stopped task. If the logs contain the error above,
try re-building your image with the platform flag `--platform linux/amd64`.
***
### Updating Mage versions
After Mage is running in your cloud, you can update the Mage Docker image
version by running the following command in your CLI tool:
```bash theme={"system"}
aws ecs update-service --cluster [cluster_name] --service [service_name] --force-new-deployment
```
> `cluster_name`
>
> This is the name of your AWS ECS cluster. Go to the AWS ECS dashboard to view
> the clusters and find the cluster with the name that contains the `app_name`
> from `variables.tf`.
> `service_name`
>
> This is the name of your AWS ECS service running in your AWS ECS cluster with
> the name identified from above.
***
### Misc
#### Security
To enable other IP addresses access to Mage, open the security group named
`mage-data-prep-sg` to whitelist IPs.
Add a new inbound rule for HTTP port 80 and use your IP address.
#### HTTPS enabling
Terraform creates a load balancer with a HTTP listener by default.
To enable HTTPS for your cloud app, you need to have SSL certificates. You can
create a SSL certificate in
[AWS Certificate Manager](https://console.aws.amazon.com/acm/home?#/certificates/list).
After you get a certificate, add a HTTPS listener in your load balancer group.
1. Open your EC2 dashboard.
2. View all load balancers.
3. Click on the load balancer with the name `mage-data-prep` in it (if you
changed the app name, then find the load balancer with that app name).
4. Click "Listeners and rules" tab.
5. Click "Add listener" button
6. For "Protocol" and "Port", choose HTTPS and keep the default port or enter a
different port.
7. In "Default actions", select "Forward to target groups" and forward to the target group that
contains your app name (`mage-data-prep-production-tg` by default)
8. In "Security Policy", keep the default security policy.
9. For "Default SSL certificate", do one of the following:
1. If you created or imported a certificate using AWS Certificate Manager,
choose From ACM and choose the certificate.
2. If you uploaded a certificate using IAM, choose From IAM and choose the
certificate.
10. Click "Add" to create the HTTPS listener.
11. Edit your HTTP listener, change the default action to redirect traffic to
your HTTPS listener.
The detailed instructions can be found in:
[https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-https-listener.html)
#### Terminate all resources
If you want to delete all resources created from deploying, run the following
command:
```bash theme={"system"}
terraform destroy
```
A sample output could look like this:
```
Destroy complete! Resources: 10 destroyed.
```
### Authentication with ECS task execution role
Mage creates an ECS task execution role with the format `{app_name}-execution-task-role`.
You can grant the ECS task permissions to access other AWS services by attaching IAM policies to this ECS task execution role.
For example, if you want to list the AWS events that can be used for event triggers, you can attach IAM Policy `CloudWatchEventsReadOnlyAccess`
to your ECS task execution role.
***
## Deploy to EKS cluster
Use Terraform scripts in [aws-eks](https://github.com/mage-ai/mage-ai-terraform-templates/tree/master/aws-eks)
folder to provision resources for AWS EKS cluster.
You can run the following command to see all the resources that will be created
by Terraform:
```bash theme={"system"}
cd aws-eks
terraform plan
```
Similar with deploying to ECS cluster, you need to set up the [environment variables](/production/deploying-to-cloud/aws/setup#1-environment-variables) first, and you can customize the Terraform settings in [./aws-eks/variables.tf](https://github.com/mage-ai/mage-ai-terraform-templates/blob/master/aws-eks/variables.tf) file.
Then, follow the steps below to set up EKS cluster and deploy Mage.
Run `terraform apply` to provision the EKS cluster
Configure `kubectl` with command `aws eks --region $(terraform output -raw region) update-kubeconfig --name $(terraform output -raw cluster_name)`
If you are planning to use workspaces in EFS, you will need to add additional configurations to the StorageClass.
```yaml theme={"system"}
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: efs-sc
provisioner: efs.csi.aws.com
# needed for EFS CSI driver dynamic provisioning
parameters:
provisioningMode: efs-ap
fileSystemId:
directoryPerms: "700"
```
Update the `EFS_VOLUME_ID` in [kube/efs-pvc.yaml](https://github.com/mage-ai/mage-ai-terraform-templates/blob/master/aws-eks/kube/efs-pvc.yaml) to the efs id in terraform output
For workspaces support, you may need to install the EFS CSI driver add-on directly from the AWS console. Make sure
to use a driver version that supports dynamic provisioning.
Run command `kubectl apply -k "github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/?ref=master"` to install efs csi driver
Run `kubectl apply -f kube/` to deploy app or use Helm to deploy Mage.
Get the node IP via command `kubectl get nodes -o wide` and access Mage with `http://[ip]:{node_port}`