We support multiple kernels in the code editor.
docker run -it -p 6789:6789 -v $(pwd):/home/src mageai/mageai /app/run_app.sh mage start [project_name]
create_emr.py
script under
scripts/spark
folder to launch a new EMR cluster. Example:
python3 create_cluster.py [project_path]
. Please make sure your AWS
crendentials are provided in ~/.aws/credentials
file or environment
variables (AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
) when executing
the script.ssh -i [path_to_key_pair] -L 0.0.0.0:9999:localhost:8998 [master_ec2_public_dns_name]
path_to_key_pair
is the path to the ec2_key_pair_name
configured in
metadata.yaml
filemaster_ec2_public_dns_name
in your newly created EMR cluster page
under attribute Master public DNS
mage run [project_name] [pipeline]
command. Example EMR config: