Quickstart
Go from zero to Mage hero in under a minute. Weâll walk you through installing Mage and running your first pipeline. đŚ¸ââď¸
Mage Pro overview
After creating a Mage Pro account or logging into Mage Pro you are directed to the cluster management portal. Here you can manage your Mage clusters.
Cluster Pages
- Creating a New Cluster:
- On the clusters page, you can create a new cluster by clicking the âLaunch new clusterâ button.
- Fill in the required field Cluster Name. Note that environment variables can be customized after the cluster is created.
If you are in a free trial you are limited to a single cluster.
- Viewing Cluster List:
- The cluster list page provides an overview of all your clusters, displaying essential information such as the cluster status, ID, UUID, cloud provider, and the number of replicas.
- Cluster Details:
- Click the âOpen clusterâ button to view detailed information about the cluster. Here, you can see various metrics and statuses that help you monitor and manage your cluster effectively.
- Follow the environment variables documentation to customize your Mage Pro cluster.
Cluster login
- Accessing the cluster:
- Click the âOpen clusterâ button to navigate to the actual Mage cluster page. Here, you can interact with your cluster and use the Mage tool as you normally would.
- Currently, we use separate databases for each cluster for enhanced security. The users credentials in the Mage Pro cluster are managed inside the cluster. Use the default owner credentials (email: admin@admin.com, password: admin) to log in. After you login, you should either update the password of the default owner user or create your own owner user.
Credentials may differ from cluster portal. For initial setup use Email or username: admin@admin.com, Password: admin.
Run your first pipeline
- Step 1: From the pipeline editor page click the âNew pipelineâ button
- Step 2: Select a template for inspiration or start from scratch by clicking the âStart from scratchâ button
- Step 3: Select your pipeline type and give it an optional new name, description, and tag
- Step 4: Click the create new pipline button
- Step 5: Add your blocks by following the Youtube video below
âľď¸ Mage OSS overview
We recommend using Docker to get started.
Docker is a tool that allows you to run Mage in a containerized environment: you can run Mage on any operating system that supports Docker, including Windows, Mac, and Linux. Using Docker means that you donât have to worry about installing dependencies or configuring your environment. If youâd like to install Mage without Docker, you can use pip
or conda
.
If youâre familiar with Docker Compose or plan on adding or extending images (e.g. Postgres) in your project, we recommend starting from the Docker compose template. Otherwise, we recommend Docker run.
đŞ Get Mage
đââď¸ Run your first pipeline
If you havenât already, open a browser to http://localhost:6789
. From the pipelines page, select example_pipeline
and open the notebook view by selecting Edit pipeline
from the left side nav.
Select the first block by clicking it and select the âplayâ icon in the top right to run the block. Youâve just ran your first Mage block & loaded data from a dataset!
Do the same for the following cells in the pipeline to transform and export the data. Congrats, youâre now a Mage ninja! đĽˇ
đ§đťââď¸ Install Mage dependencies (optional)
Mage also has the following add-on packages:
Package | Install | Description |
---|---|---|
all | mage-ai[all] | install all add-ons |
azure | mage-ai[azure] | install Azure related packages |
clickhouse | mage-ai[clickhouse] | use Clickhouse for data import or export |
dbt | mage-ai[dbt] | install dbt packages |
google-cloud-storage | mage-ai[google-cloud-storage] | use Google Cloud Storage for data import or export |
hdf5 | mage-ai[hdf5] | process data in HDF5 file format |
mysql | mage-ai[mysql] | use MySQL for data import or export |
postgres | mage-ai[postgres] | use PostgreSQL for data import or export |
redshift | mage-ai[redshift] | use Redshift for data import or export |
s3 | mage-ai[s3] | use S3 for data import or export |
snowflake | mage-ai[snowflake] | use Snowflake for data import or export |
spark | mage-ai[spark] | use Spark (EMR) in your Mage pipeline |
streaming | mage-ai[streaming] | use Streaming pipelines |
To install these, run the following command from the Mage terminal:
or add the following to your requirements.txt
file:
You can access the terminal from the side nav on the right in the pipeline editor page. Read more about installing from requirements.txt
here.
đ§ Journey on
Navigate to our tutorials to learn more about Mage and how to build your own pipelines or continue exploring our docs for advanced configuration and deployment options.
If youâre interested in connecting a database in Docker, check out our guide for more information.
Was this page helpful?