API overview - Mage AI

Want to jump-in and interact with Mage via API? Follow the quickstart below to get going! Otherwise, head to the components section to learn about how our API is structured— this will be helpful if you’re interested in adding new API endpoints or developing Mage. You can read more about the components in the Design section on the sidebar to the left— it contains pages for Resources, Policies, and Presenters, which are the three components that make up an API endpoint.

Quickstart

Obtain API Key

Initialize Session

If your Mage project requires user authentication, REQUIRE_USER_AUTHENTICATION=1, you'll need to create a new session before you can make any API calls. If your project does not require user auth, you can skip straight to making API calls with your API key.To initialize a session, see the POST /api/sessions endpoint. You'll need to supply your email and password as a JSON payload. Here's an example of how to do this in Python.Before starting, first nano ~/YOUR_SECRETS_PATH/.env.mage, replacing YOUR_SECRETS_PATH with an apporpriate directory, to create a new secrets file. Then add the following to your secrets file:

EMAIL=YOUR_EMAIL
API_KEY=YOUR_API_KEY
PASSWORD=YOUR_PASSWORD

Then, in a Python script, you can do the following:

import requests
import json
import os
from dotenv import load_dotenv
from pprint import pprint

# Be sure to replace this with the path to your secrets file
secrets_path = os.path.expanduser("~/YOUR_SECRETS_PATH/.env.mage")

load_dotenv(secrets_path)

API_KEY = os.getenv('API_KEY')
EMAIL = os.getenv('EMAIL')
PASSWORD = os.getenv('PASSWORD')

base_url = 'http://localhost:6789'

headers = {
    "Content-Type": "application/json",
    "X-API-KEY": API_KEY
}

json_data = {
    "session": {
        "email": EMAIL,
        "password": PASSWORD
    }
}

s = requests.Session()
s.headers = headers
sessions_url = base_url + "/api/sessions"

r = s.post(url=sessions_url, json=json_data)

if r.json().get('error'):
    raise ValueError(r.text)
else:
    mage_session = r.json().get('session')
    pprint(mage_session)

Authenticate Session

To authenticate your session, we recommend using the oauth_token cookie. This can be done simply in Python requests with the following, where token is the value of the token key in the response payload:

token = mage_session.get('token')
s.headers['Cookie'] = f'oauth_token={token}'

Make API calls

With your API session configured, you can now make API calls! Here’s an example of how to list all pipelines:

pipelines_url = base_url + '/api/pipelines'
r = s.get(pipelines_url)
list_pipelines = r.json()

print(list_pipelines)

Authentication

Most API endpoints require authentication. The API request must include the following:

API key
OAuth token

API key

The API request must include the API key in the query parameter as api_key or in the request payload body as api_key. You can find your API key quickly by navigating to the GUI and checking the request log from the container or going to the “Network” tab in your browser’s developer tools.

OAuth token

The OAuth token is generated by creating a new session. Sessions are only required when REQUIRE_USER_AUTHENTICATION is enabled for your project— that is, when users must login to access Mage. The token provided in the session payload is the raw value. There are two ways to authenticate with this token:

Supply the raw token in the header as:

{"Cookie": "oauth_token=[RAW-TOKEN-FROM-RESPONSE]"}

Supply the decoded token in the header as:

{"OAUTH-TOKEN": "[DECODED-TOKEN-FROM-RESPONSE]"}

To decode a token:

from mage_ai.authentication.oauth2 import decode_token

decoded_token = decode_token(token_from_session_response) ['token']

Examples

Decoded token, API key in in URL:

curl -X GET "localhost:6789/api/pipelines?api_key=zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2" \
    -H "OAUTH-TOKEN: [DECODED-TOKEN-FROM-RESPONSE]"

Decoded token, API key in body:

curl -X POST "localhost:6789/api/pipelines" \
    -H "OAUTH-TOKEN: [DECODED-TOKEN-FROM-RESPONSE]" \
    -d '{
        "api_key": "zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2",
        "pipeline": {}
    }'

Raw token, API key in body:

curl -X POST "localhost:6789/api/pipelines" \
    -H "Cookie: oauth_token=[RAW-TOKEN-FROM-RESPONSE]" \
    -d '{
        "api_key": "zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2",
        "pipeline": {}
    }'

Responses

All our responses return a 200 status in the main payload. The any error code will be stored in the error key. For example, the following is a response from a failed session auth:

{
    "error": {
        "code": 404,
        "message": "Email/username and/or password invalid.",
        "type": "record_not_found"
    },
    "status": 200
}

To check for an error, one might try a solution like the following:

err_code = r.json().get('error').get('code'):

Components

If you’re not developing our API, you can jump right in and start querying endpoints using the quickstart above. Otherwise, our API is based on three components:

If you’d like to contribute a new endpoint, you’ll need to create one of each. To update an endpoint, it’s important to understand how they work together. Resources are the core of the API. They define the endpoints and the logic behind them. They are the only component that is required to create a new endpoint. They are also the only component that is required to update an existing endpoint. Policies determine what actions can be taken on a resource: what attributes can be read, and which attributes can be written. Presenters determine what attributes of the resource are returned to the client in the response. You can have different sets of attributes be included in the response based on the action (e.g. create, detail, delete, list, update) or a custom format. To see examples of each, take a look at the mage-ai/mage_ai/api directory here.

Testing

Create a test file in mage_ai/tests/api/operations/test_[resource_name].py; replace resource_name with the name of your resource. Paste in the following sample code into your file:

from mage_ai.orchestration.db.models.oauth import User
from mage_ai.tests.api.operations.base import BaseApiTestCase
from mage_ai.tests.factory import create_user


class SomeResourceOperationTests(BaseApiTestCase):
    model_class = User

    async def test_execute_delete(self):
        user = create_user(email=self.faker.email())

        await self.base_test_execute_delete(user.id)

    async def test_execute_detail(self):
        user = create_user()
        await self.base_test_execute_detail(user.id, dict(
            email=user.email,
            username=user.username,
        ))

This is only a sample test file. It’s running tests on the UserResource. You’ll need to update the test logic to unit test your custom API endpoint.

Environment-specific configuration

API

​Quickstart

​Authentication

​API key

​OAuth token

​Examples

​Responses

​Components

​Testing