Pipeline edit page freezes browser

The data output of a block contains too much information for the browser to render.

To temporarily fix this, open the /terminal page in Mage. Then type the following command press enter to run it:

rm -rf mage_data/[project_name]/pipelines/[pipeline_name]/.variables
Change [project_name] to your Mage project name and change [pipeline_name] to the name of the pipeline that is freezing the browser.

The above command will remove the data output files for each block in your pipeline.

Can’t connect to local database from within Mage via Docker

If you’re running Mage via Docker and you’re trying to connect to a local database outside of Docker, you must use the following host or IP value when trying to connect to it: host.docker.internal.

Unpredictable Columns in an Upstream Dataframe

Sometimes, upstream data is out of our contol. If we’re expecting a column in a dataframe and writing SQL against it, we might see a SQL compilation error: invalid identifier.

Oh no

Thankfully, Mage Community member Josh Pachner has a solution.

Say you have an upstream block that looks like:

FROM {{ df }} df

Where the underlying df might all three columns… or just a subset. For example, col_1 isn’t always present.

For cases where expected columns might not in our dataframe, we can add a transformer block upstream of an exporter:

# headers = [cols we need in our dataframe]
# df = our DataFrame()

for head in headers:
  if head not in df.columns:
       df[head] = None

This is especially helpful in databases like Snowflake, which don’t have an iferror() function to check for missing columns.

InvalidToken error when accessing data in S3

Accessing data located in S3, you may see an error like this:

An error occurred (InvalidToken) when calling the GetObject operation: 
The provided token is malformed or otherwise invalid.

This Stack Overflow article details the issue. Removing the AWS_SESSION_TOKEN line from the io_config.yaml should resolve the problem.

Mage doesn’t load properly when running locally

When attempting to connect to Mage with a port other than 6789, the browser may not render the page correctly with localhost:port.

Instead, use to connect to Mage.

Using credential files in Docker

If you have files containing secret credentials (e.g. GCP credentials JSON files), you’ll need to mount them as a volume so that those files appear in your running Docker container.

Use the -v command in your docker run command to mount the directory containing your files:

docker run -it -p 6789:6789
    -v /Users/dragon/secrets:/home/secrets \
    -v $(pwd):/home/src mageai/mageai \
    /app/run_app.sh mage start [project_name]

or, using docker compose, add a volume mount to your Mage service. The following is an example of mounting the ~/.aws directory for authenticating via the AWS CLI:

    image: mageai/mageai:latest
      - ~/.aws:/root/.aws