Add Python packages to requirements.txt

Location

At the root of your Mage project directory, you’ll find a file named requirements.txt. This file is used to specify all the Python packages that your project depends on.

Path

At the root of your project directory, there is a file named requirements.txt. Open this file and add the Python package you want installed and available throughout your entire project.

For example, if your project is named demo_project then the file will be located at demo_project/requirements.txt.

Adding packages

To add a package, simply open the requirements.txt file and list each package on a new line. You can also specify version requirements if needed.

Example requirements.txt:

matplotlib==3.5.2
tqdm

In this example the requirements.txt file will install:

  • matplotlib version 3.5.2
  • tqdm

Automatic installation

When installing Mage using Docker, Mage provides two main methods for automatically installing third-party packages at runtime. These methods utilize the USER_CODE_PATH environment variable, which defines your project path (e.g. /home/src/project_name), to locate your requirements.txt file and install the packages listed there.

If you’re running Mage using pip or conda, your local machine’s environment variables are accessible within the running Mage app.

Prerequisites

  1. Ensure that the environment variable USER_CODE_PATH is set to your project path, using the command line flags or the Dockerfile methods.

    Example: /home/src/project_name

  2. Mage uses a script named run_app.sh to manage the application startup process.

Installation process

When you start your Mage server:

  1. The run_app.sh script checks the USER_CODE_PATH environment variable.
  2. It locates the requirements.txt file within the specified project directory.
  3. All packages listed in requirements.txt are automatically installed before the server starts.

This ensures that all required packages are available for your project and pipelines.