Only in Mage Pro.Try our fully managed solution to access this advanced feature.
Overview
Mage Pro workspaces provide isolated environments for developing and managing data pipelines. Each workspace operates as a separate container instance, allowing for independent resource allocation and configuration management.Core Components
- Runtime Environment: Containerized Python environment
- Storage System: Persistent volume for code and data
- Resource Manager: Kubernetes-based resource allocation
- Access Control: Role-based permission system
To utilize the collaborative workspace environment, please contact Mage to request an account upgrade.
The Mage team will need to enable this feature for your organization.
Setup Guide
- Step 1: After logging into the Mage Pro environment you will be in the Workspaces interface (url: https://cluster.mage.ai/[cluster-uuid]/manage). Press the Create new workspace button.
- Step 2: Give the workspace a name.
- Step 3: Click the dropdown arrow and toggle on the Kubernetes configuration if you need to custom configure your Kubernetes container. (see Kubernetes container configuration section for more details).
- Step 4: Click the Lifecycle (optional) dropdown to create lifecycle management policies (see Lifecycle management policies section for more information).
- Step 4: Click the create button and wait a few minutes for your workspace to start.
- Step 5: Once your workspace is running, click the workspace open button under the open column in the workspace management interface.
Kubernetes container configuration
Kubernetes provides a robust container orchestration system for managing your workspace resources and environment. The configuration above covers two key aspects:- Resource Configuration: Controls memory and CPU allocation for your workspace container, with both minimum guarantees and upper limits to ensure stable performance.
- Environment Configuration: Enables customization of environment variables, specifically the project code location if you’ve modified it from the default path.
Environment configuration
Setting USER_CODE_PATH
If you’ve modified the default USER_CODE_PATH
environment variable in the base project (Mage Pro management portal), update your Kubernetes configuration:
Adding Other Environment Variables
You can define additional environment variables in the same env section to further configure your workspace:Resource configuration (optional)
Optionally configure your workspace’s computing resources in Kubernetes:Share secrets across workspaces
To share secrets across different workspaces developers need to create a custom environment variable calledSHARE_SECRETS_ACROSS_WORKSPACES
then set the value to 1
to enable this feature. This will
allow developers to share their secrets across different workspaces within a cluster.
Inherit environment variables
Configuring your Mage Pro workspaces to inherit the main cluster’s environment variables is essential for efficiently configuring your workspace container. The following options allow you to control how variables are shared within your environment. To enable environment variable inheritance in workspaces, you must set:Managed Cloud
The USER_DEFINED_VARS environment variable lets you control which environment variables are inherited by workspaces. It accepts a comma-separated list of variable names, giving you flexibility to include only the configuration values you need across workspaces.- If
USER_DEFINED_VARS
is not set: all environment variables will be inherited automatically. - If
USER_DEFINED_VARS
is set: only the specified variables will be inherited.
DEV_ENV_*
).
Self-hosted Mage Pro (Hybrid Cloud, Private Cloud, On-prem)
You must setUSER_DEFINED_VARS
to define which variables to inherit:
Inherit variables by workspace environment (new)
You can define inheritance rules per workspace environment using theENV
variable:
{env_value}
: value of theENV
variable in the workspace.{pattern}
: wildcard/regex pattern for matching variables.
ENV=dev
inherits all variables matching DEV_ENV_*
.
Updating workspaces after environment variable changes
When you add or modify environment variables in your base cluster, existing workspaces will not automatically inherit these changes. To ensure your workspaces reflect the updated environment configuration, you must restart the affected workspaces. To update a workspace:- Open the workspace settings by clicking into the workspace you want to update
- Click the update button for your workspace
- Click the dropdown arrow on the Kubernetes configuration section
- Toggle the “update workspace settings” option to true
- Click update
Lifecycle management policies (optional)
Mage Pro workspaces support various lifecycle management policies that control how your workspace behaves throughout its runtime. These policies include automated termination settings, pre-initialization scripts, and post-initialization procedures, giving you complete control over your workspace’s behavior from startup to shutdown.Termination Policy
The termination policy controls how your workspace shuts down when inactive. You can enable auto-termination, which will automatically shut down your workspace after a specified period of inactivity. The idle time is measured in seconds, and you can set this duration based on your needs. For example, you might set it to 1800 seconds (30 minutes) or 3600 seconds (1 hour).Pre-Start Scripts
Pre-start scripts run before your workspace container is initialized. These scripts must include aget_custom_configs
method that returns a dictionary of configurations. This is where you can set up environment variables, modify container configurations, and prepare any necessary resources. The script executes before any workspace services start, making it ideal for initial setup and configuration tasks.
Post-Start Scripts
Post-start scripts execute after your workspace is fully initialized and running. You can specify either a direct command or provide a path to a script for more complex procedures. These scripts are typically used for starting services, performing health checks, or completing any additional setup tasks that require the workspace to be fully operational.Global Roles
Users can be assigned global roles that apply across all workspaces within your Mage Pro environment. Global role assignment is managed by project administrators through Users tab in left navigation menu located in the main project interface.Workspace-Specific Roles
Individual workspaces can have unique role settings. Workspace owners can:- Assign user roles specific to their workspace
- Modify existing role assignments
- Remove user access to their workspace