Kubernetes executor

If your Mage app is deployed in a Kubernetes cluster, you can configure the Kubernetes Executor (executor_type: k8s) to run each block in its own Kubernetes pod. Defaults in Mage Pro:

Namespace: default
Job name format: mage-data-prep-block-{block_run_id}

You can override these defaults at the block level, project level, or with a full Kubernetes job template.

Basic Setup

To configure a pipeline block to use Kubernetes executor, you simply just need to update the executor_type of the block to k8s in pipeline’s metadata.yaml:

blocks:
- uuid: example_data_loader
  type: data_loader
  upstream_blocks: []
  downstream_blocks: []
  executor_type: k8s
  ...

By default, Mage uses default as the Kubernetes namespace. You can customize the namespace by setting the KUBE_NAMESPACE environment variable.

export KUBE_NAMESPACE=my-namespace

Configuration Methods

You can configure Kubernetes Executor in three ways:

1. Block-Level Configuration

Add executor_config to a block in pipeline’s metadata.yaml:

blocks:
- uuid: example_data_loader
  type: data_loader
  executor_type: k8s
  executor_config:
    namespace: default
    resource_limits:
      cpu: 1000m
      memory: 2048Mi
    resource_requests:
      cpu: 500m
      memory: 1024Mi

✅ Use when: You only want to:

Run certain blocks in the Kubernetes executor
Override the k8s executor config for specific blocks

2. Project-Level Configuration (applies to all k8s executor blocks)

Set k8s_executor_config in the project’s metadata.yaml:

k8s_executor_config:
  job_name_prefix: data-prep
  namespace: default
  resource_limits:
    cpu: 1000m
    memory: 2048Mi
  resource_requests:
    cpu: 500m
    memory: 1024Mi
  service_account_name: default
  # Node scheduling configurations
  pod:
    node_selector:
      node-type: gpu
      instance-type: c5.large
    scheduler_name: custom-scheduler
    tolerations:
      - key: "nvidia.com/gpu"
        operator: "Exists"
        effect: "NoSchedule"
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: "node-type"
                  operator: "In"
                  values: ["gpu", "compute"]
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: mage-pipeline
            topologyKey: "kubernetes.io/hostname"
    volumes:
      - name: data-volume
        persistent_volume_claim:
          claim_name: mage-data-pvc
    image_pull_secrets: my-registry-secret

The Kubernetes job name is in this format: mage-{job_name_prefix}-block-{block_run_id}. The default job_name_prefix is data-prep. You can customize it in the k8s_executor_config. You can interpolate the trigger name in the job_name_prefix field with the format {trigger_name}.

GPU Support:

k8s_executor_config:
  resource_limits:
    nvidia.com/gpu: 1  # request 1 GPU
  pod:
    node_selector:
      accelerator: nvidia-tesla-v100
    tolerations:
      - key: "nvidia.com/gpu"
        operator: "Exists"
        effect: "NoSchedule"

Make sure GPU device plugins are installed.

Custom container & job spec:

k8s_executor_config:
  container_config:
    image: mageai/mageai:0.9.7
    env:
    - name: USER_CODE_PATH
      value: /home/src/k8s_project
  job:
    active_deadline_seconds: 120
    backoff_limit: 3
    ttl_seconds_after_finished: 86400

✅ Use when: You want consistent settings for all blocks using the Kubernetes Executor in the project.

3. Full Kubernetes Job Template (maximum control)

Set the K8S_CONFIG_FILE environment variable to the path of a YAML configuration file. Example template:

metadata:
  annotations:
    application: "mage"
    component: "executor"
  labels:
    application: "mage"
    type: "spark"
  namespace: "default"

pod:
  service_account_name: "mage-service-account"
  image_pull_secrets: "my-registry-secret"
  node_selector:
    node-type: "compute"
    instance-type: "c5.xlarge"
    zone: "us-west-1a"
  scheduler_name: "custom-scheduler"
  tolerations:
    - key: "nvidia.com/gpu"
      operator: "Exists"
      effect: "NoSchedule"
    - key: "spot-instance"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: "node-type"
                operator: "In"
                values: ["compute", "gpu"]
              - key: "instance-type"
                operator: "In"
                values: ["c5.xlarge", "c5.2xlarge"]
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          preference:
            matchExpressions:
              - key: "zone"
                operator: "In"
                values: ["us-west-1a", "us-west-1b"]
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchLabels:
              app: "mage-pipeline"
          topologyKey: "kubernetes.io/hostname"
  volumes:
  - name: data-pvc
    persistent_volume_claim:
      claim_name: mage-data-pvc
  - name: config-volume
    config_map:
      name: mage-config
  - name: secret-volume
    secret:
      secret_name: mage-secrets
  - name: empty-dir-volume
    empty_dir: {}

container:
  name: "mage-data"
  image: "mageai/mageai:latest"
  image_pull_policy: "IfNotPresent"
  env:
    - name: "KUBE_NAMESPACE"
      value: "default"
    - name: "secret_key"
      value: "somesecret"
  resources:
    limits:
      cpu: "1"
      memory: "1Gi"
      nvidia.com/gpu: 1
    requests:
      cpu: "0.1"
      memory: "0.5Gi"
      nvidia.com/gpu: 1
  volume_mounts:
    - mount_path: "/tmp/data"
      name: "data-pvc"
    - mount_path: "/etc/config"
      name: "config-volume"
    - mount_path: "/etc/secrets"
      name: "secret-volume"
      read_only: true

Node Scheduling Configuration

Node Selector

Use node_selector to schedule pods on specific nodes based on labels:

k8s_executor_config:
  pod:
    node_selector:
      node-type: "gpu"           # Schedule on GPU nodes
      instance-type: "c5.large"  # Schedule on specific instance types
      zone: "us-west-1a"         # Schedule in specific availability zones

Tolerations

Use tolerations to allow pods to be scheduled on tainted nodes:

k8s_executor_config:
  pod:
    tolerations:
      - key: "nvidia.com/gpu"
        operator: "Exists"
        effect: "NoSchedule"
      - key: "spot-instance"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"

Affinity Rules

Use affinity for advanced scheduling rules:

k8s_executor_config:
  pod:
    affinity:
      # Node affinity - prefer specific node types
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: "node-type"
                  operator: "In"
                  values: ["compute", "gpu"]
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            preference:
              matchExpressions:
                - key: "zone"
                  operator: "In"
                  values: ["us-west-1a", "us-west-1b"]

      # Pod anti-affinity - avoid co-locating similar pods
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: "mage-pipeline"
            topologyKey: "kubernetes.io/hostname"

Custom Scheduler

Use a custom scheduler for advanced scheduling logic:

k8s_executor_config:
  pod:
    scheduler_name: "custom-scheduler"

✅ Use when: You need full control over pod specs, annotations, volume mounts, and other Kubernetes settings.

Multi-Container Pods

If your Mage deployment runs in a multi-container pod, set the MAGE_CONTAINER_NAME environment variable to specify which container runs Mage:

env:
  - name: MAGE_CONTAINER_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name

If not set, Mage defaults to the first container in the pod.

Default config file

You can provide a default K8s executor config file that Mage reads from the path in K8S_DEFAULT_CONFIG_PATH environment variable. The file is loaded as a policy and merged with block-level and project-level config. Use this to enforce cluster-wide defaults (e.g. resource limits, service account) without editing every project. Helm chart: When deploying Mage Pro with the Mage Helm chart, configure the default via k8sExecutorConfig in values.yaml. The chart creates a ConfigMap from your config, mounts it into the control-plane (and workspace pods), and sets K8S_DEFAULT_CONFIG_PATH automatically. Do not set the env var manually when using the chart. See Using Helm for details. For non-Helm deployments, set the env var and ensure the file exists at that path (e.g. via your own ConfigMap or volume mount). File format:

enabled: true   # Set to false to disable this policy
enforce: true   # If true, policy overrides user config. If false, user config overrides.

resource_limits:
  cpu: "500m"
  memory: "1Gi"
resource_requests:
  cpu: "250m"
  memory: "512Mi"
service_account_name: default

pod:
  node_selector:
    node-type: compute
  tolerations: []
  volumes: []

container:
  env: []
  volume_mounts: []

Policy fields:

Field	Description
`enabled`	If `false`, the policy is not applied.
`enforce`	Controls how the default file is merged with user config (block + project). See Priority when merging below.
(other keys)	Same structure as project-level k8s_executor_config (e.g. `resource_limits`, `resource_requests`, `pod`, `container`, `metadata`, `job`).

The file must exist at the given path on the Mage control-plane. Use a ConfigMap or volume mount to provide it. When workspaces are created, only the K8s default config file is injected (volume + volume mount + env var) so workspace pods can read it; see Workspaces. Priority when merging default file with user config The default file is merged with the combined user config (block + project). The effective priority order (who wins for a given key) depends on enforce:

`enforce`	Effective priority (highest → lowest)	Result
`true`	1. Default config file → 2. Block → 3. Project → 4. Env vars → 5. Hardcoded	Policy is highest priority. Values from the default file override block and project for the same keys. Use this to enforce cluster-wide defaults (e.g. resource limits, service account) that users cannot override. Conflicting values can cause an error on update when strict conflict checking is used.
`false`	1. Block → 2. Project → 3. Default config file (fills gaps only) → 4. Env vars → 5. Hardcoded	User wins. Block and project override the policy for the same keys. The policy only fills in keys that block and project did not set. Use this to provide optional defaults while allowing per-block or per-project overrides.

So when enforce: true, the policy has the highest priority; when enforce: false, block and project are higher and the policy only supplies missing values.

Environment Variables Reference

General Configuration

Environment Variables Reference

Variable	Purpose	Default
`KUBE_NAMESPACE`	Namespace for job execution	default
`K8S_CONFIG_FILE`	Path to full Kubernetes job config file	—
`MAGE_CONTAINER_NAME`	Container to run Mage in multi-container pods	—

Default Resource Configuration

These environment variables set default resource limits and requests for all K8s executor jobs. They are applied when not specified in block-level or project-level configuration.

Variable	Purpose	Example Value
`K8S_EXECUTOR_DEFAULT_RESOURCE_REQUESTS_CPU`	Default CPU request for executor jobs	`250m`
`K8S_EXECUTOR_DEFAULT_RESOURCE_REQUESTS_MEMORY`	Default memory request for executor jobs	`512Mi`
`K8S_EXECUTOR_DEFAULT_RESOURCE_LIMITS_CPU`	Default CPU limit for executor jobs	`500m`
`K8S_EXECUTOR_DEFAULT_RESOURCE_LIMITS_MEMORY`	Default memory limit for executor jobs	`1Gi`
`K8S_EXECUTOR_DEFAULT_SERVICE_ACCOUNT_NAME`	Default service account for executor jobs	`default`

Priority order (highest to lowest):

Block-level executor_config in pipeline’s metadata.yaml
Project-level k8s_executor_config in project’s metadata.yaml
Environment variable defaults (listed above)
Hardcoded defaults

Deployment

Cloud infrastructure

Compute resources

Secret manager

Authentication

Kubernetes executor

Basic Setup

Configuration Methods

1. Block-Level Configuration

2. Project-Level Configuration (applies to all k8s executor blocks)

3. Full Kubernetes Job Template (maximum control)

Node Scheduling Configuration

Node Selector

Tolerations

Affinity Rules

Custom Scheduler

Multi-Container Pods

Default config file

Environment Variables Reference

General Configuration

Environment Variables Reference

Default Resource Configuration

​Basic Setup

​Configuration Methods

​1. Block-Level Configuration

​2. Project-Level Configuration (applies to all k8s executor blocks)

​3. Full Kubernetes Job Template (maximum control)

​Node Scheduling Configuration

​Node Selector

​Tolerations

​Affinity Rules

​Custom Scheduler

​Multi-Container Pods

​Default config file

​Environment Variables Reference

​General Configuration

​Environment Variables Reference

​Default Resource Configuration

Basic Setup

Configuration Methods

1. Block-Level Configuration

2. Project-Level Configuration (applies to all k8s executor blocks)

3. Full Kubernetes Job Template (maximum control)

Node Scheduling Configuration

Node Selector

Tolerations

Affinity Rules

Custom Scheduler

Multi-Container Pods

Default config file

Environment Variables Reference

General Configuration

Environment Variables Reference

Default Resource Configuration