Kubernetes executor

Use Mage Pro if you need to run your blocks in separate pods or process while developing in the code editor.
Running a block from the code editor, while developing a pipeline, won’t use this executor. Instead, the block will be executed using the same process that the code editor is running on.

Try Mage Pro for free

If your Mage app is deployed in a Kubernetes cluster, you can configure the Kubernetes Executor (executor_type: k8s) to run each block in its own Kubernetes pod. Defaults in Mage Pro:

Namespace: default
Job name format: mage-data-prep-block-{block_run_id}

You can override these defaults at the block level, project level, or with a full Kubernetes job template.

Basic Setup

To configure a pipeline block to use Kubernetes executor, you simply just need to update the executor_type of the block to k8s in pipeline’s metadata.yaml:

blocks:
- uuid: example_data_loader
  type: data_loader
  upstream_blocks: []
  downstream_blocks: []
  executor_type: k8s
  ...

By default, Mage uses default as the Kubernetes namespace. You can customize the namespace by setting the KUBE_NAMESPACE environment variable.

export KUBE_NAMESPACE=my-namespace

Configuration Methods

You can configure Kubernetes Executor in three ways:

1. Block-Level Configuration

Add executor_config to a block in pipeline’s metadata.yaml:

blocks:
- uuid: example_data_loader
  type: data_loader
  executor_type: k8s
  executor_config:
    namespace: default
    resource_limits:
      cpu: 1000m
      memory: 2048Mi
    resource_requests:
      cpu: 500m
      memory: 1024Mi

✅ Use when: You only want to:

Run certain blocks in the Kubernetes executor
Override the k8s executor config for specific blocks

2. Project-Level Configuration (applies to all k8s executor blocks)

Set k8s_executor_config in the project’s metadata.yaml:

k8s_executor_config:
  job_name_prefix: data-prep
  namespace: default
  resource_limits:
    cpu: 1000m
    memory: 2048Mi
  resource_requests:
    cpu: 500m
    memory: 1024Mi
  service_account_name: default
  # Node scheduling configurations
  pod:
    node_selector:
      node-type: gpu
      instance-type: c5.large
    scheduler_name: custom-scheduler
    tolerations:
      - key: "nvidia.com/gpu"
        operator: "Exists"
        effect: "NoSchedule"
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: "node-type"
                  operator: "In"
                  values: ["gpu", "compute"]
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: mage-pipeline
            topologyKey: "kubernetes.io/hostname"
    volumes:
      - name: data-volume
        persistent_volume_claim:
          claim_name: mage-data-pvc
    image_pull_secrets: my-registry-secret

The Kubernetes job name is in this format: mage-{job_name_prefix}-block-{block_run_id}. The default job_name_prefix is data-prep. You can customize it in the k8s_executor_config. You can interpolate the trigger name in the job_name_prefix field with the format {trigger_name}.

GPU Support:

k8s_executor_config:
  resource_limits:
    nvidia.com/gpu: 1  # request 1 GPU
  pod:
    node_selector:
      accelerator: nvidia-tesla-v100
    tolerations:
      - key: "nvidia.com/gpu"
        operator: "Exists"
        effect: "NoSchedule"

Make sure GPU device plugins are installed.

Custom container & job spec:

k8s_executor_config:
  container_config:
    image: mageai/mageai:0.9.7
    env:
    - name: USER_CODE_PATH
      value: /home/src/k8s_project
  job:
    active_deadline_seconds: 120
    backoff_limit: 3
    ttl_seconds_after_finished: 86400

✅ Use when: You want consistent settings for all blocks using the Kubernetes Executor in the project.

3. Full Kubernetes Job Template (maximum control)

Set the K8S_CONFIG_FILE environment variable to the path of a YAML configuration file. Example template:

metadata:
  annotations:
    application: "mage"
    component: "executor"
  labels:
    application: "mage"
    type: "spark"
  namespace: "default"

pod:
  service_account_name: "mage-service-account"
  image_pull_secrets: "my-registry-secret"
  node_selector:
    node-type: "compute"
    instance-type: "c5.xlarge"
    zone: "us-west-1a"
  scheduler_name: "custom-scheduler"
  tolerations:
    - key: "nvidia.com/gpu"
      operator: "Exists"
      effect: "NoSchedule"
    - key: "spot-instance"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: "node-type"
                operator: "In"
                values: ["compute", "gpu"]
              - key: "instance-type"
                operator: "In"
                values: ["c5.xlarge", "c5.2xlarge"]
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          preference:
            matchExpressions:
              - key: "zone"
                operator: "In"
                values: ["us-west-1a", "us-west-1b"]
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchLabels:
              app: "mage-pipeline"
          topologyKey: "kubernetes.io/hostname"
  volumes:
  - name: data-pvc
    persistent_volume_claim:
      claim_name: mage-data-pvc
  - name: config-volume
    config_map:
      name: mage-config
  - name: secret-volume
    secret:
      secret_name: mage-secrets
  - name: empty-dir-volume
    empty_dir: {}

container:
  name: "mage-data"
  image: "mageai/mageai:latest"
  image_pull_policy: "IfNotPresent"
  env:
    - name: "KUBE_NAMESPACE"
      value: "default"
    - name: "secret_key"
      value: "somesecret"
  resources:
    limits:
      cpu: "1"
      memory: "1Gi"
      nvidia.com/gpu: 1
    requests:
      cpu: "0.1"
      memory: "0.5Gi"
      nvidia.com/gpu: 1
  volume_mounts:
    - mount_path: "/tmp/data"
      name: "data-pvc"
    - mount_path: "/etc/config"
      name: "config-volume"
    - mount_path: "/etc/secrets"
      name: "secret-volume"
      read_only: true

Node Scheduling Configuration

Node Selector

Use node_selector to schedule pods on specific nodes based on labels:

k8s_executor_config:
  pod:
    node_selector:
      node-type: "gpu"           # Schedule on GPU nodes
      instance-type: "c5.large"  # Schedule on specific instance types
      zone: "us-west-1a"         # Schedule in specific availability zones

Tolerations

Use tolerations to allow pods to be scheduled on tainted nodes:

k8s_executor_config:
  pod:
    tolerations:
      - key: "nvidia.com/gpu"
        operator: "Exists"
        effect: "NoSchedule"
      - key: "spot-instance"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"

Affinity Rules

Use affinity for advanced scheduling rules:

k8s_executor_config:
  pod:
    affinity:
      # Node affinity - prefer specific node types
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: "node-type"
                  operator: "In"
                  values: ["compute", "gpu"]
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            preference:
              matchExpressions:
                - key: "zone"
                  operator: "In"
                  values: ["us-west-1a", "us-west-1b"]

      # Pod anti-affinity - avoid co-locating similar pods
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: "mage-pipeline"
            topologyKey: "kubernetes.io/hostname"

Custom Scheduler

Use a custom scheduler for advanced scheduling logic:

k8s_executor_config:
  pod:
    scheduler_name: "custom-scheduler"

✅ Use when: You need full control over pod specs, annotations, volume mounts, and other Kubernetes settings.

Multi-Container Pods

If your Mage deployment runs in a multi-container pod, set the MAGE_CONTAINER_NAME environment variable to specify which container runs Mage:

env:
  - name: MAGE_CONTAINER_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name

If not set, Mage defaults to the first container in the pod.

Environment Variables Reference

Variable	Purpose	Default
`KUBE_NAMESPACE`	Namespace for job execution	default
`K8S_CONFIG_FILE`	Path to full Kubernetes job config file	—
`MAGE_CONTAINER_NAME`	Container to run Mage in multi-container pods	—

Deployment

Cloud infrastructure

Compute resources

Secret manager

Authentication

Kubernetes executor

Basic Setup

Configuration Methods

1. Block-Level Configuration

2. Project-Level Configuration (applies to all k8s executor blocks)

3. Full Kubernetes Job Template (maximum control)

Node Scheduling Configuration

Node Selector

Tolerations

Affinity Rules

Custom Scheduler

Multi-Container Pods

Environment Variables Reference

Deployment

Cloud infrastructure

Compute resources

Secret manager

Authentication

​Basic Setup

​Configuration Methods

​1. Block-Level Configuration

​2. Project-Level Configuration (applies to all k8s executor blocks)

​3. Full Kubernetes Job Template (maximum control)

​Node Scheduling Configuration

​Node Selector

​Tolerations

​Affinity Rules

​Custom Scheduler

​Multi-Container Pods

​Environment Variables Reference

Basic Setup

Configuration Methods

1. Block-Level Configuration

2. Project-Level Configuration (applies to all k8s executor blocks)

3. Full Kubernetes Job Template (maximum control)

Node Scheduling Configuration

Node Selector

Tolerations

Affinity Rules

Custom Scheduler

Multi-Container Pods

Environment Variables Reference