> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mage.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Kubernetes executor

> Run Mage pipeline blocks in separate Kubernetes pods for scalability, isolation, and resource control.

export const ProOnly = ({button = 'Get started for free', description = 'Try our fully managed solution to access this advanced feature.', source = 'documentation', title = 'Only in Mage Pro.'}) => <a href={`https://cloud.mage.ai/sign-up?source=${source}`} className="block my-4 px-5 py-4 overflow-hidden rounded-xl flex gap-3 border border-emerald-500/20 bg-emerald-50/50 dark:border-emerald-500/30 dark:bg-emerald-500/10" target="_blank">
    <div style={{
  display: 'flex',
  alignItems: 'center',
  width: '100%'
}}>
      <div className="text-sm prose min-w-0 text-emerald-900 dark:text-emerald-200" style={{
  flex: 1
}}>
        {title}
        <p className="normal">{description}</p>
      </div>

      <div> </div>

      <div>
        <ProButton label={button} href={`https://cloud.mage.ai/sign-up?source=${source}`} />
      </div>
    </div>
  </a>;

export const ProBanner = ({button = 'Try Mage Pro for free', description, source = 'documentation', title = 'Our fully managed solution for teams is now available!'}) => <a href={`https://cloud.mage.ai/sign-up?source=${source}`} className="block my-4 px-5 py-4 overflow-hidden rounded-xl flex gap-3 border border-emerald-500/20 bg-emerald-50/50 dark:border-emerald-500/30 dark:bg-emerald-500/10" target="_blank">
    <div style={{
  display: 'flex',
  alignItems: 'center',
  width: '100%'
}}>
      <div className="text-sm prose min-w-0 text-emerald-900 dark:text-emerald-200" style={{
  flex: 1
}}>
        {title}
        {description && <br />}
        {description && <p className="normal">{description}</p>}
      </div>

      <div> </div>

      <div>
        <ProButton label={button} href={`https://cloud.mage.ai/sign-up?source=${source}`} />
      </div>
    </div>
  </a>;

export const ProButton = ({href, label = 'Get started with Mage Pro for free', source = 'documentation'}) => <div style={{
  height: 32,
  position: 'relative'
}}>
    <a target="_blank" className="group px-4 py-1.5 relative inline-flex items-center text-sm font-medium rounded-full" href={href ?? `https://cloud.mage.ai/sign-up?source=${source}`}>
      <span className="absolute inset-0 bg-primary-dark dark:bg-primary-light/10 border-primary-light/30 rounded-full dark:border group-hover:opacity-[0.9] dark:group-hover:border-primary-light/60">
      </span>

      <div className="mr-0.5 space-x-2.5 flex items-center">
        <span class="z-10 text-white dark:text-primary-light">
          {label}
        </span>

        <svg width="3" height="24" viewBox="0 -9 3 24" class="h-5 rotate-0 overflow-visible text-white/90 dark:text-primary-light">
          <path d="M0 0L3 3L0 6" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round"></path>
        </svg>
      </div>
    </a>
  </div>;

<ProBanner
  source="kubernetes-executor"
  title="Use Mage Pro if you need to run your blocks in separate pods or process while developing in the code editor."
  description={`
Running a block from the code editor, while developing a pipeline, won’t use this executor.
Instead, the block will be executed using the same process that the code editor is running on.
`}
/>

If your Mage app is deployed in a **Kubernetes cluster**, you can configure the **Kubernetes Executor** (`executor_type: k8s`) to run each block in its own Kubernetes pod.

**Defaults in Mage Pro:**

* Namespace: `default`
* **Job name format:** `mage-data-prep-block-{block_run_id}`

You can override these defaults at the **block level**, **project level**, or with a **full Kubernetes job template**.

## Basic Setup

To configure a pipeline block to use Kubernetes executor, you simply just need to update the `executor_type` of the block to `k8s` in pipeline's metadata.yaml:

```yaml theme={"system"}
blocks:
- uuid: example_data_loader
  type: data_loader
  upstream_blocks: []
  downstream_blocks: []
  executor_type: k8s
  ...
```

By default, Mage uses `default` as the Kubernetes namespace. You can customize the namespace by setting the `KUBE_NAMESPACE` environment variable.

```bash theme={"system"}
export KUBE_NAMESPACE=my-namespace
```

## Configuration Methods

You can configure Kubernetes Executor in **three ways**:

### 1. Block-Level Configuration

Add `executor_config` to a block in pipeline's `metadata.yaml`:

```yaml theme={"system"}
blocks:
- uuid: example_data_loader
  type: data_loader
  executor_type: k8s
  executor_config:
    namespace: default
    resource_limits:
      cpu: 1000m
      memory: 2048Mi
    resource_requests:
      cpu: 500m
      memory: 1024Mi
```

✅ **Use when:**
You only want to:

* Run certain blocks in the Kubernetes executor
* Override the k8s executor config for specific blocks

### 2. Project-Level Configuration (applies to all k8s executor blocks)

Set `k8s_executor_config` in the project's metadata.yaml:

```yaml theme={"system"}
k8s_executor_config:
  job_name_prefix: data-prep
  namespace: default
  resource_limits:
    cpu: 1000m
    memory: 2048Mi
  resource_requests:
    cpu: 500m
    memory: 1024Mi
  service_account_name: default
  # Node scheduling configurations
  pod:
    node_selector:
      node-type: gpu
      instance-type: c5.large
    scheduler_name: custom-scheduler
    tolerations:
      - key: "nvidia.com/gpu"
        operator: "Exists"
        effect: "NoSchedule"
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: "node-type"
                  operator: "In"
                  values: ["gpu", "compute"]
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: mage-pipeline
            topologyKey: "kubernetes.io/hostname"
    volumes:
      - name: data-volume
        persistent_volume_claim:
          claim_name: mage-data-pvc
    image_pull_secrets: my-registry-secret
```

* The Kubernetes job name is in this format: `mage-{job_name_prefix}-block-{block_run_id}`.
  The default `job_name_prefix` is `data-prep`.
  You can customize it in the `k8s_executor_config`.
  You can interpolate the trigger name in the `job_name_prefix` field with the format `{trigger_name}`.

- **GPU Support:**
  ```yaml theme={"system"}
  k8s_executor_config:
    resource_limits:
      nvidia.com/gpu: 1  # request 1 GPU
    pod:
      node_selector:
        accelerator: nvidia-tesla-v100
      tolerations:
        - key: "nvidia.com/gpu"
          operator: "Exists"
          effect: "NoSchedule"
  ```
  Make sure [GPU device plugins](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#using-device-plugins) are installed.

- **Custom container & job spec:**
  ```yaml theme={"system"}
  k8s_executor_config:
    container_config:
      image: mageai/mageai:0.9.7
      env:
      - name: USER_CODE_PATH
        value: /home/src/k8s_project
    job:
      active_deadline_seconds: 120
      backoff_limit: 3
      ttl_seconds_after_finished: 86400
  ```

✅ **Use when:**
You want consistent settings for all blocks using the Kubernetes Executor in the project.

### 3. Full Kubernetes Job Template (maximum control)

Set the `K8S_CONFIG_FILE` environment variable to the path of a YAML configuration file.

**Example template:**

```yaml theme={"system"}
metadata:
  annotations:
    application: "mage"
    component: "executor"
  labels:
    application: "mage"
    type: "spark"
  namespace: "default"

pod:
  service_account_name: "mage-service-account"
  image_pull_secrets: "my-registry-secret"
  node_selector:
    node-type: "compute"
    instance-type: "c5.xlarge"
    zone: "us-west-1a"
  scheduler_name: "custom-scheduler"
  tolerations:
    - key: "nvidia.com/gpu"
      operator: "Exists"
      effect: "NoSchedule"
    - key: "spot-instance"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: "node-type"
                operator: "In"
                values: ["compute", "gpu"]
              - key: "instance-type"
                operator: "In"
                values: ["c5.xlarge", "c5.2xlarge"]
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          preference:
            matchExpressions:
              - key: "zone"
                operator: "In"
                values: ["us-west-1a", "us-west-1b"]
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchLabels:
              app: "mage-pipeline"
          topologyKey: "kubernetes.io/hostname"
  volumes:
  - name: data-pvc
    persistent_volume_claim:
      claim_name: mage-data-pvc
  - name: config-volume
    config_map:
      name: mage-config
  - name: secret-volume
    secret:
      secret_name: mage-secrets
  - name: empty-dir-volume
    empty_dir: {}

container:
  name: "mage-data"
  image: "mageai/mageai:latest"
  image_pull_policy: "IfNotPresent"
  env:
    - name: "KUBE_NAMESPACE"
      value: "default"
    - name: "secret_key"
      value: "somesecret"
  resources:
    limits:
      cpu: "1"
      memory: "1Gi"
      nvidia.com/gpu: 1
    requests:
      cpu: "0.1"
      memory: "0.5Gi"
      nvidia.com/gpu: 1
  volume_mounts:
    - mount_path: "/tmp/data"
      name: "data-pvc"
    - mount_path: "/etc/config"
      name: "config-volume"
    - mount_path: "/etc/secrets"
      name: "secret-volume"
      read_only: true
```

## Node Scheduling Configuration

### Node Selector

Use `node_selector` to schedule pods on specific nodes based on labels:

```yaml theme={"system"}
k8s_executor_config:
  pod:
    node_selector:
      node-type: "gpu"           # Schedule on GPU nodes
      instance-type: "c5.large"  # Schedule on specific instance types
      zone: "us-west-1a"         # Schedule in specific availability zones
```

### Tolerations

Use `tolerations` to allow pods to be scheduled on tainted nodes:

```yaml theme={"system"}
k8s_executor_config:
  pod:
    tolerations:
      - key: "nvidia.com/gpu"
        operator: "Exists"
        effect: "NoSchedule"
      - key: "spot-instance"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"
```

### Affinity Rules

Use `affinity` for advanced scheduling rules:

```yaml theme={"system"}
k8s_executor_config:
  pod:
    affinity:
      # Node affinity - prefer specific node types
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: "node-type"
                  operator: "In"
                  values: ["compute", "gpu"]
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            preference:
              matchExpressions:
                - key: "zone"
                  operator: "In"
                  values: ["us-west-1a", "us-west-1b"]

      # Pod anti-affinity - avoid co-locating similar pods
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: "mage-pipeline"
            topologyKey: "kubernetes.io/hostname"
```

### Custom Scheduler

Use a custom scheduler for advanced scheduling logic:

```yaml theme={"system"}
k8s_executor_config:
  pod:
    scheduler_name: "custom-scheduler"
```

✅ **Use when:**
You need full control over pod specs, annotations, volume mounts, and other Kubernetes settings.

## Multi-Container Pods

If your Mage deployment runs in a **multi-container pod**, set the `MAGE_CONTAINER_NAME` environment variable to specify which container runs Mage:

```yaml theme={"system"}
env:
  - name: MAGE_CONTAINER_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
```

If not set, Mage defaults to the **first container** in the pod.

## Default config file

<ProOnly source="kubernetes-executor" />

You can provide a **default K8s executor config file** that Mage reads from the path in `K8S_DEFAULT_CONFIG_PATH` environment variable. The file is loaded as a policy and merged with block-level and project-level config. Use this to enforce cluster-wide defaults (e.g. resource limits, service account) without editing every project.

**Helm chart:** When deploying **Mage Pro** with the [Mage Helm chart](https://github.com/mage-ai/helm-charts), configure the default via **`k8sExecutorConfig`** in `values.yaml`. The chart creates a ConfigMap from your config, mounts it into the control-plane (and workspace pods), and sets `K8S_DEFAULT_CONFIG_PATH` automatically. Do not set the env var manually when using the chart. See [Using Helm](/production/deploying-to-cloud/using-helm#default-workspace-and-k8s-executor-config) for details. For non-Helm deployments, set the env var and ensure the file exists at that path (e.g. via your own ConfigMap or volume mount).

**File format:**

```yaml theme={"system"}
enabled: true   # Set to false to disable this policy
enforce: true   # If true, policy overrides user config. If false, user config overrides.

resource_limits:
  cpu: "500m"
  memory: "1Gi"
resource_requests:
  cpu: "250m"
  memory: "512Mi"
service_account_name: default

pod:
  node_selector:
    node-type: compute
  tolerations: []
  volumes: []

container:
  env: []
  volume_mounts: []
```

**Policy fields:**

| Field          | Description                                                                                                                                                                                                                              |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `enabled`      | If `false`, the policy is not applied.                                                                                                                                                                                                   |
| `enforce`      | Controls how the default file is merged with **user config** (block + project). See [Priority when merging](#priority-when-merging-default-file-with-user-config) below.                                                                 |
| *(other keys)* | Same structure as [project-level k8s\_executor\_config](/production/executors/k8#2-project-level-configuration-applies-to-all-k8s-executor-blocks) (e.g. `resource_limits`, `resource_requests`, `pod`, `container`, `metadata`, `job`). |

The file must exist at the given path on the Mage control-plane. Use a ConfigMap or volume mount to provide it. When **workspaces** are created, only the K8s default config file is injected (volume + volume mount + env var) so workspace pods can read it; see [Workspaces](/guides/developer-ux/workspaces#default-workspace-config-file).

**Priority when merging default file with user config**

The default file is **merged** with the combined user config (block + project). The **effective priority order** (who wins for a given key) depends on `enforce`:

| `enforce`   | Effective priority (highest → lowest)                                                             | Result                                                                                                                                                                                                                                                                                                          |
| ----------- | ------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`true`**  | **1. Default config file** → 2. Block → 3. Project → 4. Env vars → 5. Hardcoded                   | **Policy is highest priority.** Values from the default file override block and project for the same keys. Use this to enforce cluster-wide defaults (e.g. resource limits, service account) that users cannot override. Conflicting values can cause an error on update when strict conflict checking is used. |
| **`false`** | **1. Block** → 2. Project → 3. Default config file (fills gaps only) → 4. Env vars → 5. Hardcoded | **User wins.** Block and project override the policy for the same keys. The policy only fills in keys that block and project did not set. Use this to provide optional defaults while allowing per-block or per-project overrides.                                                                              |

So when **`enforce: true`**, the policy has the **highest** priority; when **`enforce: false`**, block and project are higher and the policy only supplies missing values.

## Environment Variables Reference

### General Configuration

| Variable              | Purpose                                       | Default |
| --------------------- | --------------------------------------------- | ------- |
| `KUBE_NAMESPACE`      | Namespace for job execution                   | default |
| `K8S_CONFIG_FILE`     | Path to full Kubernetes job config file       | —       |
| `MAGE_CONTAINER_NAME` | Container to run Mage in multi-container pods | —       |

### Default Resource Configuration

<ProOnly source="kubernetes-executor" />

These environment variables set default resource limits and requests for all K8s executor jobs.
They are applied when not specified in block-level or project-level configuration.

| Variable                                        | Purpose                                   | Example Value |
| ----------------------------------------------- | ----------------------------------------- | ------------- |
| `K8S_EXECUTOR_DEFAULT_RESOURCE_REQUESTS_CPU`    | Default CPU request for executor jobs     | `250m`        |
| `K8S_EXECUTOR_DEFAULT_RESOURCE_REQUESTS_MEMORY` | Default memory request for executor jobs  | `512Mi`       |
| `K8S_EXECUTOR_DEFAULT_RESOURCE_LIMITS_CPU`      | Default CPU limit for executor jobs       | `500m`        |
| `K8S_EXECUTOR_DEFAULT_RESOURCE_LIMITS_MEMORY`   | Default memory limit for executor jobs    | `1Gi`         |
| `K8S_EXECUTOR_DEFAULT_SERVICE_ACCOUNT_NAME`     | Default service account for executor jobs | `default`     |

**Priority order** (highest to lowest):

1. Block-level `executor_config` in pipeline's `metadata.yaml`
2. Project-level `k8s_executor_config` in project's `metadata.yaml`
3. Environment variable defaults (listed above)
4. Hardcoded defaults