Read logs - Mage AI

curl --request GET \
  --url 'http://localhost:6789/api/pipelines/example_pipeline/logs?_limit=20&_offset=0&start_timestamp=1677648116&exclude_source_category[]=kubernetes&api_key=zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2' \
  --header 'OAUTH-TOKEN=some_really_long_string' \

{
  "logs": [
    {
      "block_run_logs": [
        {
          "name": "solitary_brook.log",
          "path": "/root/.mage_data/default_repo/pipelines/example_pipeline/.logs/54/20230308T045259/solitary_brook.log",
          "content": "very_long_content"
        },
        {
          "name": "load_titanic.log",
          "path": "/root/.mage_data/default_repo/pipelines/example_pipeline/.logs/54/20230308T045259/load_titanic.log",
          "content": "very_long_content"
        }
      ],
      "pipeline_run_logs": [
        {
          "name": "pipeline.log",
          "path": "/root/.mage_data/default_repo/pipelines/example_pipeline/.logs/55/20230308T045408/pipeline.log",
          "content": "very_long_content \"timestamp\": 1678251278.430712, \"uuid\": \"3de507ebca894100b20ae4a1645724d7\"}\n"
        },
        {
          "name": "pipeline.log",
          "path": "very_long_content"
        }
      ],
      "fallback_from_disk": true,
      "source_filter_fallback_reason": "opensearch_unavailable",
      "source_filters_applied": false,
      "total_block_run_log_count": 2,
      "total_pipeline_run_log_count": 2
    }
  ],
  "metadata": {}
}

GET /api/pipelines/:pipeline_uuid/logs

string

required

Pipeline UUID of the logs.Example: example_pipeline

integer

Maximum number of logs to be returned in the API response.Example: 20

integer

Read logs after N number of logs, where N equals _offset.Example: 10

integer

Read logs on or after a specified timestamp.Example: 1677648116

integer

Read logs on or before a specified timestamp.Example: 1678252916

array of strings

Read logs matching one or more log levels.Example: INFO

array of strings

Read Mage Pro OpenSearch-backed logs from one or more source categories. Accepted values are user_code, mage_scheduler, executor, connector_library, kubernetes, system, and unknown.Example: user_code

array of strings

Exclude Mage Pro OpenSearch-backed logs from one or more source categories. To hide infrastructure logs, pass mage_scheduler, executor, kubernetes, and system.Example: kubernetes

Source category filters require Mage Pro OpenSearch-backed log search. When the API falls back to disk logs, responses with active source filters include fallback_from_disk: true, source_filters_applied: false, and source_filter_fallback_reason: "opensearch_unavailable". Legacy OpenSearch documents are classified from conservative structured fields when possible; unclassified entries remain unknown and are included in unfiltered results. Unsupported source category values return no OpenSearch-backed rows for that request instead of being ignored.

integer

Pass 1 to hide scheduler health-check log entries.Example: 1

curl --request GET \
  --url 'http://localhost:6789/api/pipelines/example_pipeline/logs?_limit=20&_offset=0&start_timestamp=1677648116&exclude_source_category[]=kubernetes&api_key=zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2' \
  --header 'OAUTH-TOKEN=some_really_long_string' \

{
  "logs": [
    {
      "block_run_logs": [
        {
          "name": "solitary_brook.log",
          "path": "/root/.mage_data/default_repo/pipelines/example_pipeline/.logs/54/20230308T045259/solitary_brook.log",
          "content": "very_long_content"
        },
        {
          "name": "load_titanic.log",
          "path": "/root/.mage_data/default_repo/pipelines/example_pipeline/.logs/54/20230308T045259/load_titanic.log",
          "content": "very_long_content"
        }
      ],
      "pipeline_run_logs": [
        {
          "name": "pipeline.log",
          "path": "/root/.mage_data/default_repo/pipelines/example_pipeline/.logs/55/20230308T045408/pipeline.log",
          "content": "very_long_content \"timestamp\": 1678251278.430712, \"uuid\": \"3de507ebca894100b20ae4a1645724d7\"}\n"
        },
        {
          "name": "pipeline.log",
          "path": "very_long_content"
        }
      ],
      "fallback_from_disk": true,
      "source_filter_fallback_reason": "opensearch_unavailable",
      "source_filters_applied": false,
      "total_block_run_log_count": 2,
      "total_pipeline_run_log_count": 2
    }
  ],
  "metadata": {}
}

curl --request GET \
  --url 'http://localhost:6789/api/pipelines/example_pipeline/logs?_limit=20&_offset=0&start_timestamp=1677648116&exclude_source_category[]=kubernetes&api_key=zkWlN0PkIKSN0C11CfUHUj84OT5XOJ6tDZ6bDRO2' \
  --header 'OAUTH-TOKEN=some_really_long_string' \

{
  "logs": [
    {
      "block_run_logs": [
        {
          "name": "solitary_brook.log",
          "path": "/root/.mage_data/default_repo/pipelines/example_pipeline/.logs/54/20230308T045259/solitary_brook.log",
          "content": "very_long_content"
        },
        {
          "name": "load_titanic.log",
          "path": "/root/.mage_data/default_repo/pipelines/example_pipeline/.logs/54/20230308T045259/load_titanic.log",
          "content": "very_long_content"
        }
      ],
      "pipeline_run_logs": [
        {
          "name": "pipeline.log",
          "path": "/root/.mage_data/default_repo/pipelines/example_pipeline/.logs/55/20230308T045408/pipeline.log",
          "content": "very_long_content \"timestamp\": 1678251278.430712, \"uuid\": \"3de507ebca894100b20ae4a1645724d7\"}\n"
        },
        {
          "name": "pipeline.log",
          "path": "very_long_content"
        }
      ],
      "fallback_from_disk": true,
      "source_filter_fallback_reason": "opensearch_unavailable",
      "source_filters_applied": false,
      "total_block_run_log_count": 2,
      "total_pipeline_run_log_count": 2
    }
  ],
  "metadata": {}
}