# Mage AI

## Docs

- [Changelog](https://docs.mage.ai/about/changelog.md): Latest new features and changelog.
- [Code of Conduct](https://docs.mage.ai/about/code-of-conduct.md): We've adopted the [Contributor Covenant](https://www.contributor-covenant.org/) Code of Conduct for the Mage community. Please review the following to understand our standards and expectations for participation.
- [Features](https://docs.mage.ai/about/features.md)
- [Frequently Asked Questions](https://docs.mage.ai/about/frequently-asked-questions.md): Here are some frequently asked questions about Mage and our best answers.
- [Releases](https://docs.mage.ai/about/releases.md): Latest new features and changelog.
- [Roadmap](https://docs.mage.ai/about/roadmap.md)
- [Help improve the tool](https://docs.mage.ai/about/statistics.md): Please contribute usage statistics to help improve the developer experience for you and everyone in the community 🤝.
- [Authentication](https://docs.mage.ai/agent/authentication.md): Log in to your Mage Pro cluster and manage credentials.
- [CLI Reference](https://docs.mage.ai/agent/cli-reference.md): All mage-agent commands and their options.
- [IDE Integration](https://docs.mage.ai/agent/ide-integration.md): Connect Cursor, Claude Code, or OpenAI Codex CLI to your Mage Pro cluster via MCP.
- [Installation](https://docs.mage.ai/agent/installation.md): System requirements and install options for mage-agent.
- [Introduction](https://docs.mage.ai/agent/introduction.md): Build and manage Mage data pipelines directly from your AI coding assistant.
- [MCP Tools](https://docs.mage.ai/agent/mcp-tools.md): Full reference for every tool your AI assistant can call via mage-agent.
- [Quickstart](https://docs.mage.ai/agent/quickstart.md): Install mage-agent, log in, and make your first AI-assisted pipeline call in under five minutes.
- [Syncing Local Files](https://docs.mage.ai/agent/sync.md): Mirror your Mage Pro project to a local directory and push edits back to the cluster.
- [Troubleshooting](https://docs.mage.ai/agent/troubleshooting.md): Solutions for common mage-agent issues.
- [AI Clients](https://docs.mage.ai/ai/ai-client.md): Use and enable different AI clients.
- [Building, Configuring, and Automating with LLMs](https://docs.mage.ai/ai/blocks.md): Learn how to create, configure, and run AI Blocks in Mage to generate text, produce structured outputs, write and validate code, and trigger other pipelines—empowering you to build advanced RAG and automation workflows with ease.
- [Customized AI resources for training and inference](https://docs.mage.ai/ai/custom-resources.md)
- [Machine learning pipeline tutorial](https://docs.mage.ai/ai/ml/train-model.md): Build a machine learning pipeline to train a model on the Titanic dataset.
- [Retrieval Augmented Generation (RAG) pipeline builder](https://docs.mage.ai/ai/rag-pipeline.md): Build and deploy RAG pipelines with ease.
- [Using AI (artificial intelligence)](https://docs.mage.ai/ai/setup.md): Use AI to perform various actions in Mage.
- [Your AI data engineer](https://docs.mage.ai/ai/sidekick/index.md): Upgrade from a co-pilot to a co-commander with your very own AI sidekick for pipelines, analytics, and machine learning.
- [Blog](https://docs.mage.ai/community/blog.md)
- [Contact](https://docs.mage.ai/community/contact.md)
- [GitHub](https://docs.mage.ai/community/github.md)
- [LinkedIn](https://docs.mage.ai/community/linkedin.md)
- [Slack](https://docs.mage.ai/community/slack.md)
- [Twitter](https://docs.mage.ai/community/twitter.md)
- [Adding an IO class](https://docs.mage.ai/contributing/backend/io/adding-a-class.md): IO classes power Mage sources and destinations. Read on to learn how you can contribute an IO class to Mage.
- [Contributing to the backend server](https://docs.mage.ai/contributing/backend/overview.md): Mage backend code is written in Python 🐍 and our server uses the Tornado 🌪️ framework. Here are some guides on adding features to the Mage backend.
- [Contributing](https://docs.mage.ai/contributing/backend/streaming/sources-and-destinations.md)
- [Testing](https://docs.mage.ai/contributing/backend/testing/overview.md): Making Mage so invincible that nothing can break it, not even itself.
- [How to add a chart](https://docs.mage.ai/contributing/charts/how-to-add.md)
- [Adapt an existing source](https://docs.mage.ai/contributing/data-integrations/adapt-existing-source.md): Mage builds data integrations from [Singer taps](https://www.singer.io/#taps), so if there's a Singer source you want to use that isn't supported yet, you can adapt it yourself!
- [Create a new destination](https://docs.mage.ai/contributing/data-integrations/add-new-destination.md)
- [Create a new source](https://docs.mage.ai/contributing/data-integrations/add-new-source.md): This guide details adding a new source to Mage. If your source already exists as a [Singer tap](https://www.singer.io/#taps), check out [our guide](/contributing/data-integrations/adapt-existing-source) for adapting an existing source instead.
- [Configure your development environment](https://docs.mage.ai/contributing/development-environment.md): Want to get started contributing code? You're in the right place! Read on to learn how to set up your development environment.
- [Documentation](https://docs.mage.ai/contributing/documentation/overview.md): Like every other part of Mage, our documentation is open-source. Read on to learn how to make edits or even write entirely new docs for Mage!
- [Contributing to the front-end client](https://docs.mage.ai/contributing/frontend/overview.md): Guides on adding features to the front-end client.
- [Frontend testing](https://docs.mage.ai/contributing/frontend/testing.md): Making Mage invincible so that nothing can break it, not even itself.
- [Contributing](https://docs.mage.ai/contributing/overview.md): We welcome all contributions to Mage; from small UI enhancements to brand new cleaning actions. We love seeing community members level up and give people power-ups!
- [Styling](https://docs.mage.ai/contributing/styling.md)
- [Using data integrations in batch pipelines (in Beta)](https://docs.mage.ai/data-integrations/batch-pipelines.md): Load data from data integration sources and export to data integration destinations as blocks inside batch pipelines.
- [Configure data integration pipelines](https://docs.mage.ai/data-integrations/configuration.md): There are many ways to configure your data integration pipeline.
- [Airtable](https://docs.mage.ai/data-integrations/destinations/airtable.md): How to configure and authenticate the Airtable destination for loading data into Airtable via the API.
- [Amazon S3](https://docs.mage.ai/data-integrations/destinations/amazon_s3.md): How to configure Amazon S3 as a destination in Mage to write data to your S3 bucket in CSV or Parquet format.
- [BigQuery](https://docs.mage.ai/data-integrations/destinations/bigquery.md): How to configure BigQuery as a destination in Mage for loading data into your Google Cloud BigQuery datasets.
- [ClickHouse](https://docs.mage.ai/data-integrations/destinations/clickhouse.md): How to configure ClickHouse as a destination in Mage to write data to a ClickHouse database.
- [Databricks](https://docs.mage.ai/data-integrations/destinations/databricks.md): How to configure Databricks as a destination in Mage to write pipeline data to Databricks SQL warehouses using Unity Catalog.
- [Delta Lake (Azure)](https://docs.mage.ai/data-integrations/destinations/delta_lake_azure.md): How to configure Delta Lake on Azure as a destination in Mage to write data to a Delta table using ABFS.
- [Delta Lake (S3)](https://docs.mage.ai/data-integrations/destinations/delta_lake_s3.md): How to configure Delta Lake on AWS S3 as a destination in Mage to write data to a Delta table using S3 URIs.
- [Doris](https://docs.mage.ai/data-integrations/destinations/doris.md): How to configure Doris as a destination in Mage to write data to a Doris database table, including SSH tunnel and optional connection settings.
- [DuckDB](https://docs.mage.ai/data-integrations/destinations/duckdb.md): How to configure DuckDB as a destination in Mage to write pipeline data to local DuckDB files for fast, embedded analytics.
- [Elasticsearch](https://docs.mage.ai/data-integrations/destinations/elasticsearch.md): How to configure Elasticsearch as a destination in Mage to stream or batch index records into an Elasticsearch cluster using custom index metadata, authentication, and bulk operations.
- [Google Cloud Storage](https://docs.mage.ai/data-integrations/destinations/google_cloud_storage.md): How to configure Google Cloud Storage (GCS) as a destination in Mage to write data files in Parquet or CSV format with optional date partitioning.
- [Iceberg S3](https://docs.mage.ai/data-integrations/destinations/iceberg_s3.md): How to configure Apache Iceberg on S3 as a destination in Mage to write data to ACID-compliant Iceberg tables stored in AWS S3.
- [Kafka](https://docs.mage.ai/data-integrations/destinations/kafka.md): How to configure Apache Kafka as a destination in Mage to publish JSON records to a Kafka topic using optional message keys.
- [MongoDB](https://docs.mage.ai/data-integrations/destinations/mongodb.md): How to configure MongoDB as a destination in Mage to write data to a MongoDB database using a connection string.
- [MSSQL (Microsoft SQL Server)](https://docs.mage.ai/data-integrations/destinations/mssql.md): How to configure Microsoft SQL Server (MSSQL) as a destination in Mage to write pipeline data into a SQL Server database using SQL authentication or Azure AD.
- [MySQL](https://docs.mage.ai/data-integrations/destinations/mysql.md): How to configure MySQL as a destination in Mage to write pipeline data to a MySQL table with optional SSH tunneling and connection parameters.
- [OpenSearch](https://docs.mage.ai/data-integrations/destinations/opensearch.md): How to configure OpenSearch as a destination in Mage to stream pipeline data into OpenSearch indices using JSON records with support for authentication, ECS metadata, and SSL.
- [OracleDB](https://docs.mage.ai/data-integrations/destinations/oracledb.md): How to configure Oracle Database (OracleDB) as a destination in Mage to write data to a specified schema using the thin or thick client.
- [Destinations technical documentation](https://docs.mage.ai/data-integrations/destinations/overview.md): A destination is a system where you want the data to be exported. A destination can be a 3rd party API, SaaS, database, data warehouse, or a data lake.
- [PostgreSQL](https://docs.mage.ai/data-integrations/destinations/postgresql.md): How to configure PostgreSQL as a destination in Mage to write pipeline data into a PostgreSQL table using direct database credentials and optional schema/table controls.
- [Redshift](https://docs.mage.ai/data-integrations/destinations/redshift.md): How to configure Amazon Redshift as a destination in Mage to write data into Redshift tables using user credentials or IAM-based authentication.
- [Salesforce](https://docs.mage.ai/data-integrations/destinations/salesforce.md): How to configure Salesforce as a destination in Mage to write records using OAuth or username/password authentication with support for insert, update, upsert, and bulk actions.
- [Snowflake](https://docs.mage.ai/data-integrations/destinations/snowflake.md): How to configure Snowflake as a destination in Mage to write pipeline data to Snowflake using batch loads, optional key-pair auth, and custom warehouse settings.
- [Teradata](https://docs.mage.ai/data-integrations/destinations/teradata.md): How to configure Teradata as a destination in Mage to write pipeline data to a Teradata database using standard host, port, and user credentials.
- [Trino](https://docs.mage.ai/data-integrations/destinations/trino.md): How to configure Trino as a destination in Mage to write pipeline data to Trino-compatible catalogs using supported connectors such as PostgreSQL, Iceberg, Delta Lake, and more.
- [Overview](https://docs.mage.ai/data-integrations/overview.md): Data integration is the process of synchronizing data between two systems.
- [Airtable](https://docs.mage.ai/data-integrations/sources/airtable.md)
- [Amazon S3](https://docs.mage.ai/data-integrations/sources/amazon_s3.md)
- [Amplitude](https://docs.mage.ai/data-integrations/sources/amplitude.md)
- [API Source Configuration](https://docs.mage.ai/data-integrations/sources/api.md): Learn how to configure API sources in Mage to extract data from RESTful and file-based endpoints. This guide covers authentication, pagination, response parsing, and advanced options available in Mage Pro.
- [Azure Blob Storage](https://docs.mage.ai/data-integrations/sources/azure_blob_storage.md)
- [BigQuery](https://docs.mage.ai/data-integrations/sources/bigquery.md)
- [Chargebee](https://docs.mage.ai/data-integrations/sources/chargebee.md)
- [Commercetools](https://docs.mage.ai/data-integrations/sources/commercetools.md)
- [Couchbase](https://docs.mage.ai/data-integrations/sources/couchbase.md)
- [Custom source](https://docs.mage.ai/data-integrations/sources/custom_source.md): Easily plug in your own data integration sources and destinations with Mage Pro.
- [Datadog](https://docs.mage.ai/data-integrations/sources/datadog.md)
- [Doris](https://docs.mage.ai/data-integrations/sources/doris.md)
- [Dremio](https://docs.mage.ai/data-integrations/sources/dremio.md)
- [Amazon DynamoDB](https://docs.mage.ai/data-integrations/sources/dynamodb.md): Use Mage to extract data from Amazon DynamoDB tables using AWS credentials or IAM roles.
- [Facebook Ads](https://docs.mage.ai/data-integrations/sources/facebook_ads.md)
- [Facebook Pages](https://docs.mage.ai/data-integrations/sources/facebook_pages.md)
- [Freshdesk](https://docs.mage.ai/data-integrations/sources/freshdesk.md)
- [Front](https://docs.mage.ai/data-integrations/sources/front.md)
- [GitHub](https://docs.mage.ai/data-integrations/sources/github.md)
- [Google Ads](https://docs.mage.ai/data-integrations/sources/google_ads.md): Sync campaign, ad group, keyword, and performance report data from the Google Ads API v23 using OAuth 2.0 authentication.
- [Google Analytics](https://docs.mage.ai/data-integrations/sources/google_analytics.md)
- [Google Campaign Manager 360](https://docs.mage.ai/data-integrations/sources/google_campaign_manager_360.md): Sync entity data and Report Builder exports from Campaign Manager 360 using the DFA Reporting & Trafficking API v5 and a service account.
- [Google Cloud Storage](https://docs.mage.ai/data-integrations/sources/google_cloud_storage.md)
- [Google Display & Video 360](https://docs.mage.ai/data-integrations/sources/google_display_video_360.md): Sync DV360 entity metadata and Bid Manager performance report data from Google's Display & Video 360 platform using a service account.
- [Google Search Console](https://docs.mage.ai/data-integrations/sources/google_search_console.md)
- [Google Sheets](https://docs.mage.ai/data-integrations/sources/google_sheets.md)
- [HubSpot](https://docs.mage.ai/data-integrations/sources/hubspot.md)
- [Instagram](https://docs.mage.ai/data-integrations/sources/instagram.md)
- [Intercom](https://docs.mage.ai/data-integrations/sources/intercom.md)
- [Knowi](https://docs.mage.ai/data-integrations/sources/knowi.md)
- [LinkedIn Ads](https://docs.mage.ai/data-integrations/sources/linkedin_ads.md)
- [Microsoft Ads](https://docs.mage.ai/data-integrations/sources/microsoft_ads.md): Sync performance reports from the Microsoft Advertising API (Bing Ads v13) using OAuth 2.0 and the official bingads Python SDK.
- [Mode](https://docs.mage.ai/data-integrations/sources/mode.md)
- [Monday](https://docs.mage.ai/data-integrations/sources/monday.md)
- [MongoDB](https://docs.mage.ai/data-integrations/sources/mongodb.md)
- [MSSQL (Microsoft SQL Server)](https://docs.mage.ai/data-integrations/sources/mssql.md)
- [MySQL](https://docs.mage.ai/data-integrations/sources/mysql.md)
- [OracleDB](https://docs.mage.ai/data-integrations/sources/oracledb.md)
- [Outreach](https://docs.mage.ai/data-integrations/sources/outreach.md)
- [Sources technical documentation](https://docs.mage.ai/data-integrations/sources/overview.md): A source is a system that you want to load data from and synchronize it into another system. A source can be a 3rd party API, SaaS, database, data warehouse, or a data lake.
- [Paystack](https://docs.mage.ai/data-integrations/sources/paystack.md)
- [Pipedrive](https://docs.mage.ai/data-integrations/sources/pipedrive.md)
- [PostgreSQL](https://docs.mage.ai/data-integrations/sources/postgresql.md)
- [Postmark](https://docs.mage.ai/data-integrations/sources/postmark.md)
- [Power BI](https://docs.mage.ai/data-integrations/sources/powerbi.md)
- [QuickBooks](https://docs.mage.ai/data-integrations/sources/quickbooks.md)
- [Amazon Redshift](https://docs.mage.ai/data-integrations/sources/redshift.md)
- [Salesforce](https://docs.mage.ai/data-integrations/sources/salesforce.md)
- [Salesforce Marketing Cloud](https://docs.mage.ai/data-integrations/sources/salesforce_marketing_cloud.md): Sync campaigns, journeys, assets, subscribers, and engagement events from Salesforce Marketing Cloud using the REST and SOAP APIs.
- [SFTP](https://docs.mage.ai/data-integrations/sources/sftp.md)
- [Snowflake](https://docs.mage.ai/data-integrations/sources/snowflake.md)
- [Stripe](https://docs.mage.ai/data-integrations/sources/stripe.md)
- [Tableau](https://docs.mage.ai/data-integrations/sources/tableau.md)
- [Teradata](https://docs.mage.ai/data-integrations/sources/teradata.md)
- [TikTok Ads](https://docs.mage.ai/data-integrations/sources/tiktok_ads.md): Sync ad management objects and performance insight reports from the TikTok Marketing API using a long-lived access token.
- [Twitter Ads](https://docs.mage.ai/data-integrations/sources/twitter_ads.md)
- [Xero](https://docs.mage.ai/data-integrations/sources/xero.md): Sync accounting data from Xero including invoices, contacts, payments, and more. Supports incremental syncing with automatic OAuth token refresh.
- [YouTube Analytics API Source](https://docs.mage.ai/data-integrations/sources/youtube_analytics.md): This guide explains how to configure the YouTube Analytics API as a data source in Mage. Learn how to authenticate with Google Cloud, request the necessary OAuth scopes, and sync data from specific channels.
- [Zendesk](https://docs.mage.ai/data-integrations/sources/zendesk.md)
- [Project Structure](https://docs.mage.ai/design/abstractions/project-structure.md): Details about how the Mage file directory works and how typical projects are structured.
- [Overview](https://docs.mage.ai/design/blocks.md): Each block in a pipeline maps to an individual file within a project. Blocks can be reused and shared across multiple pipelines within the same project.
- [Callback blocks](https://docs.mage.ai/design/blocks/callbacks.md): A callback block is associated to another block. When the parent block succeeds or fails, the callback block functions are executed.
- [Conditional blocks](https://docs.mage.ai/design/blocks/conditionals.md): A conditional block is an 'Add-on Block' associated with another block. The condition will be evaluated before the parent block is executed, determining if the parent block gets executed.
- [Data exporter](https://docs.mage.ai/design/blocks/data-exporter.md): After completing data transformations, utilize the data exporter blocks to either load the processed  data or store a machine learning model in an external data storage system.
- [Data loader](https://docs.mage.ai/design/blocks/data-loader.md): Write your code for fetching data from a remote source or loading it from disk.
- [Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks.md): Use the output of a block to dynamically create more blocks.
- [Scratchpad](https://docs.mage.ai/design/blocks/scratchpad.md): Use these blocks to experiment and write throw away code.
- [Sensor](https://docs.mage.ai/design/blocks/sensor.md): A sensor is a block that continuously evaluates a condition until it’s met or until a period of time has elapsed.
- [Transformer](https://docs.mage.ai/design/blocks/transformer.md): Use these blocks to clean, transform, and enhance data from other blocks.
- [Transformer blocks](https://docs.mage.ai/design/blocks/transformer-blocks.md): Transformer Actions are a library of modular, reusable data transformations, reducing the boilerplate to perform common transformations on your data.
- [Abstractions](https://docs.mage.ai/design/core-abstractions.md): These are the fundamental concepts that Mage uses to operate.
- [Core Design Principles](https://docs.mage.ai/design/core-design-principles.md): Every user experience and technical design decision adheres to these principles.
- [Data loader utilities](https://docs.mage.ai/design/data-loading.md)
- [Pipeline Management](https://docs.mage.ai/design/data-pipeline-management.md)
- [Callback templates](https://docs.mage.ai/development/blocks/callbacks/templates.md)
- [Conditional templates](https://docs.mage.ai/development/blocks/conditionals/templates.md)
- [Data exporter templates](https://docs.mage.ai/development/blocks/data_exporters/templates.md)
- [Data loader templates](https://docs.mage.ai/development/blocks/data_loaders/templates.md)
- [Sensor templates](https://docs.mage.ai/development/blocks/sensors/templates.md)
- [Transformer templates](https://docs.mage.ai/development/blocks/transformers/templates.md)
- [CLI Commands Reference](https://docs.mage.ai/development/cli-commands.md): Complete reference guide for all Mage CLI commands, including usage instructions and OSS vs Pro availability.
- [Database](https://docs.mage.ai/development/databases/default.md)
- [Import your own Python code](https://docs.mage.ai/development/dependencies/custom-files.md): You can import any Python code from any file that is in your Mage project directory.
- [Dependency Management](https://docs.mage.ai/development/dependencies/dependency-management.md): Advanced dependency management with isolated virtual environments, automatic caching, and intelligent package resolution in Mage Pro.
- [Install Python packages using `requirements.txt`](https://docs.mage.ai/development/dependencies/requirements.md): Mage allows you to easily integrate third-party Python packages into your project and pipelines. This guide will walk you through the process of adding and managing external dependencies.
- [Connect a local db in Docker](https://docs.mage.ai/development/docker/connecting-a-database.md): Here's how you can connect a local database to Mage in Docker for development.
- [Run docker image as non-root user](https://docs.mage.ai/development/docker/non-root-user.md): Run Mage docker image as non-root user to restrict access to the system
- [Mage Pro CI/CD overview](https://docs.mage.ai/development/git/pro-overview.md): Deploy your code in seconds!
- [Get started with Git](https://docs.mage.ai/development/git/setting-up-git.md): There are a number of ways to enable version control, collaboration, and easy deployment using Git in Mage. We'll cover them all on this page.
- [IO Config](https://docs.mage.ai/development/io_config.md): The `io_config` YAML file contains the information and credentials required to access databases, data warehouses, and data lakes.
- [IO Config Setup](https://docs.mage.ai/development/io_config_setup.md): Learn how to set up and configure the io_config file for your Mage project
- [Local timezone](https://docs.mage.ai/development/project/local-timezone.md): Display timestamps throughout the app in your local timezone.
- [Project metadata.yaml](https://docs.mage.ai/development/project/metadata.md): Reference for configuring project-level settings in metadata.yaml.
- [Project setup](https://docs.mage.ai/development/project/setup.md): How to set up a folder containing your Mage project code and other files.
- [Configuring Mage settings backend](https://docs.mage.ai/development/settings-backend.md)
- [Troubleshooting](https://docs.mage.ai/development/troubleshooting.md): Debugging common issues in Mage and other frequently asked questions.
- [Updating Mage](https://docs.mage.ai/development/updating-mage.md): How to get the latest and greatest from the Mage team.
- [Mage block variables](https://docs.mage.ai/development/variables/block-variables.md): Use pipeline-specific variables scoped to an individual block.
- [Mage environment variables](https://docs.mage.ai/development/variables/environment-variables.md): Configure Mage settings through environment variables.
- [Mage variables](https://docs.mage.ai/development/variables/overview.md): There are a few ways to set up variables in Mage, here's how you can use them to make the best pipelines the world has ever seen. 🌎
- [Accessing variables in Mage](https://docs.mage.ai/development/variables/referencing-variables.md): You've got all of these great variables and secrets, now let's put them to use. Learn how to access them in your code. 🦸🏻‍♂️
- [Secrets](https://docs.mage.ai/development/variables/secrets.md): Storing and using secrets through Mage
- [Create backfill](https://docs.mage.ai/extensibility/api-reference/backfills/create-backfills.md)
- [Delete backfill](https://docs.mage.ai/extensibility/api-reference/backfills/delete-backfill.md)
- [Overview](https://docs.mage.ai/extensibility/api-reference/backfills/overview.md): Programmatically create multiple pipeline runs to backfill a pipeline.
- [Read backfills](https://docs.mage.ai/extensibility/api-reference/backfills/read-backfills.md)
- [Update backfill](https://docs.mage.ai/extensibility/api-reference/backfills/update-backfill.md)
- [Create block](https://docs.mage.ai/extensibility/api-reference/blocks/create-block.md)
- [Delete block](https://docs.mage.ai/extensibility/api-reference/blocks/delete-block.md)
- [Overview](https://docs.mage.ai/extensibility/api-reference/blocks/overview.md): The Blocks API is used to create, update, and delete blocks objects from a pipeline.
- [Read block](https://docs.mage.ai/extensibility/api-reference/blocks/read-block.md)
- [Update block](https://docs.mage.ai/extensibility/api-reference/blocks/update-block.md)
- [Overview](https://docs.mage.ai/extensibility/api-reference/logs/overview.md): Fetch log entries for your Mage instance.
- [Read logs](https://docs.mage.ai/extensibility/api-reference/logs/read-logs.md)
- [Overview](https://docs.mage.ai/extensibility/api-reference/oauth-access-tokens/overview.md): Fetch OAuth access tokens.
- [Read OAuth tokens](https://docs.mage.ai/extensibility/api-reference/oauth-access-tokens/read-oauth-access-tokens.md)
- [API overview](https://docs.mage.ai/extensibility/api-reference/overview.md): Our API powers everything in the app— our UI makes calls to the backend, which operates the app. You can learn more about how these work and make your own calls in this section.
- [Overview](https://docs.mage.ai/extensibility/api-reference/pipeline-runs/overview.md): Status and metrics for an individual run of a pipeline.
- [Read pipeline runs](https://docs.mage.ai/extensibility/api-reference/pipeline-runs/read-pipeline-runs.md)
- [Trigger pipeline](https://docs.mage.ai/extensibility/api-reference/pipeline-runs/trigger-pipeline.md)
- [Create pipeline schedule](https://docs.mage.ai/extensibility/api-reference/pipeline-schedules/create-pipeline-schedule.md)
- [Overview](https://docs.mage.ai/extensibility/api-reference/pipeline-schedules/overview.md): Triggers for a pipeline.
- [Read pipeline schedules](https://docs.mage.ai/extensibility/api-reference/pipeline-schedules/read-pipeline-schedules.md)
- [Update schedule](https://docs.mage.ai/extensibility/api-reference/pipeline-schedules/update-pipeline-schedule.md)
- [Create pipeline](https://docs.mage.ai/extensibility/api-reference/pipelines/create-pipeline.md)
- [Delete pipeline](https://docs.mage.ai/extensibility/api-reference/pipelines/delete-pipeline.md)
- [Overview](https://docs.mage.ai/extensibility/api-reference/pipelines/overview.md): Data pipeline containing execution settings, resources, block information, and block dependencies.
- [Read pipelines](https://docs.mage.ai/extensibility/api-reference/pipelines/read-pipeline.md)
- [Update pipeline](https://docs.mage.ai/extensibility/api-reference/pipelines/update-pipeline.md)
- [API policies](https://docs.mage.ai/extensibility/api-reference/policies.md): Policies for resources determine what actions can be taken on the resource, what attributes can be read, and which attributes can be written.
- [API presenters](https://docs.mage.ai/extensibility/api-reference/presenters.md): This will determine what attributes of the resource are returned to the client in the response. You can have different sets of attributes be included in the response based on the action (e.g. create, detail, delete, list, update) or a custom format.
- [API resources](https://docs.mage.ai/extensibility/api-reference/resources.md): This file handles the CRUD operations.
- [Create session](https://docs.mage.ai/extensibility/api-reference/sessions/create-session.md)
- [Overview](https://docs.mage.ai/extensibility/api-reference/sessions/overview.md): Sign in and retrieve an OAuth token.
- [Pipeline configuration environment overrides](https://docs.mage.ai/extensibility/env-config/pipeline.md): Add environment-specific overrides for your pipeline configuration.
- [Project configuration environment overrides](https://docs.mage.ai/extensibility/env-config/project.md): Add environment-specific overrides for your project configuration.
- [Global data products](https://docs.mage.ai/extensibility/global-data-products/overview.md): Generate and orchestrate the final output of a pipeline (aka data product). Data products can be globally referenced from any pipeline and its data output can be used in any block.
- [Global Hooks Guide](https://docs.mage.ai/extensibility/global-hooks/guides.md): Follow this step by step guide on creating a global hook in Mage.
- [Global Hooks](https://docs.mage.ai/extensibility/global-hooks/overview.md): Global Hooks are a powerful feature in Mage that allows you to execute custom functions before or after an API request. These hooks can be used to perform additional operations, validate data, or integrate with external systems.
- [Create Deployment](https://docs.mage.ai/extensibility/pro-api-reference/deployments/overview.md)
- [Mage Pro API overview](https://docs.mage.ai/extensibility/pro-api-reference/overview.md)
- [Update Workspace](https://docs.mage.ai/extensibility/pro-api-reference/workspaces/update-workspace.md)
- [Mage Pro data pipeline](https://docs.mage.ai/getting-started/build-pro-pipeline.md): Build your first Mage Pro data pipeline
- [Cluster Management](https://docs.mage.ai/getting-started/cluster-management-portal.md): Manage all your Mage Pro clusters
- [Install dependencies](https://docs.mage.ai/getting-started/install-dependencies.md): Install dependencies using requirements.txt with automatic environment management
- [Python kernels](https://docs.mage.ai/getting-started/kernels.md): We support multiple kernels in the code editor.
- [Mage Pro API](https://docs.mage.ai/getting-started/mage-pro-api.md): Deploy your code in seconds!
- [Pipeline variables and keyword arguments](https://docs.mage.ai/getting-started/runtime-variable.md)
- [Quickstart](https://docs.mage.ai/getting-started/setup.md): Go from zero to Mage hero in under a minute. We'll walk you through installing Mage and running your first pipeline. 🦸‍♀️
- [Mage Pro pipeline trigger](https://docs.mage.ai/getting-started/trigger-pipeline.md): Trigger your first Mage Pro data pipeline
- [Batch pipeline tutorial](https://docs.mage.ai/guides/batch-pipeline.md)
- [Best Practices](https://docs.mage.ai/guides/best-practices.md): Learn best practices for building, maintaining, and scaling data pipelines in Mage Pro.
- [Level 99 dynamic abilities](https://docs.mage.ai/guides/blocks/batch-read-write.md): Granular block settings for controlling read/write data partitions using output size, number of chunks, and item count.
- [Custom blocks](https://docs.mage.ai/guides/blocks/custom-blocks.md): Custom blocks are generic blocks for organizing your pipeline that may not fit into any of the other category of blocks. Unlike Scratchpad blocks, Custom blocks are executed as part of the pipeline.
- [Detaching blocks](https://docs.mage.ai/guides/blocks/detach-blocks.md): Detach other pipeline associations from a block shared by multiple pipelines.
- [Dynamic blocks](https://docs.mage.ai/guides/blocks/dynamic-blocks.md): Create a pipeline with dynamic blocks.
- [Level 99 dynamic abilities](https://docs.mage.ai/guides/blocks/dynamic-blocks-2.md): Execute 100,000+ dynamically created block runs concurrently with upgraded memory management and concurrency.
- [Markdown blocks](https://docs.mage.ai/guides/blocks/markdown-blocks.md): A markdown block is a standalone text block that supports markdown and can be used to make notes or organize your blocks.
- [Native support for Polars](https://docs.mage.ai/guides/blocks/polars.md): Serialize and deserialize data using Polars DataFrames.
- [R blocks](https://docs.mage.ai/guides/blocks/r-blocks.md): You can write R language to transform data in blocks.
- [Reordering blocks](https://docs.mage.ai/guides/blocks/reorder-blocks.md): Reorder blocks in the notebook when editing a pipeline.
- [Replicating blocks](https://docs.mage.ai/guides/blocks/replicate-blocks.md): Reuse the same block multiple times within a single pipeline.
- [Reuse blocks](https://docs.mage.ai/guides/blocks/reuse-blocks.md): Learn how to reuse blocks within pipelines, across pipelines, and across projects in Mage for better code organization and efficiency.
- [SQL blocks](https://docs.mage.ai/guides/blocks/sql-blocks.md): Execute SQL commands directly in your database, data warehouse, etc.
- [Transformer blocks](https://docs.mage.ai/guides/blocks/transformer-blocks.md): Transformer Actions are a library of modular, reusable data transformations, reducing the boilerplate to perform common transformations on your data.
- [User-defined templates](https://docs.mage.ai/guides/blocks/user-defined-templates.md): Learn how to create and use custom block templates and pipeline templates in Mage for better code reusability and standardization.
- [Community Examples](https://docs.mage.ai/guides/community-examples.md): A collection of Mage projects and examples created by our community.
- [How to build a data integration pipeline](https://docs.mage.ai/guides/data-integration-pipeline.md): Here are the high level steps to build a data integration pipeline:
- [Mage Pro CI/CD workflow](https://docs.mage.ai/guides/data-sync/cicd-workflow.md): Deploy and manage your pipeline code safely across environments using Mage Pro's integrated deployment application.
- [Using GitClient in Mage blocks](https://docs.mage.ai/guides/data-sync/git-client.md): Programmatically perform Git operations (add, commit, push, pull) directly within your Mage pipeline blocks using the GitClient class.
- [Git integration for code version control](https://docs.mage.ai/guides/data-sync/git-integration.md): Mage's Git integration provides a way for you to push changes to a remote repository— for example collaborating on a Mage project with your teammates.
- [One-way code sync using Git](https://docs.mage.ai/guides/data-sync/git-sync.md): Enabling one-way Git sync in Mage ensures data will be synced in one direction with a specified git repository.
- [Git terminal with built-in authentication and shortcuts](https://docs.mage.ai/guides/data-sync/git-terminal.md): Mage Pro supports Git-based version control, making it easy to  collaborate, track changes, and manage code across environments.
- [Version control application with Git](https://docs.mage.ai/guides/data-sync/version-control.md): Authenticate with a Git provider then pull from a remote repository, push local changes to a remote repository, and create pull requests for a remote repository.
- [Mage Pro Version Control](https://docs.mage.ai/guides/data-sync/version-control-guide.md): Connect Mage Pro to GitHub, Bitbucket, GitLab, or Azure DevOps using OAuth, HTTPS+PAT, or SSH. Manage data pipelines with Git and deploy commits to your cluster.
- [Add an existing dbt project to Mage](https://docs.mage.ai/guides/dbt/add-existing-dbt.md): If you have an existing dbt project, you can add it to your Mage project.
- [dbt connection profiles](https://docs.mage.ai/guides/dbt/connection-profiles.md)
- [Develop dbt in Mage](https://docs.mage.ai/guides/dbt/developing-dbt-in-mage.md): Learn how to develop your favorite dbt models in Mage.
- [Serve dbt docs in production](https://docs.mage.ai/guides/dbt/docs.md): To serve dbt docs in production, you will need to enable a container to host the dbt docs webserver in the cloud service you are using.
- [Incremental models](https://docs.mage.ai/guides/dbt/incremental-models.md)
- [Use dbt with Mage: Simplify Your Data Transformation Pipelines](https://docs.mage.ai/guides/dbt/overview.md): Learn how Mage integrates with dbt to orchestrate, schedule, and monitor dbt models as part of your data pipelines.
- [Run dbt-spark against a pySpark session.](https://docs.mage.ai/guides/dbt/run-dbt-spark.md)
- [Run multiple models](https://docs.mage.ai/guides/dbt/run-selected-model.md): (and optionally exclude others)
- [Run a single model](https://docs.mage.ai/guides/dbt/run-single-model.md)
- [Running dbt tests](https://docs.mage.ai/guides/dbt/running-dbt-tests.md)
- [Make dbt magical](https://docs.mage.ai/guides/dbt/setup-dbt.md): ⏰ Run dbt models in Mage in under a minute.
- [Run dbt snapshot](https://docs.mage.ai/guides/dbt/snapshots.md): Run multiple individual snapshots or bulk snapshots
- [dbt sources and upstream dependencies](https://docs.mage.ai/guides/dbt/sources.md)
- [dbt variable interpolation](https://docs.mage.ai/guides/dbt/variable-interpolation.md)
- [Block dependency tree](https://docs.mage.ai/guides/developer-ux/block-dependency-tree.md): Money does grow on trees... if you plant them with magic beans.
- [Code editor with enhanced productivity](https://docs.mage.ai/guides/developer-ux/code-browser.md): Code browser and editor with syntax highlighting, code formatting, and version history.
- [Pipeline editor canvas](https://docs.mage.ai/guides/developer-ux/code-canvas.md): Build, run, and manage complex pipelines with the new canvas mode.
- [Writing data pipeline code](https://docs.mage.ai/guides/developer-ux/code-editor.md): Build the future you want to see.
- [Browse all files](https://docs.mage.ai/guides/developer-ux/file-browser.md): Add, edit, move, and delete files or folders in your project.
- [Personalize your workspace](https://docs.mage.ai/guides/developer-ux/personalize.md): Customize workspace themes, personalize accounts, and re-configure UI component layouts.
- [Cross-pipeline dependencies](https://docs.mage.ai/guides/developer-ux/pipeline-dependencies.md): Manage cross-pipeline dependencies and execution flow across every pipeline within a project.
- [Pipeline sidekick](https://docs.mage.ai/guides/developer-ux/sidekick.md): Have you ever wished you had a sidekick like Batman and Robin? Now you have one.
- [Develop Mage Pro Code with VS Code or Cursor](https://docs.mage.ai/guides/developer-ux/visual-studio-code.md): Learn how to develop Mage Pro pipelines and projects using Visual Studio Code or Cursor. This guide walks you through secure remote access via Tailscale and SSH, environment configuration, and integrating your local IDE with Mage Pro.
- [Mage Pro workspaces](https://docs.mage.ai/guides/developer-ux/workspaces.md): Set up Mage Pro workspaces for organizing your own personal environment.
- [Configure Git](https://docs.mage.ai/guides/git/configure.md): This page will walk you through the process of authenticating with SSH or HTTPS and configuring Git for use with Mage Git Sync.
- [ETL pipeline tutorial](https://docs.mage.ai/guides/load-api-data.md): Build a data pipeline that loads restaurant data, transforms it, then exports it to a DuckDB database. 🦆
- [Mage Guides](https://docs.mage.ai/guides/overview.md): Mage guides for getting started, building pipelines, and more.
- [Importing pipelines](https://docs.mage.ai/guides/pipelines/importing-pipelines.md): Import a pipeline zip file into your project.
- [ActiveMQ](https://docs.mage.ai/guides/streaming/destinations/activemq.md)
- [Amazon S3](https://docs.mage.ai/guides/streaming/destinations/amazon-s3.md)
- [Azure Data Lake](https://docs.mage.ai/guides/streaming/destinations/azure_data_lake.md)
- [BigQuery](https://docs.mage.ai/guides/streaming/destinations/bigquery.md): Stream data to Google BigQuery using the Storage Write API, with optional dead-letter queue and metadata interpolation for CDC.
- [ClickHouse](https://docs.mage.ai/guides/streaming/destinations/clickhouse.md)
- [DuckDB](https://docs.mage.ai/guides/streaming/destinations/duckdb.md)
- [Dummy](https://docs.mage.ai/guides/streaming/destinations/dummy.md)
- [Elasticsearch](https://docs.mage.ai/guides/streaming/destinations/elasticsearch.md)
- [Google Cloud PubSub](https://docs.mage.ai/guides/streaming/destinations/google-cloud-pubsub.md)
- [Google Cloud Storage](https://docs.mage.ai/guides/streaming/destinations/google-cloud-storage.md)
- [InfluxDB](https://docs.mage.ai/guides/streaming/destinations/influxdb.md)
- [Kafka](https://docs.mage.ai/guides/streaming/destinations/kafka.md)
- [Kinesis](https://docs.mage.ai/guides/streaming/destinations/kinesis.md)
- [MongoDB](https://docs.mage.ai/guides/streaming/destinations/mongodb.md)
- [Microsoft SQL Server](https://docs.mage.ai/guides/streaming/destinations/mssql.md)
- [MySQL](https://docs.mage.ai/guides/streaming/destinations/mysql.md)
- [Opensearch](https://docs.mage.ai/guides/streaming/destinations/opensearch.md)
- [Oracle Database](https://docs.mage.ai/guides/streaming/destinations/oracledb.md)
- [Postgres](https://docs.mage.ai/guides/streaming/destinations/postgres.md)
- [RabbitMQ](https://docs.mage.ai/guides/streaming/destinations/rabbitmq.md)
- [Redshift](https://docs.mage.ai/guides/streaming/destinations/redshift.md)
- [Snowflake](https://docs.mage.ai/guides/streaming/destinations/snowflake.md)
- [Trino](https://docs.mage.ai/guides/streaming/destinations/trino.md)
- [Overview](https://docs.mage.ai/guides/streaming/introduction.md)
- [ActiveMQ](https://docs.mage.ai/guides/streaming/sources/activemq.md): This guide will walk you through the process of setting up and using ActiveMQ with Mage for streaming data pipelines.
- [Amazon SQS](https://docs.mage.ai/guides/streaming/sources/amazon-sqs.md)
- [Azure Event Hub](https://docs.mage.ai/guides/streaming/sources/azure-event-hub.md): This guide provides step-by-step instructions to set up and use Azure Event Hub with Mage. By following these instructions,  you can create a new streaming pipeline in Mage, authenticate with Azure Event Hub, and configure your data loader, transformer,  and exporter blocks to process and manage you…
- [Google Cloud PubSub](https://docs.mage.ai/guides/streaming/sources/google-cloud-pubsub.md): Ingest data from Google Cloud PubSub event streaming sources.
- [InfluxDB](https://docs.mage.ai/guides/streaming/sources/influxdb.md)
- [Kafka](https://docs.mage.ai/guides/streaming/sources/kafka.md)
- [Kinesis](https://docs.mage.ai/guides/streaming/sources/kinesis.md)
- [MongoDB](https://docs.mage.ai/guides/streaming/sources/mongodb.md)
- [Microsoft SQL Server CDC](https://docs.mage.ai/guides/streaming/sources/mssql-cdc.md): Real-time Change Data Capture from SQL Server CDC change tables with automatic schema tracking
- [MySQL CDC](https://docs.mage.ai/guides/streaming/sources/mysql-cdc.md): Real-time Change Data Capture from MySQL binary logs with schema evolution tracking
- [NATS.io - JetStream](https://docs.mage.ai/guides/streaming/sources/nats.md)
- [OracleDB](https://docs.mage.ai/guides/streaming/sources/oracledb.md): Stream changes from OracleDB with OracleDB CDC.
- [PostgreSQL CDC](https://docs.mage.ai/guides/streaming/sources/postgres-cdc.md): Real-time Change Data Capture from PostgreSQL WAL with logical replication
- [RabbitMQ](https://docs.mage.ai/guides/streaming/sources/rabbitmq.md)
- [🎏 Local stream to dbt transform](https://docs.mage.ai/guides/streaming/tutorials/magic-devcontainer.md): This project uses sample data from the Google NYC Taxi Pubsub to create a streaming pipeline in Mage that reads and transforms a sample stream, ultimately writing output to a SCD Type-2 table in Postgres.
- [Kafka streaming pipeline](https://docs.mage.ai/guides/streaming/tutorials/streaming-pipeline.md): Build pipelines that ingest data from event streaming sources like Kafka.
- [RabbitMQ streaming pipeline](https://docs.mage.ai/guides/streaming/tutorials/streaming-pipeline-rabbitmq.md): Build pipelines that ingest data from event streaming sources like RabbitMQ.
- [Stateful Streaming Pipeline](https://docs.mage.ai/guides/streaming/tutorials/streaming-stateful-store.md): Create and use stateful store in a streaming pipeline.
- [Mage tips & tricks](https://docs.mage.ai/guides/tips-and-tricks.md): Here are some neat ways you can make the most of Mage! Tips and tricks to help you make the most magical pipelines possible.
- [Use a completed pipeline](https://docs.mage.ai/guides/train/complete-project.md)
- [File versioning and history](https://docs.mage.ai/guides/version-control/file-versions.md): Local file edit tracking and version history for restoring changes made in the past.
- [Airbyte in Mage](https://docs.mage.ai/integrations/airbyte.md): Trigger a connection sync in Airbyte.
- [Run Mage pipelines in Airflow](https://docs.mage.ai/integrations/airflow/index.md)
- [Integrate Mage into an existing Airflow project](https://docs.mage.ai/integrations/airflow/integrate-mage-airflow.md): In this tutorial, we’ll create a DAG in Airflow for scheduling and running a data pipeline; all from the Mage UI.
- [Databricks](https://docs.mage.ai/integrations/compute/databricks.md): This is a guide for using Databricks Spark cluster with Mage.
- [Compute management for Spark](https://docs.mage.ai/integrations/compute/management.md): Manage your Spark compute resources and track Spark pipeline execution metrics.
- [PySpark in Mage Pro](https://docs.mage.ai/integrations/compute/spark-pro.md): This guide explains how to use PySpark in Mage Pro, including how to run PySpark blocks in batch pipelines, connect to Spark on Kubernetes, manage dependencies like Apache Iceberg, and access cloud storage securely.
- [Spark and PySpark](https://docs.mage.ai/integrations/compute/spark-pyspark.md): This is a guide for using Spark (PySpark) with Mage in different cloud providers or Kubernetes cluster.
- [Algolia Integration in Mage](https://docs.mage.ai/integrations/databases/Algolia.md)
- [Athena](https://docs.mage.ai/integrations/databases/Athena.md)
- [Azure Blob Storage](https://docs.mage.ai/integrations/databases/AzureBlobStorage.md)
- [Azure Data Lake Storage Gen2](https://docs.mage.ai/integrations/databases/AzureDataLakeStorage.md)
- [BigQuery](https://docs.mage.ai/integrations/databases/BigQuery.md)
- [Chroma](https://docs.mage.ai/integrations/databases/Chroma.md)
- [ClickHouse](https://docs.mage.ai/integrations/databases/ClickHouse.md)
- [Databricks](https://docs.mage.ai/integrations/databases/Databricks.md)
- [Druid](https://docs.mage.ai/integrations/databases/Druid.md)
- [DuckDB](https://docs.mage.ai/integrations/databases/DuckDB.md): Execute SQL commands in DuckDB.
- [Google Cloud Storage](https://docs.mage.ai/integrations/databases/GoogleCloudStorage.md)
- [Google Sheets](https://docs.mage.ai/integrations/databases/GoogleSheets.md): Mage supports writing/reading from Google Sheets using the Google Sheets API via the gspread library. Read on to learn about reading/writing data to Sheets.
- [Iceberg](https://docs.mage.ai/integrations/databases/Iceberg.md): Connect to Apache Iceberg tables stored in Amazon S3 using various catalog types
- [Microsoft Fabric Warehouse](https://docs.mage.ai/integrations/databases/MicrosoftFabricWarehouse.md)
- [Microsoft SQL Server](https://docs.mage.ai/integrations/databases/MicrosoftSQLServer.md)
- [MongoDB](https://docs.mage.ai/integrations/databases/MongoDB.md)
- [MotherDuck](https://docs.mage.ai/integrations/databases/MotherDuck.md): Execute SQL commands against a MotherDuck database.
- [MySQL](https://docs.mage.ai/integrations/databases/MySQL.md)
- [OracleDB](https://docs.mage.ai/integrations/databases/OracleDB.md)
- [Pinot](https://docs.mage.ai/integrations/databases/Pinot.md)
- [PostgreSQL](https://docs.mage.ai/integrations/databases/PostgreSQL.md)
- [Power BI](https://docs.mage.ai/integrations/databases/PowerBI.md)
- [Qdrant](https://docs.mage.ai/integrations/databases/Qdrant.md)
- [Redshift](https://docs.mage.ai/integrations/databases/Redshift.md)
- [S3](https://docs.mage.ai/integrations/databases/S3.md)
- [SQLite](https://docs.mage.ai/integrations/databases/SQLite.md)
- [Snowflake](https://docs.mage.ai/integrations/databases/Snowflake.md)
- [Spark](https://docs.mage.ai/integrations/databases/Spark.md)
- [Weaviate](https://docs.mage.ai/integrations/databases/Weaviate.md)
- [Trino SQL blocks](https://docs.mage.ai/integrations/databases/trino.md): Execute SQL commands in Trino.
- [dbt Cloud in Mage](https://docs.mage.ai/integrations/dbt-cloud.md): Trigger model runs in dbt Cloud.
- [Hightouch in Mage](https://docs.mage.ai/integrations/hightouch.md): Trigger syncs in Hightouch.
- [Run Mage pipelines in Prefect](https://docs.mage.ai/integrations/prefect.md): We support running the pipeline in Prefect flows.
- [Stitch in Mage](https://docs.mage.ai/integrations/stitch.md): Trigger syncs in Stitch.
- [StrongDM](https://docs.mage.ai/integrations/strongdm.md)
- [Using an external IDE](https://docs.mage.ai/integrations/text-editor.md): We love our UI, but here's how to develop in your favorite IDE, too.
- [It’s magic.](https://docs.mage.ai/introduction/about-mage-pro.md): Give your data team magical powers.
- [It’s magic.](https://docs.mage.ai/introduction/overview.md): Give your data team magical powers.
- [Migrate from Airbyte to Mage Pro](https://docs.mage.ai/migrations/airbyte-to-mage-pro.md)
- [Migrate from Airflow to Mage Pro](https://docs.mage.ai/migrations/airflow-to-mage-pro.md)
- [Migrate from Dagster to Mage Pro](https://docs.mage.ai/migrations/dagster-to-mage-pro.md)
- [Migrate from dbt to Mage Pro](https://docs.mage.ai/migrations/dbt-to-mage-pro.md)
- [Migrate from Estuary to Mage Pro](https://docs.mage.ai/migrations/estuary-to-mage-pro.md)
- [Migrate from Fivetran to Mage Pro](https://docs.mage.ai/migrations/fivetran-to-mage-pro.md)
- [Migrate from Mage OSS to Mage Pro](https://docs.mage.ai/migrations/mage-oss-to-mage-pro.md)
- [Migrate from Prefect to Mage Pro](https://docs.mage.ai/migrations/prefect-to-mage-pro.md)
- [Alerting status updates in Discord](https://docs.mage.ai/observability/alerting/alerting-discord.md): Get status updates in your Discord channel.
- [Alerting status updates in Email](https://docs.mage.ai/observability/alerting/alerting-email.md): Get status updates sent to your email inbox.
- [Alerting status updates in Opsgenie](https://docs.mage.ai/observability/alerting/alerting-opsgenie.md): Get status updates in Opsgenie.
- [Alerting status updates in Slack](https://docs.mage.ai/observability/alerting/alerting-slack.md): Get real-time status updates from Mage pipelines directly in your Slack channels.
- [Alerting status updates in Teams](https://docs.mage.ai/observability/alerting/alerting-teams.md): Get status updates in your Teams channel.
- [Alerting status updates in Telegram](https://docs.mage.ai/observability/alerting/alerting-telegram.md): Get status updates in your Telegram Group.
- [Alerting status updates in Zendesk](https://docs.mage.ai/observability/alerting/alerting-zendesk.md): Create Zendesk tickets from Mage pipeline events to track incidents and resolution.
- [Writing metrics to Datadog](https://docs.mage.ai/observability/external/datadog.md)
- [Metaplane in Mage](https://docs.mage.ai/observability/external/metaplane.md): Run monitors in Metaplane.
- [Monitoring with New Relic](https://docs.mage.ai/observability/external/newrelic.md)
- [Monitoring with OpenTelemetry](https://docs.mage.ai/observability/external/opentelemetry.md)
- [Monitoring with Prometheus](https://docs.mage.ai/observability/external/prometheus.md)
- [Monitoring with Sentry](https://docs.mage.ai/observability/external/sentry.md)
- [Logging](https://docs.mage.ai/observability/logging.md)
- [Monitoring](https://docs.mage.ai/observability/monitoring.md): You can monitor many metrics for each of your pipelines and blocks.
- [Data validation](https://docs.mage.ai/observability/testing/data-validation.md): Every data loader and transformer block has data validation capabilities built-in.
- [Unit tests](https://docs.mage.ai/observability/testing/unit-tests.md): Write unit tests to test pipeline code.
- [Customizable dashboards](https://docs.mage.ai/observability/visualizations/dashboards.md): There are 2 dashboards: a dashboard for all your pipelines and a dashboard for each pipeline. You can add charts of various types with different sources of data. Use these dashboards for observability or for analytics.
- [Backfill guide](https://docs.mage.ai/orchestration/backfills/guides.md): How to create a backfill.
- [Backfilling pipelines](https://docs.mage.ai/orchestration/backfills/overview.md): Run a pipeline multiple times.
- [Query pipeline run metadata from database](https://docs.mage.ai/orchestration/pipeline-runs/database-query.md)
- [Pipeline Run](https://docs.mage.ai/orchestration/pipeline-runs/overview.md): Learn how to monitor, manage, and troubleshoot pipeline runs in Mage for optimal data pipeline performance.
- [Retrying block runs from a pipeline run](https://docs.mage.ai/orchestration/pipeline-runs/retrying-block-runs.md)
- [Saving block output as CSV file](https://docs.mage.ai/orchestration/pipeline-runs/saving-block-output-as-csv.md)
- [Configure triggers in code](https://docs.mage.ai/orchestration/triggers/configure-triggers-in-code.md): Configure triggers in triggers.yaml under pipeline folder.
- [Query trigger metadata from database](https://docs.mage.ai/orchestration/triggers/database-query.md)
- [Schedules](https://docs.mage.ai/orchestration/triggers/schedule-pipelines.md): Schedule pipelines to run periodically
- [Trigger pipeline from a block](https://docs.mage.ai/orchestration/triggers/trigger-pipeline.md): You can trigger another pipeline from a block within a different pipeline.
- [Triggering pipeline via API request](https://docs.mage.ai/orchestration/triggers/trigger-pipeline-api.md): You can trigger a pipeline by making an API request.
- [Pipeline Trigger Overview](https://docs.mage.ai/orchestration/triggers/triggering-pipelines.md): Learn how to configure and manage pipeline triggers in Mage
- [Pipeline filtering and tagging](https://docs.mage.ai/pipelines/pipeline-tagging.md): How to organize, filter, group, and tag pipelines in Mage.
- [Deployment architecture overview](https://docs.mage.ai/pro-architecture/architecture-overview.md): Deploy Mage Pro in any environment
- [Hybrid cloud deployment](https://docs.mage.ai/pro-architecture/hybrid-cloud.md): Deploy the data plane in your cloud environment
- [On-prem deployment](https://docs.mage.ai/pro-architecture/on-prem.md): Deploy securely from your own on-prem servers
- [Private cloud deployment](https://docs.mage.ai/pro-architecture/private-cloud.md): Deploy securely in your private cloud environment
- [Managed cloud](https://docs.mage.ai/pro-architecture/saas.md): Fully managed Mage Pro
- [Sign in with Google](https://docs.mage.ai/production/authentication/google.md): Enable signing in with a Google account in Mage.
- [Lightweight Directory Access Protocol](https://docs.mage.ai/production/authentication/ldap.md): Sign in and authenticate users using LDAP
- [Sign in with Microsoft Entra ID aka Active Directory](https://docs.mage.ai/production/authentication/microsoft.md): Enable signing in with a Microsoft account in Mage.
- [Sign in with OIDC](https://docs.mage.ai/production/authentication/oidc.md): Enable signing in with OIDC in Mage.
- [Sign in with Okta](https://docs.mage.ai/production/authentication/okta.md): Enable signing in with Okta in Mage.
- [User authentication](https://docs.mage.ai/production/authentication/overview.md): Create users, manage users, and require sign in to authenticate and use Mage.
- [Role-based access controls (RBAC)](https://docs.mage.ai/production/authentication/permissions/overview.md): Create user defined permissions for CRUD operations on any action or user experience. Add permissions to roles and add roles to users.
- [Permissions](https://docs.mage.ai/production/authentication/permissions/permissions.md): Create granular permissions for CRUD operations on any API endpoint.
- [Roles](https://docs.mage.ai/production/authentication/permissions/roles.md): Add permissions to roles and add roles to users.
- [Cloud cloud](https://docs.mage.ai/production/ci-cd/cloud-cloud.md)
- [Buildkite deployments](https://docs.mage.ai/production/ci-cd/local-cloud/buildkite.md): CI/CD with Buildkite.
- [GitHub Actions](https://docs.mage.ai/production/ci-cd/local-cloud/github-actions.md): Development (local) and production (cloud) using GitHub Actions.
- [GitLab deployments](https://docs.mage.ai/production/ci-cd/local-cloud/gitlab-ci-cd.md): Development (local) and production (cloud) using GitLab CI/CD.
- [Repository setup](https://docs.mage.ai/production/ci-cd/local-cloud/repository-setup.md): Setup your Mage project repository.
- [Continuous deployment with Mage Pro](https://docs.mage.ai/production/ci-cd/mage/pro.md): Manage, deploy, and rollback new code changes, from a remote repository to any environment, all from within the Mage platform.
- [Overview](https://docs.mage.ai/production/ci-cd/overview.md): There are 4 development and deployment workflows.
- [GitHub Actions](https://docs.mage.ai/production/ci-cd/staging-production/github-actions.md): Development (local), staging (cloud), and production (cloud) using GitHub Actions
- [Compute resources](https://docs.mage.ai/production/configuring-production-settings/compute-resource.md)
- [Production Configuration](https://docs.mage.ai/production/configuring-production-settings/overview.md)
- [Architecture](https://docs.mage.ai/production/deploying-to-cloud/architecture.md)
- [Mage and Native Cloud Environments](https://docs.mage.ai/production/deploying-to-cloud/aws-without-terraform.md): Just like a traditional notebook, Mage supports execution in native cloud environments. Below are guides on how to integrate with native cloud resources.
- [AWS ECS deployment architecture](https://docs.mage.ai/production/deploying-to-cloud/aws/aws-ecs-architecture.md)
- [Deploy to AWS ECS with AWS CodePipeline](https://docs.mage.ai/production/deploying-to-cloud/aws/code-pipeline.md)
- [EMR policies](https://docs.mage.ai/production/deploying-to-cloud/aws/emr-policy.md)
- [Terraform AWS resources](https://docs.mage.ai/production/deploying-to-cloud/aws/resources.md)
- [Deploy to AWS with Terraform](https://docs.mage.ai/production/deploying-to-cloud/aws/setup.md)
- [AWS policies for Terraform apply](https://docs.mage.ai/production/deploying-to-cloud/aws/terraform-apply-policy.md)
- [AWS policies for Terraform destroy](https://docs.mage.ai/production/deploying-to-cloud/aws/terraform-destroy-policy.md)
- [Terraform Azure resources](https://docs.mage.ai/production/deploying-to-cloud/azure/resources.md)
- [Deploy to Azure with Terraform](https://docs.mage.ai/production/deploying-to-cloud/azure/setup.md)
- [Deploy to DigitalOcean with Terraform](https://docs.mage.ai/production/deploying-to-cloud/digitalocean/setup.md)
- [Push Docker Image to GCP Artifact Registry](https://docs.mage.ai/production/deploying-to-cloud/gcp/gcp-artifact-registry.md)
- [Terraform GCP resources](https://docs.mage.ai/production/deploying-to-cloud/gcp/resources.md)
- [Deploy to GCP with Terraform](https://docs.mage.ai/production/deploying-to-cloud/gcp/setup.md)
- [AWS Secrets Manager](https://docs.mage.ai/production/deploying-to-cloud/secrets/AWS.md): Attention Magers: This classified document contains vital intel on securing sensitive data within the AWS Secrets Manager vault. Study the protocols carefully to create and operationalize confidential secrets, granting you clearance to integrate them into your Mage data pipeline projects.
- [Azure Key Vault](https://docs.mage.ai/production/deploying-to-cloud/secrets/Azure.md)
- [GCP Secret Management](https://docs.mage.ai/production/deploying-to-cloud/secrets/GCP.md)
- [HashiCorp Vault](https://docs.mage.ai/production/deploying-to-cloud/secrets/Vault.md): Securely manage and inject sensitive credentials into your Mage pipelines using HashiCorp Vault. This feature helps enterprises meet security and compliance standards by keeping secrets like API keys, database passwords, and tokens outside of source code.
- [Mage’s built-in secret manager](https://docs.mage.ai/production/deploying-to-cloud/secrets/mage.md): Storing and using secrets through Mage.
- [Helm](https://docs.mage.ai/production/deploying-to-cloud/using-helm.md)
- [Terraform](https://docs.mage.ai/production/deploying-to-cloud/using-terraform.md)
- [AWS ECS executor](https://docs.mage.ai/production/executors/aws.md): Execute block runs in separate tasks.
- [Azure Container Instance executor](https://docs.mage.ai/production/executors/azure.md): Execute block runs in separate container instances.
- [GCP Cloud Run executor](https://docs.mage.ai/production/executors/gcp.md): Execute block runs in separate jobs.
- [Kubernetes executor](https://docs.mage.ai/production/executors/k8.md): Run Mage pipeline blocks in separate Kubernetes pods for scalability, isolation, and resource control.
- [Local python executor](https://docs.mage.ai/production/executors/python.md): Local python executors are ran within the same container as the scheduler service.
- [PySpark executor](https://docs.mage.ai/production/executors/spark.md)
- [Mage Pro](https://docs.mage.ai/production/mage/pro.md): A powerful, cloud-native solution to orchestrate and scale your data pipelines — available as fully managed, private, or hybrid cloud deployments.