dbt Integration

​

Overview

Build, run, and manage your dbt models with Mage.

​

Analytics engineers

With Mage and dbt combined, you can expand build and run complex data pipelines.

​

Data engineers

Simplify your data infrastructure and reduce the amount of redundant tools in your “modern” data stack. Replace dbt cloud and Airflow with Mage’s native integration with dbt.

​

Features

​

Schedule dbt model runs

Trigger your dbt model runs on a regular schedule, from an event, or via API request.

​

Run specific dbt models and their dependencies

​

Run all models and optionally exclude others

​

dbt models can depend on non-dbt related tasks

Examples:

• Build model after data ingestion from API
• Build model after another pipeline completes

​

Preview dbt model results as you write SQL

​

Build dynamic dbt pipelines using flexible variable interpolation

Examples:

• {{ env_var('...') }}
• {{ variables('...') }}
• {{ mage_secret_var('...') }}

​

Automatically run dbt tests every time a pipeline runs

Write checks using dbt tests, then Mage will run them and fail the pipeline if the test results produce any failures.

​

Observability built-in

Monitor your dbt pipelines and get alerted when things break.

​

Supported connectors

1. dbt-bigquery
2. dbt-core
3. dbt-postgres
4. dbt-redshift
5. dbt-snowflake
6. dbt-sqlserver
7. dbt-trino
8. dbt-spark
9. [coming soon] dbt-dremio
10. [coming soon] dbt-clickhouse

​

Tutorials

• Set up dbt models & orchestrate dbt runs