Overview
Data integration is the process of synchronizing data between two systems.
Requirements
Data integrations are only supported when running Mage using Docker.
Use cases
Syncing external data → internal systems
If you have data in 3rd party systems (e.g other SaaS tools) and you want that data available in your database, data warehouse, or data lake, you can use Mage to sync data between those systems.
Syncing internal data → external systems (reverse)
Your internal database or data warehouse may contain application data generated by your users or customers. This data can be valuable if used in conjunction with 3rd party SaaS tools.
You can use Mage to sync data from your internal systems, and have that data appear in various 3rd party SaaS tools.
How to build a data integration pipeline
Sources
A source is a system that you want to load data from and synchronize it into another system. A source can be a 3rd party API, SaaS, database, data warehouse, or a data lake.
Read more about Sources:
Available sources
- Amazon S3
- Amplitude
- API
- Azure Blob Storage
- BigQuery
- Chargebee
- Commercetools
- Couchbase
- Datadog
- Dremio
- DynamoDB
- Facebook Ads
- Freshdesk
- Front
- Google Ads
- Google Analytics
- Google Cloud Storage
- Google Search Console
- Google Sheets
- HubSpot
- Intercom
- Knowi
- LinkedIn Ads
- Microsoft SQL Server
- Mode
- Monday
- MongoDB
- MySQL
- OracleDB
- Outreach
- Paystack
- Pipedrive
- PostgreSQL
- Postmark
- PowerBI
- Redshift (Amazon Web Services)
- Salesforce
- SFTP
- Snowflake
- Stripe
- Tableau
- Twitter Ads
- Zendesk
- Airtable (coming soon)
- Shopify (coming soon)
Mage adheres to the data engineering standard for data integrations called the Singer Spec.
All sources (aka taps) that has been developed by the open-source community can be adapted to use in Mage. Here is a short list of available ones.
Don’t see a source you need?
Check out the next section on Add a new source or tell us on slack and we can help you build it.
Destinations
A destination is a system that you want to export data into. A destination can be a 3rd party API, SaaS, database, data warehouse, or a data lake.
Read more about destinations:
Available destinations
- Amazon Web Services (AWS) Redshift
- Amazon Web Services (AWS) S3
- BigQuery
- Clickhouse
- Delta Lake (S3)
- Delta Lake (Azure)
- Elasticsearch
- Google Cloud Storage
- Kafka
- Microsoft SQL Server
- MongoDB
- MySQL
- Opensearch
- OracleDB
- PostgreSQL
- Salesforce
- Snowflake
- Trino
- Azure Blob Storage (coming soon)
- HubSpot (coming soon)
Mage adheres to the data engineering standard for data integrations called the Singer Spec.
All destinations (aka targets) that has been developed by the open-source community can be adapted to use in Mage. Here is a short list of available ones.
Don’t see a destination you need?
Check out the next section on Add a new destination or tell us on Slack. and we can help you build it.
Was this page helpful?