Give your data team magical powers.
🎶 | Orchestration | Schedule and manage data pipelines with observability. |
📓 | Notebook editor | Interactive Python, SQL, & R editor for coding data pipelines. |
🏗️ | Data integration | Synchronize data from 3rd party sources to your internal destinations. |
🚰 | Streaming | Ingest and transform real-time data. |
🧱 | dbt | Build, run, and manage your dbt models with Mage. |
💻 | Easy developer experience | Open-source engine that comes with a custom notebook UI for building data pipelines. |
🚢 | Engineering best practices | Build and deploy data pipelines using modular code. No more writing throwaway code or trying to turn notebooks into scripts. |
💳 | Data as a first-class citizen | Designed from the ground up specifically for running data-intensive workflows. |
🪐 | Scaling made simple | Analyze and process large data quickly for rapid iteration. |
🏢 | Project | Like a repository on GitHub; this is where you write all your code. |
🪈 | Pipeline | Contains references to all the blocks of code you want to run, charts for visualizing data, and organizes the dependency between each block of code. |
🧱 | Block | A file with code that can be executed independently or within a pipeline. |
🤓 | Data product | Every block produces data after it’s been executed. These are called data products in Mage. |
⏰ | Trigger | A set of instructions that determine when or how a pipeline should run. |
🏃♂️ | Run | Stores information about when it was started, its status, when it was completed, any runtime variables used in the execution of the pipeline or block, etc. |
🧙 Mage Magic is indistinguishable from advanced technology. A mage is someone who uses magic (aka advanced technology).Together, we’re Magers!
🧙♂️🧙 Magers (/ˈmājər/
)
A group of mages who help each other realize their full potential! Join us on Slack.
✨ This documentation & project are brought to you by the following magical individuals (learn more about contributing here):