IO classes power Mage sources and destinations. Read on to learn how you can contribute an IO class to Mage.
io_config
to use a source or destination, you’re telling Mage to use a specific IO class.
IO classes are defined here in the Mage repository. You can see that there are a few different types of IO classes— some are for reading data, some are for writing data, and some are for both.
mage_ai/io
directory. The file should be named after the IO class you’re contributing. For example, if you’re contributing an IO class called MyIOClass
, you should create a file called my_io_class.py
.
Some IO classes are relatively straightforward, like exporting to local files or writing to Google Sheets. Others, like full database integrations, can be pretty complex. Google BigQuery is a good example of a complex IO class.
Most classes are platform specific, but for a database, you might need the following methods:
alter_table
: Alter the table schemaload
: Load data from the databaseexport
: Export data to the databaseexecute
: Execute a query on the database
execute_queries
: Execute multiple queries on the database__init__
: Initialize the classwith_config
: Initialize the database client from the configuration loader. You’ll notice that this method is used in all of our templates.Fork the Mage GitHub repo
Create a branch
Create a pull request
Tag Mage team members for a review
@wangxiaoyou1993
(backend/IO)@johnson-mage
(frontend/docs/website)@tommydangerous