What’s an IO class?

IO classes are at the heart of Mage— they’re the code that enables Mage to read and write data. When you configure your io_config to use a source or destination, you’re telling Mage to use a specific IO class.

IO classes are defined here in the Mage repository. You can see that there are a few different types of IO classes— some are for reading data, some are for writing data, and some are for both.

Why should I contribute?

If you have a favorite IO class that’s not currently supported, you can reach out or contribute it to Mage! Contributing is a great way to build your skills, become a part of the Mage community, and help others.

How do I contribute?

Configure your development environment

The first step to contributing an IO class is to configure your development environment. You can find instructions for doing so here.

Create a new IO class

Once configured, you’ll want to create a new file in the mage_ai/io directory. The file should be named after the IO class you’re contributing. For example, if you’re contributing an IO class called MyIOClass, you should create a file called my_io_class.py.

Some IO classes are relatively straightforward, like exporting to local files or writing to Google Sheets. Others, like full database integrations, can be pretty complex. Google BigQuery is a good example of a complex IO class.

Most classes are platform specific, but for a database, you might need the following methods:

  • alter_table: Alter the table schema
  • load: Load data from the database
  • export: Export data to the database
  • execute: Execute a query on the database
    • execute_queries: Execute multiple queries on the database

Additionally, every class will need to have the following methods:

  • __init__: Initialize the class
  • with_config: Initialize the database client from the configuration loader. You’ll notice that this method is used in all of our templates.

Test your IO class

In order to test your class, you should perform all of the methods you defined in the class itself. Once you pass our linting checks (outlined in the development environment link above) and your tests pass, you’re ready to submit a pull request! Learn more about pull requests on the contribution.

Create a pull request

1

Fork the Mage GitHub repo

Visit GitHub and fork the Mage repo, then clone it to your machine.

2

Create a branch

Create a branch in your forked repo and commit your changes.

3

Create a pull request

Once you've made and tested your changes, create a pull request on the Mage repo.

4

Tag Mage team members for a review

Add members for review in GitHub, please tag one of the following:

  • @wangxiaoyou1993 (backend/IO)
  • @dy46 (backend)
  • @johnson-mage (frontend/docs/website)
  • @tommydangerous