Storage
MySQL
Add credentials
- Create a new pipeline or open an existing pipeline.
- Expand the left side of your screen to view the file browser.
- Scroll down and click on a file named
io_config.yaml
. - Enter the following keys and values under the key named
default
(you can have multiple profiles, add it under whichever is relevant to you)
version: 0.1.1
default:
MYSQL_DATABASE: ...
MYSQL_HOST: ...
MYSQL_PASSWORD: ...
MYSQL_PORT: 3306
MYSQL_USER: root
Using SQL block
- Create a new pipeline or open an existing pipeline.
- Add a data loader, transformer, or data exporter block.
- Select
SQL
. - Under the
Data provider
/Connection
dropdown, selectMysql
. - Under the
Profile
dropdown, selectdefault
(or the profile you added credentials underneath). - Enter the optional table name of the table to write to.
- Under the
Write policy
dropdown, selectReplace
orAppend
(please see SQL blocks guide for more information on write policies). - Enter in this test query:
SELECT 1
. - Run the block.
Using Python block
import mysql.connector
conn = mysql.connector.connect(
host='...',
password='...',
port=3306,
user='root',
database='...',
)
sql = """
SELECT
id
, email
FROM auth_user
"""
cursor = conn.cursor()
cursor.execute(sql)
rows = cursor.fetchall()
import pandas as pd
df = pd.DataFrame(rows, columns=['id', 'email'])
df.to_csv('default_repo/users.csv', index=False)