io_config.yaml
.default
(you can
have multiple profiles, add it under whichever is relevant to you)If you want to connect to Postgres cluster with ssh tunnel, update the value of POSTGRES_CONNECTION_METHOD
to ssh_tunnel
, and enter the values for keys with prefix POSTGRES_SSH
.
Key | Description | Sample value |
---|---|---|
POSTGRES_SSH_HOST | The host of the intermediate bastion server. | 123.45.67.89 |
POSTGRES_SSH_PORT | The port of the intermediate bastion server. Default value: 22 | 22 |
POSTGRES_SSH_USERNAME | The username used to connect to the bastion server. | username |
POSTGRES_SSH_PASSWORD | (Optional) The password used to connect to the bastion server. It should be set if you authenticate with the bastion server with password. | password |
POSTGRES_SSH_PKEY | (Optional) The path to the private key used to connect to the bastion server. It should be set if you authenticate with the bastion server with private key. | /path/to/private/key |
SQL
.Data provider
dropdown, select PostgreSQL
.Profile
dropdown, select default
(or the profile you added
credentials underneath).Save to schema
label, enter the schema name you want this block
to save data to.Write policy
dropdown, select Replace
or Append
(please see
SQL blocks guide for more
information on write policies).SELECT 1
.Generic (no template)
.config_profile
from default
if
you have a different profile):Here is an example code snippet to export a dataframe to Postgres:
To overwrite a column type when running a python export block, simply specify the column name and type in the overwrite_types
dict in data exporter config
Here is an example code snippet:
Field name | Description | Example values |
---|---|---|
allow_reserved_words | Whether to allow using reserved words as column names. | True/False (default: False) |
auto_clean_name | Whether to automatically clean the column name (replace the empty space with underscore, avoid using number as the prefix of the column name) | True/False (default: True) |
case_sensitive | Whether to support case sensitive columns | True/False (default: False) |
cascade_on_drop | Whether to cascade the table drop. | True/False (default: False) |
drop_table_on_replace | Whether to drop the table when “if_exists” param is set to “replace”. | True/False (default: False) |
if_exists | Specify resolution policy if table name already exists | ”fail”/“replace”/“append” (default: “replace”) |
overwrite_types | Overwrite the column types | {'column1': 'INTEGER', 'column2': 'VARCHAR'} |
unique_conflict_method | How to handle the conflict on unique constrants. | ‘UPDATE’ (default: None) |
unique_constraints | The unique constraints of the table. | [‘col1’, ‘col2’] (default: None) |