Sources
Doris
Configuration
To set up the Doris source, provide the following configuration parameters:
| Key | Description | Sample Value |
|---|---|---|
database | The name of the database you want to read data from. | demo |
host | Host name of your Doris database. | mage.abc.us-west-2.rds.amazonaws.com |
port | Port where the database is running (typically 9030). | 9030 |
username | Username with permissions to read and write to the specified schema. | root |
password | Password for the user. | abc123... |
connection_method | Connection method to Doris server: direct or ssh_tunnel. | direct or ssh_tunnel |
conn_kwargs | (Optional) Extra connection keyword arguments in dictionary format. | {"ssl_ca": "CARoot.pem", "ssl_cert": "certificate.pem", "ssl_key": "key.pem"} |
ssh_host | (Optional) Host of the intermediate bastion server. | 123.45.67.89 |
ssh_port | (Optional) Port of the bastion server. Default is 22. | 22 |
ssh_username | (Optional) Username used to connect to the bastion server. | username |
ssh_password | (Optional) Password used to connect to the bastion server (if authenticating with password). | password |
ssh_pkey | (Optional) Path to the private key used to connect to the bastion server (if authenticating with private key). | /path/to/private/key |
Optional Configuration
| Key | Description | Sample Value |
|---|---|---|
batch_fetch_limit | Number of rows to fetch in each batch (defaults to 50,000). Adjust higher if your instance has more memory. | 50000 |