Sources
Snowflake
Configuration
Connect to your Snowflake data warehouse and extract data by providing the following configuration parameters:| Key | Description | Sample Value |
|---|---|---|
account | Your Snowflake account identifier. | abc1234.us-east-1 |
database | The name of the database you want to query data from. | DEMO_DB |
schema | Schema containing the tables you want to extract data from. | PUBLIC |
username | Username for accessing the database (must have read/write permissions). | guest |
warehouse | Name of the warehouse for executing queries. | COMPUTE_WH |
Optional Connection Configs
| Key | Description | Sample Value |
|---|---|---|
password | Password for the database user. | abc123... |
private_key_file | Path to the Snowflake private key file (supported in version >= 0.9.76). | /path/to/snowflake_key |
private_key_file_pwd | Passphrase for the private key file (supported in version >= 0.9.76). | abc123... |
role | Role of the user to access the database. | ROLE |
For setting up key-pair authentication, follow this guide.
Other Optional Configs
| Key | Description | Sample Value |
|---|---|---|
batch_fetch_limit | Number of rows to fetch per batch (default: 50,000). Increase for high-memory systems. | 50000 |
What is Snowflake?
Snowflake is a leading cloud-based data warehouse platform that provides fast, scalable, and flexible data storage and analytics. It separates compute and storage for cost efficiency and performance, supporting structured and semi-structured data with SQL-based querying.Why Integrate Snowflake with Mage?
- Seamless ETL pipelines: Automate extraction of large datasets from Snowflake into your data lakes or warehouses.
- Scalable performance: Handle millions of rows with ease using high-speed, cloud-native architecture.
- Flexible authentication: Use either password-based or secure key-pair authentication.
- Cost-effective: Optimize compute and storage separately for your data workflows.
- Enterprise-ready security: Built-in role-based access control and encryption.