Use BigQuery as a Mage destination to load structured data into Google Cloud’s fully-managed, serverless data warehouse.


Configuration Parameters

When configuring BigQuery as a destination, provide the following parameters:

KeyDescriptionExample ValueRequired
path_to_credentials_json_fileAbsolute path to your Google Cloud service account credentials file (JSON). If Mage is running on a GCP instance, leave this as null to use the instance’s service account./path/to/credentials.json
project_idThe GCP project ID where your BigQuery dataset exists.example_project
datasetName of the BigQuery dataset where data will be written.example_dataset
location(Optional) The region of your BigQuery dataset. If not set, BigQuery may infer it automatically.us-west1
disable_update_column_typesIf false, Mage will alter existing BigQuery column types to match the schema when necessary. If true, existing types will not be modified.false (default)
use_batch_loadUse BigQuery load jobs instead of the streaming API. Recommended for better performance and cost efficiency.true (default)
credentials_infoAlternative to path_to_credentials_json_file. Directly embed your credentials JSON as a dictionary (useful for environment-based configs).See structure below

credentials_info Structure

If you’re passing credentials inline instead of a path, structure it like this:

auth_provider_x509_cert_url: str
auth_uri: str
client_email: str
client_id: str
client_x509_cert_url: str
private_key: str
private_key_id: str
project_id: str
token_uri: str
type: str

Optional Configuration

KeyDescriptionExample ValueDefault
skip_schema_creationIf true, Mage will not run a CREATE SCHEMA command. Useful if the schema already exists. See issue #3416.truefalse
lower_caseIf true, column names will be automatically converted to lowercase before writing to BigQuery.truetrue

Required Permissions

Your GCP service account or user must have the following permissions to use BigQuery as a destination:

bigquery.datasets.create
bigquery.jobs.create
bigquery.readsessions.create
bigquery.readsessions.getData

Additionally, the account must have the BigQuery Data Editor role on the target dataset:

  • Role: roles/bigquery.dataEditor