Configuration

To set up the BigQuery source, provide the following configuration parameters:

KeyDescriptionSample Value
path_to_credentials_json_filePath to your Google service account credentials JSON file. If Mage is running on GCP, you can leave this null to use the instance’s service account./path/to/service_account_credentials.json
datasetBigQuery dataset you want to read data from.example_dataset
credentials_infoAn alternative way to specify Google service account credentials directly as a dictionary.Structure shown below

credentials_info structure:

auth_provider_x509_cert_url: str
auth_uri: str
client_email: str
client_id: str
client_x509_cert_url: str
private_key: str
private_key_id: str
project_id: str
token_uri: str
type: str

Optional Configs

KeyDescriptionSample value
skip_schema_creationIf true, Mage won’t run CREATE SCHEMA command. For more information, see this issuetrue
lower_caseIf true, Mage will set all columns name as lower case. Default is truetrue

Required permissions

Your BigQuery account should at least have the below permissiont to use the BigQuery destination

bigquery.datasets.create
bigquery.jobs.create
bigquery.readsessions.create
bigquery.readsessions.getData

Your account should also have “BigQuery Data Editor” role for the BigQuery dataset you specified in the config.