Backblaze B2

​

Prerequisites

pip install "mage-ai[s3]"

​

Add credentials

1. Create a new pipeline or open an existing pipeline.
2. Expand the left side of your screen to view the file browser.
3. Scroll down and click on a file named io_config.yaml.
4. Enter the following keys and values under the key named default (you can have multiple profiles, add it under whichever is relevant to you)
   version: 0.1.1
   default:
     B2_APPLICATION_KEY_ID: ...
     B2_APPLICATION_KEY: ...
     # Optional. Defaults to us-west-004. If your bucket is in another region,
     # uncomment and set this to that region's endpoint (shown on the B2 bucket
     # page), e.g. us-west-001, us-east-005, eu-central-003:
     # B2_ENDPOINT_URL: https://s3.us-east-005.backblazeb2.com
   
   For backwards compatibility, the loader also falls back to AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_ENDPOINT (for the endpoint URL) when the corresponding B2_* keys are not set.

​

Using Python block

1. Create a new pipeline or open an existing pipeline.
2. Add a data loader or transformer block (the code snippet below is for a data loader).
3. Select Generic (no template).
4. Enter this code snippet (note: change the config_profile from default if you have a different profile):
   from mage_ai.settings.repo import get_repo_path
   from mage_ai.io.config import ConfigFileLoader
   from mage_ai.io.backblaze_b2 import BackblazeB2
   from os import path
   from pandas import DataFrame
   
   if 'data_loader' not in globals():
       from mage_ai.data_preparation.decorators import data_loader
   
   
   @data_loader
   def load_from_backblaze_b2(**kwargs) -> DataFrame:
       config_path = path.join(get_repo_path(), 'io_config.yaml')
       config_profile = 'default'
   
       bucket_name = '...'  # Change to your B2 bucket name
       object_key = '...'   # Change to your object key
   
       return BackblazeB2.with_config(ConfigFileLoader(config_path, config_profile)).load(
           bucket_name,
           object_key,
       )
   
5. Run the block.

​

Endpoint override

default:
  B2_ENDPOINT_URL: https://s3.us-east-005.backblazeb2.com

​

Errors

​

Permissions

• readFiles
• writeFiles
• listFiles

• Read data from B2 (e.g. .csv, .parquet, .json)
• Write query results or transformed data to B2
• List contents of a bucket when needed