Open the file named io_config.yaml at the root of your Mage project and enter weaviate required fields:

version: 0.1.1
  WEAVIATE_COLLECTION: collectionn_name

Required Weaviate Libraries

Add weaviate-client==3.25.3 into requirements.txt to install the required weaviate library.

Using Python block

  1. Create a new pipeline or open an existing pipeline.
  2. Add a data loader or data exporter with Template. Under “Databases” category you can find the “Weaviate” template. Then use corresponding load and export functions.
  • data loader arguments:

    • properties (List): list of columns to load.
    • collection (str): name of the collection. Defaults to the name defined in io_config.yaml.
    • with_limit (int): limit number of results returned.
    • with_text (str): text to query.
  • data exporter arguments:

    • df (DataFrame): data frame to write.
    • collection (str): name of the collection.
  1. Add your customized code into the loader, exporter or add extra transformer blocks.
  2. Run the block.