Releases

☁️ Google cloud storage source

You can now effortlessly configure your data integration source blocks to Google Cloud Storage using this snippet. It specifies the destination bucket, sets the file type to Parquet, and provides the path to your service account credentials needed for authentication.

by @TalaatHasanin in https://github.com/mage-ai/mage-ai/pull/5334

🌬️ 🍽️ Airtable Integration

Effortlessly streamline your data data integration source blocks with Airtable using this tailored script. It establishes a connection to your Airtable base, designates the specific table for integration, and securely integrates your Airtable API key for access.

by @TalaatHasanin in https://github.com/mage-ai/mage-ai/pull/5404

✂️ Add trim reformat action to transformer block

A new “trim reformat” action has been added to the Python transformer block, allowing for the removal of leading and trailing whitespace from specified text columns. This enhancement ensures cleaner and more consistent data formatting by automatically stripping unnecessary spaces around text entries.

by @cristopheridlc in https://github.com/mage-ai/mage-ai/pull/5321

🐻‍❄️ Enable polars dataframe in GCS data exporter

This update enhances the export method of the GCS IO module, expanding its functionality to support exporting Polars DataFrames in addition to the previously supported Pandas DataFrames. This new capability allows users to seamlessly work with both data formats, offering greater flexibility in managing and exporting data from their workflows.

by @TalaatHasanin in https://github.com/mage-ai/mage-ai/pull/5348

Add support to scheduler name on k8s executor

This update adds the ability to customize Kubernetes scheduler options when using the Kubernetes Executor. Users can now directly configure scheduler settings, providing increased flexibility and control over how pods are scheduled within Kubernetes environments.

by @messerzen in https://github.com/mage-ai/mage-ai/pull/5412

🧠 Memory management upgrade

This feature enhances system performance and stability by optimizing allocation, deallocation, and overall utilization of RAM resources, resulting in improved efficiency and reduced memory-related errors.

Check out the doc for details.

🔄 Dynamic blocks 2.0

Introducing an enhanced Dynamic blocks feature for creating dynamic content blocks that can adapt based on user input or data changes, offering improved flexibility and interactivity for users. This update aims to streamline workflows and enhance the overall user experience by making content more responsive and customizable.

Learn more in this doc.

🔐 Azure DB connection via Key Vault

Securely retrieve a database connection URL stored in Azure Key Vault using environment variables for authentication.

AZURE_KEY_VAULT_URL
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET
AZURE_TENANT_ID

by @wangxiaoyou1993 in #5302

👁️ Workspace monitoring

Add an “Overview” page and “Pipeline runs” page to the Workspace Management UI. This provides some overall monitoring for all of the pipeline runs in the different workspaces without having to individually open up each workspace instance.

by @johnson-mage in https://github.com/mage-ai/mage-ai/pull/5311