​
Overview

Use OpenSearch as a destination in Mage to publish JSON-formatted records into an OpenSearch index. This is ideal for search indexing, log aggregation, and real-time analytics.

By default, Mage uses the pipeline’s table name to set the OpenSearch index name. You can also define dynamic index patterns using JSONPath expressions.

​
Configuration Parameters

KeyDescriptionDefaultRequired
schemeHTTP scheme used for connecting to OpenSearch. Use http or https.http✅
hostHostname or IP address of the OpenSearch node or cluster.localhost✅
portPort number for the OpenSearch HTTP endpoint.9200✅
username(Optional) Username for basic authentication.None❌
password(Optional) Password for basic authentication.None❌
bearer_token(Optional) Bearer token for token-based authentication.None❌
api_key_id(Optional) API key ID for key-based authorization.None❌
api_key(Optional) API key secret for key-based authorization.None❌
ssl_ca_file(Optional) Path to the SSL CA certificate file for verifying HTTPS connections.None❌
index_schema_fields(Optional) JSONPath mapping used to dynamically generate the index name based on record values.None❌
metadata_fields(Optional) Dictionary mapping fields (using JSONPath) to populate ECS-style metadata fields in the index request (e.g., _id, @timestamp).None❌

​
Notes

  • Mage automatically converts each record into JSON format and indexes it into the specified OpenSearch index.
  • If index_schema_fields is configured, Mage will use the JSONPath-extracted values from each record to generate dynamic index names.
  • metadata_fields is useful for setting OpenSearch ECS-compatible fields like _id, @timestamp, etc., from the incoming data stream.
  • Authentication can be configured using basic auth, bearer tokens, or API key pairs.
  • For production use, we recommend enabling https and providing a trusted ssl_ca_file.