User-defined templates

​

Overview

​

Block Templates

​

Creating block templates

1. Navigate to Block templates - Click “Templates” in the left navigation menu, then open Block templates. You can also go directly to /apps/templates/blocks.
2. Create new template - Click “New block template” button
3. Define template properties:
   • Template UUID - Enter a unique identifier (e.g., sql_data_loader)
   • Block type - Select the type (Data loader, Transformer, Data exporter, etc.)
   • Language - Choose the programming language (Python, SQL, R, YAML, etc.)
4. Edit template content - Write the template code with placeholders
5. Save template - Click “Save template” to create the template

​

Template File Structure

custom_templates/
└── blocks/
    └── {template_uuid}/
        ├── metadata.yaml    # Template metadata and configuration
        └── {template_uuid}.py  # Template code file

• metadata.yaml - Contains template metadata, block type, language, and other configuration
• {template_uuid}.py - The actual template code file that users will customize

custom_templates/
└── blocks/
    └── test_template/
        ├── metadata.yaml
        └── test_template.py

​

Template Library Interface

• Template categories - Filter templates by type (Blocks or Pipelines)
• Block template categories - Filter block templates by block type
• Team pipeline templates - Manage pipeline templates created from pipelines in your project; built-in starting points are available from the pipeline creation flow
• Pipeline template discovery - Search and filter the broader built-in catalog from New pipeline > From a template; the customization page remains focused on Team templates
• Template preview - View template details before using
• Template management - Edit, delete, or duplicate existing templates

1. From left navigation - Click “Templates” to access the template library. The default Templates page redirects to the block template library at /apps/templates/blocks.
2. From pipeline editor - Use “Browse templates” when adding new blocks
3. From pipeline creation - Select “From a template” when creating new pipelines
4. Template organization - Templates are organized by type and category for easy discovery

• Open Pipeline templates - Click the Pipelines tab in the template library or go directly to /apps/templates/pipelines
• Manage team templates - This customization page lists only reusable pipeline templates created and maintained in your Mage project
• Browse built-in starting points - From the Pipelines page, choose New pipeline, then From a template to browse Basics, General purpose, and Industry solutions alongside Team templates
• View template details - See template name, description, and metadata
• Select a team template - Click a Team template to inspect or customize it

​

Using block templates

1. Open pipeline editor - Navigate to the pipeline where you want to add the block
2. Add new block - Click the ”+” button to add a new block
3. Browse templates - Select “Browse templates” from the block creation options
4. Select template - Choose the template you want to use from the available templates
5. Name the block - Enter a specific name for your new block instance
6. Customize as needed - Modify the generated code for your specific use case

​

Pipeline Templates

​

Built-in pipeline template catalog

• Basics are remotely provided example pipelines for learning Mage fundamentals and common block patterns.
• General purpose templates are starting points for recurring workflows such as ingestion, data quality, streaming, and AI context preparation.
• Industry solutions adapt common patterns to outcomes in areas such as retail, financial services, healthcare, manufacturing, and logistics.

• Put non-sensitive settings in each block’s USER CONFIGURATION dictionary. Runtime variables can override matching lowercase settings when they must differ by run or environment.
• Put credentials in io_config.yaml. Source and destination blocks select connections by their configured tags, so keep secrets out of block configuration and runtime variables.

1. Read the overview and setup block.
2. Supply the required settings and create the named connection profiles.
3. Start in dry_run mode. Loaders return a sample DataFrame and exporters skip destination writes, so you can inspect the transformation without connecting to external systems.
4. Turn off dry_run, then run the blocks in order with the smallest representative input supported by the template.
5. Before scheduling, choose an idempotent write policy and add your organization’s monitoring, recovery, privacy, and retention controls.

​

Creating pipeline templates

1. Navigate to Team pipeline templates - Open Templates, then Pipeline templates, or go directly to /apps/templates/pipelines. This customization page lists only templates created and maintained by your team.
2. Create new template from an existing pipeline - From the pipeline list, right-click the pipeline you want to reuse and select the action to create a template. This opens /apps/templates/pipelines/new?pipeline_uuid={pipeline_uuid}.
3. Configure template settings - Set the template UUID, name, description, and other properties.
4. Save template - The pipeline structure will be saved as a reusable template.

custom_templates/pipelines/
└── example_pipeline_template/
    ├── metadata.yaml
    └── triggers.yaml

​

Using pipeline templates

1. Navigate to the pipelines page in your project
2. Click “New” button to open the pipeline creation menu
3. Select “From a template” from the dropdown options
4. Choose a source - Start in All templates, or select Basics, General purpose, Industry solutions, or Team
5. Find a template - Search the catalog or apply the filters available for the selected source
6. Choose your template from the available pipeline templates
7. Review the configuration guidance - For a General purpose or Industry solution template, open its first overview and setup Markdown block and supply the required credentials, identifiers, schemas, and business rules
8. Customize the pipeline - Modify blocks, connections, and settings as needed
9. Save the pipeline with your specific configuration

​

Best Practices

​

Template design

• Keep templates focused - Each template should serve a specific purpose
• Use clear variable names - Make variable purposes obvious
• Include comprehensive documentation - Explain usage and customization options
• Test with different configurations - Ensure templates work with various inputs
• Follow coding standards - Maintain consistent style and practices

​

Template maintenance

• Regular updates - Keep templates current with latest Mage features
• User feedback - Collect and incorporate user suggestions
• Performance optimization - Monitor and improve template performance
• Security review - Ensure templates follow security best practices