Skip to main content

​
Overview

Pipeline triggers are automated scheduling mechanisms that determine when and how data pipelines should run in Mage. A single pipeline can have multiple triggers, each with different configurations for various use cases like production runs, testing, or data backfills.

​
Key Benefits of Pipeline Triggers:

  • Automated execution - Run pipelines without manual intervention
  • Flexible scheduling - Support for cron expressions, API calls, and event-based triggers
  • Environment management - Different trigger configurations for dev, staging, and production
  • Resource optimization - Control when pipelines consume compute resources
  • Monitoring and SLA - Built-in performance tracking and alerting
Pipeline Schedule and Trigger are the same thing in Mage. You may see either term used throughout the product.

​
Trigger Types

​
Schedule triggers

Schedule triggers are the most common type, designed for recurring data pipeline execution at predetermined intervals. They’re perfect for ETL processes, data updates, and regular analytics jobs. Available scheduling frequencies include:
  • Run exactly once - Execute the pipeline once and then disable the trigger
  • Hourly - Run every hour
  • Daily - Run once per day
  • Weekly - Run once per week
  • Monthly - Run once per month
  • Always on - Immediately create new pipeline run when previous pipeline run completes (either succeeds or fails)
  • Custom (cron syntax) - Use cron expressions for advanced scheduling patterns
Custom frequency allows you to use cron syntax for advanced scheduling patterns. This gives you complete control over when your pipeline runs.

​
Common cron examples

Here are some common cron patterns you can use with custom frequency: 
# Every 15 minutes
*/15 * * * *

# Every weekday at 9 AM
0 9 * * 1-5

# Every day at 2:30 PM
30 14 * * *

# Every Sunday at midnight
0 0 * * 0

# Every 1st of the month at 6 AM
0 6 1 * *

# Every 5 minutes during business hours (9 AM - 5 PM, weekdays)
*/5 9-17 * * 1-5
Cron expression format: [minute] [hour] [day(month)] [month] [day(week)]
Timezone behavior: If you have the display_local_timezone setting enabled, the local cron expression will match your local timezone only if the minute and hour values are single values without any special characters (comma, hyphen, or slash). You can still use cron expressions with special characters for the minute/hour values, but they will be based in UTC time.
Use online cron expression generators or cron documentation to create more complex scheduling patterns that fit your specific needs.

​
API triggers

API triggers enable on-demand pipeline execution through HTTP requests. They’re ideal for webhook integrations, external system triggers, and manual pipeline execution from external applications. For more details, refer to the API Trigger documentation. Use cases for API triggers:
  • Webhook integrations with external services
  • Manual pipeline execution from external applications
  • Event-driven data processing
  • Integration with CI/CD pipelines
You can make a POST request to an endpoint provided in the UI when creating or editing a trigger. The endpoint depends on the Mage domain and the trigger’s unique ID. API endpoint: 
POST {MAGE_HOST}/api/pipeline_schedules/3/pipeline_runs/abc123
Request payload: Runtime variables can optionally be included in the request payload and are accessible within each pipeline block. 
{
  "pipeline_run": {
    "variables": {
      "env": "staging",
      "schema": "public"
    }
  }
}

​
Event triggers

An event-type trigger instructs the pipeline to run whenever a specific event occurs. For example, you can configure a pipeline to run when a database query completes or when a new object is created in Amazon S3 or Google Cloud Storage. You can also trigger a pipeline using a custom event by making a POST request to the /api/events endpoint with your event payload: 
POST /api/events
Content-Type: application/json

{
  "event_type": "custom_event",
  "payload": {
    "key": "value"
  }
}
Check out this tutorial on how to create an event trigger.

​
How to Create and Manage Pipeline Triggers

​
Creating a New Trigger

To create a new trigger for your pipeline:
  1. Navigate to your pipeline in the Mage UI
  2. Click the Triggers tab in the pipeline view
    • Direct URL: /pipelines/[pipeline_uuid]/triggers
  3. Click + New trigger button
  4. Select the trigger type (Schedule or API)
  5. Configure the trigger settings based on the type you selected
  6. Click Save changes to create the trigger
Trigger storage: Triggers created via the UI are stored in the database. For environment synchronization and version control, you can save triggers in code to sync them across different environments.

​
Trigger configuration fields

When creating or editing a trigger, you’ll see the following configuration fields: Basic settings:
  • Name - A descriptive name for your trigger
  • Description - Optional description to explain the trigger’s purpose
  • Tags - Add or search tags to organize and categorize your triggers
Schedule-specific fields:
  • Frequency - Choose from predefined options (Hourly, Daily, Weekly, Monthly) or select Custom for CRON syntax
  • Start date and time - When the trigger should begin running
  • Enable landing time - Instead of starting at a specific time, schedule runs to finish by a specified time
Additional options:
  • Runtime variables - Override pipeline variables for this specific trigger
  • Run settings - Configure timeouts, SLA, and failure handling
  • Advanced settings - Additional trigger-specific configurations

​
Viewing and Monitoring Triggers

​
Pipeline-Specific Trigger Management

To view all triggers for a specific pipeline:
  1. Navigate to your pipeline in the Mage UI
  2. Click the Triggers tab
    • Direct URL: /pipelines/[pipeline_uuid]/triggers
  3. You’ll see a comprehensive table listing all configured triggers with the following information:

​
Project-Wide Trigger Dashboard

To view all triggers across your entire project:
  1. Go to the Mage homepage
  2. Click Triggers in the left navigation menu
    • Direct URL: /triggers
  3. You’ll see all triggers from all pipelines in your project
Trigger list columns:
  • Active - Toggle switch showing if the trigger is enabled/disabled
  • Logs - Icon to access trigger execution logs
  • Name - The trigger name (clickable link with arrow icon for code-defined triggers)
  • Type - Trigger type (schedule, API, etc.)
  • Frequency - How often the trigger runs (e.g., @once, @daily)
  • Status - Current status of the trigger
  • Next - Scheduled time for the next execution (with info icon for details)
  • Runs - Total number of pipeline runs initiated by this trigger
  • Tags - Associated tags for organization
  • Created - Timestamp when the trigger was created (with info icon for details)
You can click on the logs icon to view detailed execution history and logs for each trigger. Enabling/Disabling triggers:
  • From trigger list: Use the toggle switch in the “Active” column to enable or disable a trigger
  • From trigger detail page: Navigate to /pipelines/[pipeline_uuid]/triggers/[trigger_id] and use the “Disable trigger” button
  • Disabled triggers: Will not run automatically but remain configured for future use
  • Enabled triggers: Will run according to their configured schedule or trigger conditions
Testing triggers:
  • Run once button: Click the “Run@once” button in the trigger list or detail page to manually trigger a pipeline run
  • Testing purpose: Use this to test your trigger configuration before enabling it for automatic execution
  • Immediate execution: The pipeline will start running immediately when you click the button
Backfill triggers:
  • Historical data processing: Create backfill triggers to process historical data for a specific date range
  • One-time execution: Backfill triggers run once and then disable themselves after completion
  • Data gap filling: Use backfills to fill in missing data or reprocess data with updated logic
Deleting triggers:
  • Mage OSS: Click the delete (trash) icon in the Actions column to delete a trigger
  • Mage Pro: Right-click on a trigger row and select Delete from the context menu

​
Advanced Trigger Configuration

​
Run Settings and Performance Optimization

​
Timeout configuration

  • Set a timeout for each run of this trigger (optional)
    • Configure a timeout in seconds for pipeline runs initiated by this trigger
    • If a run exceeds the timeout, it will be marked with the specified timeout status
  • Status for runs that exceed the timeout (default: failed)
    • Choose how to handle runs that exceed the configured timeout
    • Options typically include: failed, cancelled, or warning

​
SLA configuration

  • Configure trigger SLA
    • Set up Service Level Agreement monitoring for your trigger
    • Define expected completion time with configurable time units (minutes, hours, days)
    • Helps monitor trigger performance against business requirements

​
Additional trigger settings

  • Override runtime variables
    • You can override the runtime variables defined in the pipeline to use different values for different triggers.
    • This option only appears if you have runtime variables defined in the pipeline.
    • Benefits: Allows you to run the same pipeline with different configurations for different use cases (e.g., different data sources, environments, or processing parameters) without creating separate pipelines.
  • Keep running pipeline even if blocks fail
    • When enabled, the pipeline continues running even if individual blocks fail
    • The pipeline will continue running blocks that are not dependent on the failed blocks until completion
  • (Schedule triggers only) Skip run if previous run still in progress
    • When enabled, the scheduler will not start a new run if a previous run for this trigger is still in progress.
  • (Schedule triggers only)
    Available starting in v0.9.62
    Create initial pipeline run if start date is before current execution period
    • When enabled, this creates an initial pipeline run if the schedule’s execution date is after its start datetime but before the current time.
    • Example: If current time is 1/4/24 12:00 UTC and you create a daily trigger with start date 1/3/24 12:00 UTC, a pipeline run with execution date 2024-01-04 00:00 is immediately created.

​
Frequently Asked Questions

​
What is a trigger in Mage?

A trigger is a set of instructions that determine when or how a pipeline should run. There are three types of triggers in Mage: Schedule, Event, and API.
  • Schedule: Runs pipelines on a set interval (e.g., hourly, daily, weekly, monthly, or custom cron)
  • Event: Runs pipelines in response to specific events (e.g., file arrival, database update)
  • API: Runs pipelines when a specific API endpoint is called, optionally with runtime variables

​
How do I create a trigger for my pipeline?

You can create triggers via the Mage UI or by defining them in a triggers.yaml file within your pipeline folder. In the UI, go to the pipeline editor, select “Triggers,” and click “New trigger.” In code, add your trigger configuration to pipelines/[pipeline_uuid]/triggers.yaml.

​
Can I configure triggers in code and in the UI? How are changes synced?

Yes, you can configure triggers in both the UI and code. Changes made in the UI are synced to the YAML file, and vice versa. However, deleting a trigger from the YAML file does not remove it from the database; you must delete it in the UI.

​
Why does my daily trigger run at 2AM instead of my specified time?

By default, @daily triggers run at midnight UTC. If you want your pipeline to run at a specific time, use the “custom” frequency option and enter a cron expression that matches your desired schedule.

​
What happens if I delete a pipeline—are its triggers deleted too?

No, deleting a pipeline does not automatically delete its associated triggers. You need to manually remove triggers in the UI to avoid orphaned triggers.

​
How do I prevent a pipeline from running if a previous run is still in progress?

There is a trigger setting called “Skip run if previous run still in progress.” Enabling this will prevent the scheduler from starting a new run if the previous one is still active.

​
Can I allow a pipeline to continue even if some blocks fail?

Yes, you can enable the “Allow blocks to fail” setting in your trigger configuration. This allows the pipeline to continue running blocks that are not dependent on failed blocks.

​
How do I trigger a pipeline via API?

You can set up an API-type trigger, which allows you to start a pipeline by making a POST request to a specific endpoint. You can also pass runtime variables in your request payload.
⌘I