Pipeline Trigger Overview

​

Overview

​

Key Benefits of Pipeline Triggers:

• Automated execution - Run pipelines without manual intervention
• Flexible scheduling - Support for cron expressions and API calls
• Environment management - Different trigger configurations for dev, staging, and production
• Resource optimization - Control when pipelines consume compute resources
• Monitoring and SLA - Built-in performance tracking and alerting

​

Trigger Types

​

Schedule triggers

• Run exactly once - Run one time when the schedule starts, then disable the trigger
• Hourly - Run at minute 0 of each hour in UTC
• Daily - Run once per day at 00:00 UTC
• Weekly - Run every Monday at 00:00 UTC
• Monthly - Run on the first day of each month at 00:00 UTC
• Always on - Keep streaming pipelines running continuously by starting a new run when the previous run completes
• Custom (cron syntax) - Use cron expressions for exact clock times and advanced scheduling patterns

​

Common cron examples

# Every 15 minutes
*/15 * * * *

# Every weekday at 9 AM
0 9 * * 1-5

# Every day at 2:30 PM
30 14 * * *

# Every Sunday at midnight
0 0 * * 0

# Every 1st of the month at 6 AM
0 6 1 * *

# Every 5 minutes during business hours (9 AM - 5 PM, weekdays)
*/5 9-17 * * 1-5

​

API triggers

• Webhook integrations with external services
• Manual pipeline execution from external applications
• Event-driven data processing
• Integration with CI/CD pipelines

POST {MAGE_HOST}/api/pipeline_schedules/3/pipeline_runs/abc123

{
  "pipeline_run": {
    "variables": {
      "env": "staging",
      "schema": "public"
    }
  }
}

​

Deprecated Event triggers

​

How to Create and Manage Pipeline Triggers

​

Creating a New Trigger

1. Navigate to your pipeline in the Mage UI
2. Click the Triggers tab in the pipeline view
   • Direct URL: /pipelines/[pipeline_uuid]/triggers
3. Click + New trigger button
4. Select the trigger type (Schedule or API)
5. Configure the trigger settings based on the type you selected
6. Click Create trigger to create the trigger

​

Trigger configuration fields

• Name - A descriptive name for your trigger. Names are trimmed before validation, must be unique within the pipeline, must include at least one letter or number, and cannot exceed 255 characters.
• Description - Optional description to explain the trigger’s purpose
• Tags - Add or search tags to organize and categorize your triggers. Each tag name can contain up to 70 characters.

• Frequency - Choose from preset options (Hourly, Daily, Weekly, Monthly, Once, Always on) or select Custom for CRON syntax when you need more precise timing
• Schedule start - Choose Start at next Frequency time to let the trigger run at the next scheduled Frequency time, or Do not start before date/time to set the earliest allowed start. That date/time is not the run time; Frequency still controls when runs happen. Schedule start only controls when that trigger plan is allowed to begin.
• Schedule load guidance - For @once triggers with Do not start before date/time, Mage may show load or quiet-hour guidance for the selected start time. This is planning guidance only; use it to decide whether the selected start time is acceptable. If overlap is high, use suggested lower-load times, or keep your selected time if overlap is acceptable.
• Enable landing time - Instead of scheduling starts at a specific time, schedule runs so the pipeline can complete by a specified time

• Runtime variables - Override pipeline variables for this specific trigger
• Run settings - Configure timeouts, SLA, and failure handling
• Advanced settings - Additional trigger-specific configurations

​

Viewing and Monitoring Triggers

​

Pipeline-Specific Trigger Management

1. Navigate to your pipeline in the Mage UI
2. Click the Triggers tab
   • Direct URL: /pipelines/[pipeline_uuid]/triggers
3. You’ll see a comprehensive table listing all configured triggers with the following information:

​

Project-Wide Trigger Dashboard

1. Go to the Mage homepage
2. Click Triggers in the left navigation menu
   • Direct URL: /triggers
3. You’ll see all triggers from all pipelines in your project

• Active - Toggle switch showing if the trigger is enabled/disabled
• Logs - Icon to access trigger execution logs
• Name - The trigger name (clickable link with arrow icon for code-defined triggers)
• Type - Trigger type (Schedule, API, or deprecated legacy Event)
• Frequency - How often the trigger runs (e.g., @once, @daily)
• Status - Current status of the trigger
• Next - Scheduled time for the next execution (with info icon for details)
• Runs - Total number of pipeline runs initiated by this trigger
• Tags - Associated tags for organization
• Created - Timestamp when the trigger was created (with info icon for details)

• From trigger list: Use the toggle switch in the “Active” column to enable or disable a trigger
• From trigger detail page: Navigate to /pipelines/[pipeline_uuid]/triggers/[trigger_id] and use the “Disable trigger” button
• Disable all triggers in Mage Pro: On the global Triggers page, the current search and filters define the scope; only matching active triggers are disabled
• Disabled triggers: Will not run automatically but remain configured for future use
• Enabled triggers: Will run according to their configured schedule, API endpoint, or legacy event conditions

• Run once button: Click the “Run@once” button in the trigger list or detail page to manually trigger a pipeline run
• Testing purpose: Use this to test your trigger configuration before enabling it for automatic execution
• Immediate execution: The pipeline will start running immediately when you click the button

• Historical data processing: Create backfill triggers to process historical data for a specific date range
• One-time execution: Backfill triggers run once and then disable themselves after completion
• Data gap filling: Use backfills to fill in missing data or reprocess data with updated logic

• Mage OSS: Click the delete (trash) icon in the Actions column to delete a trigger
• Mage Pro: Right-click a trigger row to open its context menu. Select Copy URL to copy the trigger’s direct URL, or select Delete and confirm Yes to delete the trigger.

​

Advanced Trigger Configuration

​

Run Settings and Performance Optimization

​

Timeout configuration

• Set a timeout for each run of this trigger (optional)
  • Configure a timeout in seconds for pipeline runs initiated by this trigger
  • If a run exceeds the timeout, it will be marked with the specified timeout status
• Status for runs that exceed the timeout (default: failed)
  • Choose how to handle runs that exceed the configured timeout
  • Options typically include: failed, cancelled, or warning

​

SLA configuration

• Configure trigger SLA
  • Set up Service Level Agreement monitoring for your trigger
  • Define expected completion time with configurable time units (minutes, hours, days)
  • Helps monitor trigger performance against business requirements

​

Additional trigger settings

• Override runtime variables
  • You can override the runtime variables defined in the pipeline to use different values for different triggers.
  • This option only appears if you have runtime variables defined in the pipeline.
  • Use Form mode to add, edit, or delete runtime variables one at a time.
  • Switch to JSON mode to edit the full runtime variables object in a code editor. Click Format JSON to reformat valid JSON before saving.
  • Runtime variable JSON must be an object. Variable names cannot contain whitespace, and reserved Mage variables cannot be included.
  • Benefits: Allows you to run the same pipeline with different configurations for different use cases (e.g., different data sources, environments, or processing parameters) without creating separate pipelines.
• Keep running pipeline even if blocks fail
  • When enabled, the pipeline continues running even if individual blocks fail
  • The pipeline will continue running blocks that are not dependent on the failed blocks until completion
• (Schedule triggers only) Skip run if previous run still in progress
  • When enabled, the scheduler will not start a new run if a previous run for this trigger is still in progress.
• (Schedule triggers only)
  Available starting in v0.9.62
  Create initial pipeline run if start date is before current execution period
  • When enabled, this creates an initial pipeline run if the schedule’s execution date is after its start datetime but before the current time.
  • Example: If current time is 1/4/24 12:00 UTC and you create a daily trigger with start date 1/3/24 12:00 UTC, a pipeline run with execution date 2024-01-04 00:00 is immediately created.

​

Frequently Asked Questions

​

What is a trigger in Mage?

• Schedule: Runs pipelines on a set interval (e.g., hourly, daily, weekly, monthly, or custom cron)
• API: Runs pipelines when a specific API endpoint is called, optionally with runtime variables

​

How do I create a trigger for my pipeline?

​

Can I configure triggers in code and in the UI? How are changes synced?

​

Why does my daily trigger run at 2AM instead of my specified time?

​

What happens if I delete a pipeline—are its triggers deleted too?

​

How do I prevent a pipeline from running if a previous run is still in progress?

​

Can I allow a pipeline to continue even if some blocks fail?

​

How do I trigger a pipeline via API?