Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 37 ยท Triggers and Alerts

Zabbix Actions, Media Types and Notifications: Email, Webhooks, Escalations

Configure Zabbix alerting: trigger actions with conditions, operations and escalation steps, media types (email, webhook, script), user media, message macros and recovery operations.

A trigger by itself only creates a problem in the frontend. To reach a person, you need an action: a rule that says "when an event matching these conditions happens, do these operations". Operations can send a message, run a command or script, or escalate to somebody else if nobody reacts. Messages travel through media types (email, chat, SMS, webhook) to the media configured on each user.

Zabbix does not run on this site. The menus are described in words, and TryIt blocks are Python stand-ins that model the matching and escalation logic.

The chain from event to message

flowchart LR
    E["Event<br/>problem or recovery"] --> A{"Action<br/>conditions match?"}
    A -->|"yes"| O["Operations<br/>(steps, delays)"]
    O --> U["User or user group"]
    U --> M["Media (user's address)"]
    M --> T["Media type<br/>email / webhook / SMS"]
    T --> N["Message delivered"]

Every notification passes through the same chain.

Media types

Define them in Alerts > Media types. Built-in types include:

  • Email: SMTP server, port, security (none, STARTTLS, SSL/TLS), authentication, "From" address. After saving, click Test to send a message.
  • Webhook: JavaScript code that calls an HTTP API. Zabbix ships ready-made webhooks for Slack, Microsoft Teams, Telegram, PagerDuty, Opsgenie, Jira, ServiceNow, Mattermost, Discord and more; you only fill in parameters like a token or URL.
  • SMS: through a GSM modem.
  • Script: runs a script on the server (in AlertScriptsPath) with the message passed as parameters.

Each media type has message templates for problem, recovery and update events, written with macros. You can also set concurrent sessions, attempts and retry interval.

User media

A media type is not enough: each user (Users > Users > Media tab) adds a medium per media type: the address (an email address or Telegram chat ID), the When active schedule (for example 1-5,09:00-18:00) and the Use if severity levels. This is where you say "Anna receives only High and Disaster, by Telegram, on weekdays".

Because routing lives in users and groups, a common practice is a user group per team (like db-oncall) with one or two media each, and actions that address groups.

Trigger actions

Alerts > Actions > Trigger actions > Create action has three tabs.

1. Action: name and conditions

Conditions decide which events the action handles. Typical conditions:

  • Host group equals Prod/DB
  • Tag value team equals db
  • Trigger severity is greater than or equals High
  • Problem is suppressed is No (do not notify during maintenance)
  • Trigger equals a specific trigger, or Host equals

Combine them with And, Or, And/Or (same type uses OR, different types use AND), or a Custom expression like (A or B) and C.

2. Operations

An operation says what to do and when. Default operation step: send a message to a user group using a media type, with the default message or a custom one. Fields:

  • Steps (from-to) and step duration (default 1 hour) define an escalation.
  • Send to user groups / users, and Send only to a specific media type (or all).
  • Other types: Run script (run a command on the host, the agent, or the server: for example restart a service), Notify all involved.

Example escalation:

Step Delay Operation
1 immediately Email to db-team
2 after 10 minutes Telegram to db-oncall
3 after 30 minutes Phone/SMS to engineering-managers

If the problem is acknowledged or resolved earlier, steps not yet reached stop: escalation only continues while the problem is open (and, with a condition, not yet acknowledged).

3. Recovery operations and update operations

  • Recovery operations send "RESOLVED" messages, close tickets, or notify everyone who was alerted ("Notify all involved").
  • Update operations run when someone acknowledges, comments or changes severity, so the team sees that somebody is working on it.
Python Stand-in: action conditions and escalation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last two rows show why conditions matter: the warning and the Dev problem match nothing, so nobody is bothered.

Message macros

Messages are text with macros that Zabbix replaces when sending. The most useful:

Macro Value
{EVENT.NAME} Problem name (the trigger name with macros expanded)
{EVENT.SEVERITY}, {EVENT.STATUS} Severity and PROBLEM/OK
{EVENT.DATE}, {EVENT.TIME}, {EVENT.AGE}, {EVENT.DURATION} When and how long
{EVENT.OPDATA} Operational data
{EVENT.TAGS} Tags as text
{HOST.NAME}, {HOST.IP}, {HOST.CONN} Host details
{TRIGGER.DESCRIPTION}, {TRIGGER.URL} Trigger documentation
{ITEM.NAME1}, {ITEM.LASTVALUE1} First item in the expression
{ESC.HISTORY} History of earlier escalation messages
{ACK.MESSAGE}, {USER.FULLNAME} For update messages

A good message answers four questions: what is wrong, where, how bad, and what to do next (a runbook link). Example subject: [{EVENT.SEVERITY}] {EVENT.NAME} on {HOST.NAME}; body: operational data, time, and {TRIGGER.URL}.

Python Stand-in: render a message

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Scripts and remote commands

An operation can run a script such as systemctl restart nginx on the monitored host through the agent. For security, the agent must allow it (AllowKey=system.run[*] for agent 2, or EnableRemoteCommands in older agents), and the action runs as the agent's user. Use this for safe, idempotent remediations (restart a stuck service) and log it. Never allow unrestricted system.run in production without strict conditions.

Other action types

Zabbix also has discovery actions (when a network discovery finds a device: add the host, link a template), autoregistration actions (when an agent registers itself: add it with templates by hostname metadata), and internal actions (notify when items become unsupported or low-level discovery rules fail). The discovery chapter uses them.

Testing and troubleshooting alerts

  1. Use the media type Test button first.
  2. Check Reports > Action log to see each alert: sent, failed, or not sent, with the error.
  3. Common failures: user has no media for that type; media "Use if severity" excludes this level; "When active" window closed; the user group lacks permission to the host group (then the user gets no messages for it); action conditions do not match; problem is suppressed.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Where do you configure an individual user's email address?

    A media type defines the channel; each user adds a medium with their own address.
  2. 2. What defines an escalation in an action?

    Steps with delays let you notify different people at different times.
  3. 3. Which tab of the action sends a 'RESOLVED' message?

    Recovery operations run when the problem is resolved.
  4. 4. Where do you look to see why a notification was not delivered?

    The action log shows each alert with its status and error.

Summary

  • Event, action conditions, operations, user media, media type, message.
  • Media types: email, webhook, SMS, script; users define their addresses, hours and severities.
  • Escalations use steps with durations; recovery and update operations close the loop.
  • Message macros should state what, where, how bad and what to do.
  • Debug with Test buttons and the Action log.

Next: Maintenance and dependencies