Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 37 ยท Alerting

Grafana Silences, Mute Timings and Handling Alert Noise

Pause Grafana notifications the right way: one-off silences with matchers, recurring mute timings, pausing rules, and a practical plan for reducing alert noise and fatigue.

Not every alert deserves a page at 3 a.m., and during planned maintenance none of them do. Grafana gives you three ways to stop notifications without disabling your monitoring: silences, mute timings and pausing a rule. Knowing which one fits each situation keeps your alerting trustworthy. This lesson also covers a practical approach to alert noise.

Grafana is not available on this site, so the interface is described in words, and the TryIt blocks are Python stand-ins for the matching logic and the time-window checks.

What a silence does

A silence is a temporary, one-off rule: "for this period, do not send notifications for alerts whose labels match these matchers". The alerts still evaluate and still show up in the alert list, and their state changes are still recorded, but no notification leaves Grafana. When the silence expires, notifications for alerts that are still firing resume.

To create one, go to Alerting > Silences > Add silence (or click Silence on an alert instance, which pre-fills the matchers from its labels). Fields:

  • Start and end (or a duration like 2h).
  • Matchers: label, operator (=, !=, =~, !~) and value, with the same semantics as notification policies. All matchers must match.
  • Comment: why you did this. Always write one; the next person will ask.
  • The creator is recorded automatically.

A silence has a state: Pending (starts in the future), Active, or Expired. You can edit an active one (which creates a new silence internally) or end it early. Silences are stored in the Alertmanager and need the right permission to create.

Use silences for unplanned, temporary situations: a known incident that fires many alerts, a noisy test run, a deployment you forgot to schedule.

Python Stand-in: does a silence cover this alert?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The critical alert is not silenced because the severity matcher excludes it, which is exactly the point of careful matchers: silence what you mean, nothing more.

Mute timings

A mute timing is a recurring window, defined once and attached to notification policies. During the window, that policy sends no notifications. Typical uses: "no non-critical alerts outside business hours", "no alerts during the weekly maintenance window".

Create one at Alerting > Notification policies > Mute Timings (or Time intervals in newer versions). A timing has one or more time intervals. Within one interval, fields combine with AND:

  • Time range: from and to as times of day, for example 22:00 to 06:00.
  • Days of the week: monday:friday or a list such as saturday, sunday.
  • Days of the month: 1, 1:7, or negative numbers to count from the end (-1 is the last day).
  • Months: names or numbers, ranges allowed.
  • Years.
  • Location: a time zone name such as America/New_York; default UTC.

Multiple intervals inside one timing combine with OR (the timing is active if any interval matches). Then, in a notification policy, choose Mute timings and select it. Newer versions also let you choose an active timing, the opposite: the policy only notifies during the interval (for example, "page only in business hours").

Python Stand-in: is a mute timing active right now?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two intervals are OR-ed: Wednesday 23:30 is muted by the night interval, and Saturday 11:00 by the weekend interval.

Pausing and disabling rules

Sometimes the alert itself is the problem. In the rule list you can pause evaluation of a rule (menu > Pause evaluation). A paused rule does not evaluate or fire at all, which is stronger than a silence. Use it for rules whose source is being migrated. Remember to resume it: paused rules are a classic source of "why did nothing alert?" incidents. Rules can be filtered by state in the list, so a monthly review should list paused ones.

Tool Scope Recurring Alerts still evaluate
Silence By label matchers, one time window No Yes
Mute timing One notification policy Yes Yes
Pause evaluation One rule No (manual) No

Reducing alert noise

Alert fatigue makes teams ignore pages. A practical plan:

  1. Measure: which alerts fire most, and which led to action? Remove or fix the rest.
  2. Alert on symptoms, not every cause: a user-facing error rate beats a CPU spike.
  3. Lengthen the pending period for blips; add keep firing where available to stop flapping.
  4. Group related alerts so a cluster failure is one message (see notification policies).
  5. Separate urgency: critical pages a human, warnings go to a channel, info goes to a dashboard.
  6. Use inhibition thinking: if the whole data center is down, suppress per-service alerts; Grafana's embedded Alertmanager supports silences and mute timings, and you can model dependencies with labels and routing.
  7. Document every alert with a runbook; an alert without a clear action is a candidate for deletion.
  8. Review on a schedule: a short monthly alert review keeps the system healthy.
flowchart TD
    A["Alert fires"] --> M{"Mute timing active<br/>for the policy?"}
    M -- yes --> X["No notification"]
    M -- no --> S{"Active silence<br/>matches labels?"}
    S -- yes --> X
    S -- no --> N["Notification sent<br/>to contact point"]

Notifications are checked against mute timings and silences before they leave.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the difference between a silence and a mute timing?

    Silences are ad hoc and matcher-based; mute timings are recurring and attached to notification policies.
  2. 2. Does an alert evaluate while it is silenced?

    Silences only suppress notifications; evaluation and state changes continue.
  3. 3. How do multiple intervals inside one mute timing combine?

    Any matching interval makes the timing active; the fields inside a single interval are AND-ed.
  4. 4. Which is the strongest, rule-level way to stop an alert completely?

    Pausing evaluation stops the rule from running at all, so it cannot fire.

Summary

  • Silences suppress notifications for alerts matching label matchers during one window; always add a comment.
  • Mute timings are recurring windows attached to notification policies (time of day, weekday, day of month, month, year, location).
  • Pausing evaluation stops a rule entirely; resume paused rules.
  • Reduce noise by alerting on symptoms, tuning pending periods, grouping, separating urgency and reviewing regularly.

Next: Provisioning with Files