Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 37 ยท Detection Rules and Correlation

Tuning, False Positives and Rule Quality

Cut alert noise without creating blind spots: measure precision, add safe exclusions, use allow-lists and suppression, and track rule quality with runnable Python examples.

A SIEM that raises thousands of alerts a day trains analysts to ignore it, and then it misses the one that matters. This is alert fatigue. Tuning is the ongoing work of making rules quieter and sharper while keeping them sensitive to real attacks. It is never finished, but it can be done safely with a method.

Measure before you change

For each rule, keep counts from the cases analysts closed:

Measure Formula Meaning
Precision TP / (TP + FP) Of the alerts, how many were real
Volume alerts per day Workload on analysts
Mean time to triage average minutes Cost per alert
Benign TP share benign TP / alerts Context problems rather than rule problems

Recall (how many real attacks you catch) is harder to measure; you estimate it with purple-team exercises and your lab's planted events (lesson 140).

Python Rule scorecard: which rules to tune first

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The rule that wastes the most hours (a scanner that was recorded as 140 benign TPs) is the best first target: an exclusion for the authorized scanner removes most of its noise.

Tuning techniques

  1. Narrow the logic. Add conditions that attackers must satisfy but normal activity rarely does (parent process, path, rare command-line flags).
  2. Exclude known-good, precisely: a specific source host plus a specific action, not "all of subnet 10.0.0.0/8".
  3. Raise the threshold (with data to show the effect).
  4. Add context: asset criticality, user role, threat-intel match.
  5. Lower the severity or route to a report rather than a page.
  6. Suppress duplicates: one alert per entity per hour.
  7. Split a rule that mixes risky and harmless cases.
  8. Retire rules that never lead to action.

Allow-lists done safely

An allow-list is a data table the rule consults. It should hold: the value, the reason, who approved it, and an expiry or review date. Without these, exclusions silently become permanent blind spots that attackers can abuse.

Python Allow-list with reasons and expiry

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Suppression and deduplication

Even a good rule can fire hundreds of times for one incident. Group by a key and alert once per period, counting the rest.

Python Deduplicate repeated alerts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Don't tune blindly

Every exclusion is a hole. Ask: could an attacker deliberately look like this exception? For example, excluding anything run from C:\Temp lets an attacker simply run from C:\Temp. Prefer exclusions that include a verified identity (signed binary, specific host, specific account), not just a path or name.

Test the effect of a change against historical data:

  1. Replay last month's logs against the new rule.
  2. Confirm all known true positives still alert.
  3. Count the alerts removed.
  4. Roll out and watch for two weeks.
Python Compare old and new rule on history

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The new rule keeps all three attacks and drops 40 noisy alerts: a safe tune. If it had missed an attack, you would reject the change.

Culture and process

  • Give analysts a one-click way to mark "false positive" with a reason; it feeds tuning.
  • Hold a weekly tuning meeting on the top noisy rules.
  • Track the alert-to-incident ratio and trend it.
  • Celebrate removing noise as much as adding detections.

A silent rule is not necessarily a good rule. Test that quiet rules still fire, using planted events or purple-team exercises, so you know they work.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does precision measure?

    Precision is TP divided by all alerts (TP plus FP).
  2. 2. Which exclusion is safer?

    Narrow, documented exclusions with an owner and review date limit the blind spot.
  3. 3. Why replay historical logs before deploying a tuned rule?

    Replay shows the effect of a change before it affects live monitoring.
  4. 4. What is alert fatigue?

    Noise erodes attention, which makes real attacks easier to miss.

Summary

  • Measure precision, volume and triage time per rule, and tune the biggest time sinks first.
  • Prefer narrow, documented, expiring exclusions over broad ones.
  • Deduplicate and suppress repeats so one incident produces one alert.
  • Test changes on history and keep quiet rules honest with planted events.

Next: The Incident Response Lifecycle