Skip to content
Tutorials Center
My learning

Topics

Lesson 16 of 37 ยท Detection Rules and Correlation

Correlation Rules and Event Sequences

Combine events from different sources into one detection: ordered sequences, joins, lookups and risk scoring. Build working correlation logic in Python on synthetic events.

One event is a fact. Several related events are a story. Correlation is the SIEM's ability to connect events across time and sources so a pattern that no single log shows becomes visible. It is the capability that separates a SIEM from a log search tool.

Four kinds of correlation

Kind Example
Sequence A, then B within 10 minutes, same user
Join across sources VPN login joined with a firewall deny from the same IP
Aggregation + condition 20 failures AND a success
Absence Log source sends nothing, or a scheduled backup did not run

Sequence rules

Sequences require an ordering and a join key that ties events to the same actor: a user, a host, a session ID or an IP.

flowchart LR
    A["Phishing link clicked<br/>(proxy)"] --> B["Macro runs PowerShell<br/>(endpoint)"]
    B --> C["New scheduled task<br/>(endpoint)"]
    C --> D["Outbound to rare domain<br/>(DNS)"]
    D --> E["ALERT: likely intrusion chain"]

Each step alone is weak evidence; together they are strong.

Python Ordered sequence within a time window

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Host pc17 completes the three steps inside ten minutes and alerts. Host pc22 starts the chain but never finishes it in time, so it does not.

Joining sources

Joins add information from a second dataset. Typical joins: sign-in logs with HR terminations ("login by a departed employee"), firewall logs with threat intelligence, or endpoint events with vulnerability data.

Python Join VPN logins with the terminated-users list

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Risk-based alerting

Instead of alerting on every weak signal, give each signal a risk score attached to an entity (user or host). Alert only when an entity's total crosses a limit in a period. This reduces noise because many harmless events never reach the limit, while a real attacker triggers several.

Python Entity risk scoring

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Absence rules

Alert when something expected does not happen: a heartbeat stops, a backup job logs no success by 04:00, a domain controller sends no events. These are easy to forget and valuable.

Practical advice

  • Pick the right join key: hostnames change case, IPs change owners. Normalize first.
  • Limit the window: long windows cost memory and create accidental matches.
  • Think about ordering delays: events may arrive out of order; use event time, and allow a small grace period.
  • Explain the chain: the alert should list every contributing event so an analyst sees the story.
  • Test with a full scenario from your lab data, not only isolated events.

Correlation in SIEM products appears under names such as "correlation searches", "sequence rules", "analytics rules", "detection rules with entity mapping" or "UEBA". The logic is the same as shown here.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is a join key in a correlation rule?

    The join key (user, host, session, IP) links events that belong to the same story.
  2. 2. Why use risk-based alerting?

    Aggregating scores per entity reduces alert volume and highlights multi-step behavior.
  3. 3. What does an absence rule detect?

    Missing heartbeats, backups or log sources are signals too.
  4. 4. What should a sequence alert include?

    The story is the value; listing every step lets analysts verify it quickly.

Summary

  • Correlation links events by sequence, joins, aggregation and absence.
  • A join key and a bounded time window are essential.
  • Risk scores per entity turn many weak signals into fewer, better alerts.
  • Always show the contributing events and test with full scenarios.

Next: Sigma Rules: Vendor-Neutral Detections