Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 37 ยท Projects and Practice

Project: A Brute-Force Detection Pipeline

Build a mini SIEM pipeline step by step: parse SSH-style log lines, normalize events, detect failed-then-successful logins in a sliding window and raise enriched alerts.

In this project you build a small but complete detection pipeline: raw log lines go in, enriched alerts come out. It touches every stage a real SIEM performs (collect, parse, normalize, correlate, enrich, alert) at a size you can read in one sitting. The log lines are invented, in the style of an SSH server's authentication log, and the addresses come from reserved documentation ranges. Nothing connects to anything.

flowchart LR
    A["Raw log lines"] --> B["Parse with regex"]
    B --> C["Normalize to common fields"]
    C --> D["Correlate: sliding window per source and user"]
    D --> E["Enrich: watchlist, asset, geo"]
    E --> F["Alert with severity and next step"]

Six stages. Each step below adds one.

Step 1: parse and normalize

Logs arrive as text. Parsing extracts fields; normalizing renames them to a common schema (time, user, src_ip, outcome) so that later steps do not care which system produced the line.

Python Step 1: parse log lines into events

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that a line that does not match is counted, not ignored. A parser that silently drops lines is how detection quietly breaks when a log format changes.

Step 2: correlate with a sliding window

The rule: raise an alert when a source address has at least N failures within W seconds and then a success. The loop keeps, per source address, the recent failures.

Python Step 2: failed-then-success rule

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The window is the heart of the rule. Carol's one failure is below the threshold. The external address with four failures across two account names, then a success, trips it.

Step 3: enrich and score

Add the context from IOCs and enrichment: is the source internal, is it on a watchlist, is the account privileged? Then compute severity and a recommended action, in the spirit of triage.

Python Step 3: enrich the alert and choose a response

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Step 4: test the rule

A rule without tests is a rumor. Write cases with known right answers: logs that must alert and logs that must not. Run them whenever the rule changes.

Python Step 4: unit tests for the detection logic

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Extend the project

Try these on your own (keep every experiment on invented data):

  1. Spray detection: alert when one source tries many different users with few attempts each (password spraying), which the per-user threshold misses.
  2. Impossible travel: add a table of country by address and alert when the same user logs in from two far-apart countries within an hour.
  3. Success outside hours: raise severity for successes between 00:00 and 05:00 for non-shift accounts.
  4. Output: write each alert as a JSON line with the ATT&CK technique T1110 (coverage).
  5. Tuning: add an allow-list for your vulnerability scanner with an owner and an expiry (tuning).

What you practiced

  • Regex parsing with a counter for unparsable lines.
  • A sliding-window correlation rule keyed by source address.
  • Enrichment, scoring and a response recommendation.
  • Test cases for both alerting and not alerting.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why should a parser count lines it cannot parse?

    Silent drops create blind spots; counting and reviewing them reveals format changes.
  2. 2. What does the sliding window do in the correlation rule?

    Old failures expire, so slow, harmless failures do not accumulate into an alert.
  3. 3. Why keep state per source address?

    Grouping by entity is what turns many events into a meaningful sequence.
  4. 4. Which test is essential for a detection rule?

    Testing both directions guards precision and recall.

Summary

  • A pipeline is parse, normalize, correlate, enrich, alert, and each stage can be tested separately.
  • Sliding windows and per-entity state turn raw events into behavior-based detections.
  • Enrichment and scoring decide severity and the next step.
  • Unit tests with positive and negative cases keep a rule trustworthy as it changes.

Next: Project: A Mini Threat Hunt