Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 37 ยท Threat Intelligence and Hunting

Threat Hunting Methodology

Learn how proactive threat hunting works: forming hypotheses, choosing data, analyzing for anomalies, documenting results and turning hunts into detections.

Detection rules catch what you already know to look for. Threat hunting is the proactive search for attackers who got past those rules. A hunter starts from the assumption that a determined adversary may already be inside, then looks through the data for evidence of how they would behave. Hunting is human-led and investigative, unlike alerts, which are machine-led and reactive.

Good hunting has two outcomes you can count on, even if no attacker is found: it finds visibility gaps (data you wish you had), and it produces new detections that run automatically forever after.

Hunting versus alerting

Alert-driven work Hunting
Trigger A rule fires A question or hypothesis
Assumption Something specific happened Something may be hiding
Output Close, escalate Findings, new rules, data gaps
Cadence Continuous Scheduled campaigns

The hunting loop

flowchart LR
    H["1. Form hypothesis"] --> D["2. Identify data and scope"]
    D --> Q["3. Query and analyze"]
    Q --> F{"Finding?"}
    F -- "Yes" --> I["Open incident"]
    F -- "No" --> R["4. Record result and gaps"]
    I --> R
    R --> A["5. Automate as detection"]
    A --> H

Every hunt ends with a written result, even when nothing was found.

1. Form a hypothesis

A hypothesis is a testable statement about attacker behavior in your environment, and it should name a technique and a data source. Sources of ideas:

  • A threat report: "Actors targeting our sector use scheduled tasks for persistence."
  • The ATT&CK matrix: pick techniques with weak detection coverage.
  • A past incident: "Could the same trick have worked on other servers?"
  • Baseline oddities: "Why do some servers make outbound connections at night?"

Weak: "Look for malware." Strong: "If an attacker used a web shell on our internet-facing servers, the web server process would spawn a command shell, which it normally never does."

Hunt types

Hypothesis-driven Test an idea about attacker behavior
Intel-driven Look for indicators from a report
Data-driven Find anomalies in baselines

Most mature hunts combine all three.

1 / 2

2. Identify data and scope

Decide which logs can show the behavior: process creation, authentication, DNS, proxy, cloud audit logs. Check that the data exists and covers the systems in scope for the period you need. If it does not, the finding is the gap itself. Write the scope down: which systems, which dates, which fields.

3. Query and analyze

Techniques you will use again and again:

  • Search for known-bad behavior: a particular parent-child process pair.
  • Stack counting (frequency analysis): count how often each value occurs and examine the rarest. Malware often appears on one or two machines while normal software appears on hundreds.
  • Baselining and outliers: compare to what is normal for a user, host or time of day.
  • Timeline and pivoting: when you find one oddity, pivot on its host, user, IP and time to see what else surrounds it.
  • Grouping and sequencing: events that are harmless alone but suspicious in sequence.

4. Record the result

Write a short hunt report: hypothesis, data, queries, results, conclusion, follow-ups. Negative results are valuable: they show a technique was checked on a date, with these queries, and nothing was found.

5. Automate

If the hunt produced a reliable query, convert it into a detection rule and test it (hunting with log queries). Hunting should keep moving to new questions rather than repeating the same one every month.

Try it: stack counting to find the rare

You hunt for suspicious scheduled-task creation. Among the many creations, legitimate tools create the same tasks on many hosts; the rare ones deserve a look. The data is invented.

Python Frequency analysis of task names

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice svchosts-sync, a name that imitates the real svchost Windows process. Rare plus look-alike is a classic reason to pivot. BackupNightly is also rare but probably benign: you would confirm with the host owner, which is why hunting still needs human judgment.

Try it: spot a regular beacon

Command-and-control software often "phones home" at regular intervals. Human browsing is irregular. This stand-in measures how regular the gaps between connections are.

Python Beaconing regularity

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Regularity alone is not proof (software update checks are regular too), so a hunter then asks what the destination is, how rare it is across the company, and which process opened the connection.

Maturity and roles

Teams grow from ad-hoc searching on indicators, to scheduled hypothesis-driven hunts, to statistical and automated analysis with a feedback loop into detection engineering. A hunter needs knowledge of operating systems, networks and attacker behavior, curiosity and patience, and the ability to write queries. Hunting needs good data first: if endpoint and authentication logs are missing, fix collection before hiring a hunter (see chapter 3 on log collection).

Hunt only in data and on systems you are authorized to examine, and follow privacy rules for personal data in the logs.

Common mistakes

  • Starting with no hypothesis and wandering through data.
  • Hunting on data that does not exist or is incomplete, and not noticing.
  • Not documenting negative results, so the same hunt is repeated.
  • Never converting successful hunts into detections.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What distinguishes hunting from alert handling?

    Hunters go looking for what rules missed; analysts respond to what rules found.
  2. 2. What is a good hypothesis?

    A strong hypothesis is specific, testable and names behavior and data.
  3. 3. Why is stack counting useful?

    Legitimate software appears widely; threats and mistakes often appear on very few systems.
  4. 4. A hunt finds no attacker. What should happen next?

    Negative results, gaps and automation are the lasting value of a hunt.

Summary

  • Hunting is proactive, hypothesis-driven and human-led, complementing automated detection.
  • The loop: hypothesis, data and scope, query and analyze, record, automate.
  • Stack counting, baselining, pivoting and sequence analysis are core techniques.
  • Every hunt yields incidents, visibility gaps or new detections, so document even empty results.

Next: Hunting with Log Queries