Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 38 ยท Projects and Practice

Project: A Cloud Log Threat Hunter

Build a threat-hunting tool for cloud audit logs: parse events, learn a baseline per identity, flag rare actions and new addresses, score sessions and write an incident summary.

Threat hunting is the proactive search for signs of an attacker that your alert rules did not catch. Instead of waiting for an alarm, a hunter forms a hypothesis ("a stolen key would be used from an address and with API calls that this identity has never used") and looks for evidence in the logs. In this project you will turn that hypothesis into a small tool.

The tool reads audit events, learns what is normal for each identity from a quiet period, flags deviations in a later period, scores the suspicious sessions and writes a short summary. It uses ideas from cloud logging, detecting suspicious activity and incident response. The data is invented, with addresses from the documentation ranges. Only hunt in logs you are authorised to analyse.

The hunting loop

flowchart LR
    H["Hypothesis"] --> D["Collect and parse data"]
    D --> B["Build baseline"]
    B --> F["Find deviations"]
    F --> S["Score and triage"]
    S --> R["Report and respond"]
    R --> N["New detection rule"]
    N --> H

Every successful hunt should end as a permanent, automatic detection.

Step 1: parse the logs and build baselines

We use one week of normal history for three identities and the following day as the "hunt window". The baseline records, per identity, the set of actions, the set of source addresses and the usual active hours.

Python Step 1: baselines from history

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A baseline from only three days is thin; real hunting uses weeks and handles seasonality such as month-end jobs. The principle is identical.

Step 2: find deviations and score them

For each event in the hunt window we check four signals: a new action for this identity, a new source address, an unusual hour (outside the identity's seen hours by more than an hour) and a high-risk action such as creating keys or changing logging. Each signal adds to a score. Scores are summed per identity, because several weak signals together are more convincing than one.

Python Step 2: score deviations per identity

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

bob scores very high: unfamiliar address, night-time activity, a key creation and logging being switched off. ci-deploy shows a medium signal, a deployment from a new address in the afternoon, which could be a new runner or a stolen token. Hunters must also reason about innocent explanations, which is why the next step collects context.

Step 3: the incident summary

A hunt is only useful when its result can be acted on. The final step turns the highest-scoring identity into a summary with a timeline, suspected technique and first actions. The techniques below are labelled in the style of the MITRE ATT&CK for Cloud matrix.

Python Step 3: summarise the lead

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

That last line is the most important one. A hunt that ends with a new automatic rule improves everybody's defence permanently. For ways to run such rules continuously and route their alerts, see alerting and SIEM.

Ideas for extending the hunter

  1. Rolling baselines: recompute baselines weekly and weight recent days more.
  2. Peer groups: compare an identity with others in the same role. If every deployer uses UpdateFunctionCode but only one calls CreateAccessKey, that one stands out.
  3. Rare-event statistics: compute how many identities ever used each action and give rare actions higher weight.
  4. Session grouping: group events by identity and source address with a 30-minute gap, then score sessions, not single events.
  5. Enrichment: add geolocation, a threat-intelligence list or asset owners to each address.
  6. Hypothesis library: keep a file of hunts (stolen key, rogue role, data staging in storage) with their queries and outcomes.
  7. Metrics: record how many hunts led to new rules, false-positive rate and time spent.

Avoiding hunting mistakes

  • Confirmation bias: look for evidence that would disprove your hypothesis as well.
  • Missing data: if a log source is off, the hunt cannot see that part. Report such gaps; they are findings too.
  • Privacy: logs contain personal data. Limit access, record who hunts, and keep results only as long as necessary (data privacy).
  • Scope and authorisation: hunt only in systems and logs you are authorised to analyse, and follow your incident procedures when you find something real.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the difference between threat hunting and alert-based detection?

    Alerts react to predefined rules. Hunters look for what the rules missed, then convert useful findings into new rules.
  2. 2. Why does the tool add scores from several weak signals together?

    A new address alone may be a VPN. A new address, night activity and a logging change together are much more suspicious.
  3. 3. What should a successful hunt produce besides a report?

    Turning findings into permanent detections is how hunting improves your defences over time.
  4. 4. Why is a baseline from only three days of history a weakness?

    Short baselines do not capture weekly or monthly patterns. Longer history and peer comparison reduce noise.

Summary

  • Threat hunting starts with a hypothesis and searches logs for deviations from normal behaviour.
  • A per-identity baseline of actions, addresses and hours makes new activity visible; weak signals are combined into a score.
  • A lead is summarised with a timeline, techniques and first actions, and every good hunt becomes a permanent rule.
  • Watch for bias, missing data and privacy, and hunt only where you are authorised.

Next: Cloud Security Review and Checklist