Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 38 ยท Monitoring and Detection

Alerting and SIEM in the Cloud

Understand how a SIEM collects, normalises and correlates cloud logs, how to design alerts people can act on, and how to cut noise with deduplication and routing.

A SIEM (security information and event management system) is the place where logs from many sources come together so that you can search them, correlate them and raise alerts. Examples include Microsoft Sentinel, Google Chronicle, Splunk and Elastic Security, and each cloud also has native services such as AWS Security Hub with GuardDuty, Microsoft Defender for Cloud and Google Security Command Center. The names differ, but the job is the same: turn a flood of events into a short list of things a human should look at.

Good alerting is as much about people as about technology. An alert nobody reads is worse than no alert, because it creates a false sense of safety. This lesson covers the SIEM pipeline, how to write alerts that are actionable and how to control noise. The runnable code simulates the main steps on invented data.

The SIEM pipeline

flowchart LR
    S1["Cloud audit logs"] --> I["Ingest"]
    S2["Network and DNS logs"] --> I
    S3["Identity provider logs"] --> I
    S4["App and container logs"] --> I
    I --> N["Parse and normalise"]
    N --> E["Enrich: geo, asset owner, threat intel"]
    E --> D["Detect and correlate"]
    D --> A["Alert, case and ticket"]
    A --> R["Respond: SOAR playbook or human"]
    N --> Q[("Searchable store")]

Parse, enrich, detect, then route. Stored data supports later hunting and investigations.

  1. Ingest. Pull logs from storage buckets, streaming services or agents. Check that every source is actually arriving: a silent source is a blind spot.
  2. Normalise. Different sources name the same thing differently (sourceIPAddress, client_ip, src). Map them to one schema (for example the Elastic Common Schema or OCSF) so one rule works on all sources.
  3. Enrich. Add context: geolocation, whether the IP is on a threat list, which team owns the account, whether the asset is production.
  4. Detect and correlate. Run rules and analytics as described in detecting suspicious activity.
  5. Alert and case management. Create alerts, group related alerts into incidents and assign them.
  6. Respond. SOAR (security orchestration, automation and response) tools run playbooks: disable a key, isolate an instance, open a ticket.

Try it: normalise events from two sources

Python Map different log formats to one schema

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

After normalisation, one query such as src_ip = X finds the address in all sources.

Designing alerts people can act on

Every alert should pass four tests:

  • Actionable. There is a clear next step. "CPU high" is not a security alert; "root user signed in without MFA" is.
  • Specific. It names the who, what, where and when, and links to the raw events.
  • Prioritised. Severity reflects both the likelihood of malicious activity and the value of the asset.
  • Owned. Someone is on call for it, with a runbook that says how to confirm, contain and escalate.

A good alert message looks like this:

TEXT
[HIGH] Audit logging disabled
account: 111122223333 (prod)   actor: bob   source: 192.0.2.77
event: StopLogging on trail org-trail at 2026-03-04T02:20:00Z
runbook: https://wiki.example.com/runbooks/logging-disabled
first step: confirm change ticket; if none, disable bob's sessions and re-enable the trail

Cutting noise

Noise comes from duplicates, flapping and low-value rules. Three techniques cover most of it:

  • Deduplication: the same rule firing for the same entity within a window becomes one alert with a counter.
  • Rate limiting and grouping: group alerts by user, host or incident, so a hundred events become one case.
  • Severity routing: page a person only for critical alerts, send medium ones to a queue for the working day and keep low ones as searchable data.

Try it: dedupe and route alerts

Python Deduplicate alerts and route by severity

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Five raw bruteforce alerts for alice become two grouped alerts (a burst of three, and a later single one), and only the logging change pages anybody.

Tuning is a continuous job

  • Review the top ten noisiest rules every week and the false-positive rate of each.
  • Keep an exception list with owner and expiry date.
  • Add suppressions narrowly (one account, one rule), never by disabling a rule globally.
  • Track metrics: mean time to detect (MTTD), mean time to respond (MTTR), alerts per analyst per day and percentage of alerts closed as false positives.
  • Run exercises: replay known-bad data or perform authorised attack simulations in a test account and confirm that alerts arrive and are understood.

Cloud-native alerts versus a SIEM

Cloud-native services (GuardDuty, Defender for Cloud, Security Command Center) are quick to switch on and understand their own platform deeply, so enable them first. A SIEM adds cross-cloud and cross-tool correlation, long retention, your own custom rules and case workflow. Many organisations use both: native services produce findings, which the SIEM ingests alongside other logs. See posture management and misconfiguration for the preventive side.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does a SIEM normalise log fields?

    Different sources name the same data differently. A common schema lets a detection query use one field name everywhere.
  2. 2. Which alert is most actionable?

    An actionable alert is specific, includes context and points to the next step.
  3. 3. What is deduplication of alerts for?

    Grouping repeated alerts reduces noise so analysts see one case instead of many identical ones.
  4. 4. A log source stops sending data. What is the risk?

    Missing data means missing detection. Monitor ingestion health and alert on sources that go quiet.

Summary

  • A SIEM ingests, normalises, enriches, detects, alerts and supports response; native cloud services and a SIEM complement each other.
  • Alerts must be actionable, specific, prioritised and owned, with a runbook.
  • Deduplicate, group and route by severity to keep people focused.
  • Monitor ingestion health, tune rules continuously and measure MTTD and MTTR.

Next: Cloud Incident Response