Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 38 ยท Monitoring and Detection

Cloud Logging and Audit Trails

Learn which cloud logs matter (control plane, data plane, network, application), how to structure and protect them, and how to read an audit event, with runnable examples.

You cannot defend what you cannot see. When something goes wrong in the cloud, such as a leaked access key, a bucket suddenly opened to the public or an unexpected admin account, the first questions are always the same: who did it, what exactly happened, when, from where and what else did they touch? Logs answer those questions. If the right logs were not switched on before the incident, the answers are gone for good.

This lesson explains the kinds of cloud logs, what a good audit event contains, how to protect logs from tampering and how to start reading them. The log samples are invented but follow the shape that real providers use. All code is Python that runs in your browser, with no real accounts or services involved.

The four families of cloud logs

flowchart TD
    L["Cloud logs"] --> A["Control plane (management) logs"]
    L --> B["Data plane logs"]
    L --> C["Network logs"]
    L --> D["Application and OS logs"]
    A --> A1["Who changed which resource: API calls, console sign-ins"]
    B --> B1["Who read or wrote data: object access, DB queries"]
    C --> C1["Who talked to whom: flow logs, DNS, firewall, load balancer"]
    D --> D1["What your code and hosts did: app logs, syslog, containers"]

Most investigations need evidence from more than one family.

Family Examples by provider Typical question
Control plane AWS CloudTrail, Azure Activity Log, Google Cloud Audit Logs (Admin Activity) "Who created this access key?"
Data plane S3 server access/data events, Azure Storage logs, Cloud Audit Data Access logs "Who downloaded this file?"
Network VPC Flow Logs, NSG flow logs, VPC flow logs, DNS query logs, WAF logs "Which address connected to this server?"
Application and host Application logs, container stdout, OS audit logs "What did the process do?"

Control plane logs are usually on by default for recent events, but you must create a trail or a log sink to keep them longer and to include every region. Data plane and network logs often have to be enabled on purpose, because they are large and cost money. Decide early which resources deserve them: storage with sensitive data, databases, identity systems and internet-facing endpoints.

Anatomy of a good audit event

A useful event answers the five Ws: who, what, when, where from and with what result. This invented, CloudTrail-style event shows the fields:

JSON
{
  "eventTime": "2026-03-04T02:14:09Z",
  "eventSource": "iam.example.cloud",
  "eventName": "CreateAccessKey",
  "userIdentity": { "type": "AssumedRole", "principal": "ci-deploy-role", "mfa": false },
  "sourceIPAddress": "198.51.100.23",
  "userAgent": "example-cli/2.1",
  "requestParameters": { "userName": "alice" },
  "errorCode": null,
  "recipientAccount": "111122223333",
  "region": "eu-west-1"
}

Read it as a sentence: at 02:14 UTC on 4 March, the ci-deploy-role (without MFA) called from 198.51.100.23 asked to create an access key for user alice, and it succeeded. Whether that is normal depends on context, which is why detection needs a baseline (see detecting suspicious activity).

Try it: parse and summarise audit events

Python Summarise a batch of audit events

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

DeleteTrail stands out: someone turned off a logging trail. Attackers often try to blind defenders this way, so changes to logging configuration deserve a high-priority alert.

Make logs trustworthy

Logs are evidence, so protect them like evidence:

  1. Send them somewhere else. Store logs in a dedicated logging account or project with its own administrators, separate from the workloads. If an attacker takes over the workload account, they should not be able to erase the history.
  2. Make them append-only. Use object versioning, write-once (object lock/immutable) storage and restrictive policies so that even admins cannot delete logs before the retention period ends.
  3. Turn on integrity validation. Many providers can sign digest files for log batches so that tampering can be detected.
  4. Encrypt logs at rest and in transit with managed keys.
  5. Restrict reading as well: logs contain IP addresses, user names and sometimes secrets that were logged by mistake.
  6. Synchronise clocks. Correlating events across systems needs consistent UTC timestamps.

Try it: tamper-evident logs with a hash chain

Integrity validation often relies on chaining: each record's hash includes the hash of the previous record, so changing or deleting one record breaks everything after it. This stand-in uses SHA-256 from Python's standard library.

Python Detect a tampered log entry

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The edit at index 2 is detected at exactly that position. In a real system the final digest is also stored or signed somewhere the attacker cannot reach, otherwise they could recompute the whole chain.

Structure and retention

  • Use structured logs (JSON) with consistent field names, so queries and detections do not depend on fragile text matching.
  • Include correlation IDs (request ID, trace ID) so one user action can be followed across services.
  • Never log secrets, passwords, full card numbers or tokens. Mask or hash sensitive fields.
  • Set retention to match needs. Hot, searchable storage for weeks to months for investigations; cheaper archive storage for a year or more when regulations require it (see compliance frameworks).
  • Watch the cost. Log volume can explode. Sample noisy debug logs, filter health checks, but do not filter security-relevant events to save money.

A minimum logging baseline

Enable these in every account or subscription on day one: the management/audit log across all regions, sign-in and identity provider logs, storage access logs for sensitive buckets, DNS and network flow logs for production networks, load balancer and WAF logs for public endpoints, and application logs from every service. Ship all of it to central storage and then to your alerting and SIEM system.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which log family tells you who created an access key?

    Creating an access key is a management API call, so it appears in the control plane audit log.
  2. 2. Why store logs in a separate logging account?

    Separation of duties protects evidence. Combine it with immutable storage and restricted delete permissions.
  3. 3. In a hash chain, what happens if one record in the middle is edited?

    Each hash depends on the previous hash, so an edited record breaks the chain from that point forward.
  4. 4. Which event should usually raise a high-priority alert?

    Disabling logging is a classic step to hide activity, so changes to logging configuration need immediate review.

Summary

  • Cloud logs come in four families: control plane, data plane, network and application; incidents usually need several.
  • A good event records who, what, when, where from and the result, in structured form.
  • Keep logs in a separate, write-once, encrypted location with integrity validation and consistent time.
  • Never log secrets, balance retention against cost and alert when logging itself is changed.

Next: Detecting Suspicious Activity