Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 37 ยท Log Collection and Normalization

Timestamps, Time Zones and Enrichment

Get event time right and add context. Learn UTC, event versus ingest time, clock drift, plus enrichment with asset, user and reputation data, with runnable Python examples.

Two things turn a pile of events into evidence: correct time and useful context. If clocks disagree, you cannot order events and correlation fails. If events lack context, analysts waste time looking things up. This lesson covers both.

Time rules

  1. Store in UTC. Convert to local time only when displaying. Mixing zones is the most common cause of wrong timelines.
  2. Use ISO 8601: 2026-11-12T10:15:02.123Z. The Z means UTC.
  3. Keep two timestamps. @timestamp is when the event happened (from the source); event.ingested is when the SIEM received it. The gap shows delays.
  4. Synchronize clocks with NTP on every system, including network devices.
  5. Record the source zone for logs that lack one (syslog 3164).
Python Normalize mixed time formats to UTC

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Three different inputs, three different local times, one UTC format. Notice we had to add the year and zone ourselves for the syslog line, and documented that assumption.

Clock drift and event versus ingest time

Suppose a server's clock runs 7 minutes fast. A correlation rule that expects "failed login, then success within 2 minutes" may see events in the wrong order. Always compare event time with ingest time to spot drift and delays.

Python Find sources with clock drift or delivery lag

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Negative lag (event in the future) means the source clock is ahead; large positive lag means delayed delivery or a clock behind.

Enrichment

Enrichment adds fields the log did not contain. Do it at ingest for stable facts (so searches are fast and the value is frozen at event time) or at search time for facts that change (so you always see current data).

Enrichment Adds Example use
Asset inventory Owner, criticality, role Escalate alerts on domain controllers
User directory Department, manager, privileged flag Spot a finance user on a build server
GeoIP Country, city, ASN Impossible travel, unusual country
Threat intelligence Reputation, campaign Match bad IPs and domains
DNS / reverse lookups Hostname for an IP Readable investigations
CMDB / vulnerability data Known weaknesses Prioritize attacks on unpatched hosts
Python Enrich events with asset, user and geo data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The country names and AS numbers are invented. In production you would use a maintained GeoIP database and keep it updated.

Pitfalls

  • Stale data: an asset list that is a year old mislabels hosts. Automate updates.
  • Dynamic addresses: DHCP changes who owns an IP. Join on time, not only on address.
  • Over-enrichment: every added field costs storage and time. Add what rules and analysts really use.
  • Trusting enrichment blindly: GeoIP can be wrong, and attackers use VPNs. Treat it as a hint.
  • Privacy: attaching personal data to every event widens who can see it.

Store the enrichment source and date with the value when it matters (for example threat.list_version), so an old alert can be explained months later.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In which time zone should event timestamps normally be stored?

    UTC prevents ordering errors and daylight-saving confusion; convert for display only.
  2. 2. What does a negative lag (event time after ingest time) usually indicate?

    An event cannot occur after it is received, so the source clock is wrong.
  3. 3. Which enrichment helps decide how serious an alert on a host is?

    Alerts on domain controllers or other critical assets deserve higher priority.
  4. 4. Why join DHCP-assigned IPs to hosts using time?

    Without time, you may attribute activity to the wrong device.

Summary

  • Store timestamps in UTC (ISO 8601) and record both event and ingest time.
  • Sync clocks with NTP and monitor drift and lag per source.
  • Enrich with asset, user, GeoIP and threat-intelligence data, either at ingest or at search time.
  • Keep enrichment data fresh, join on time, and treat hints like GeoIP with caution.

Next: Detection Rule Basics