Skip to content
Tutorials Center
My learning

Topics

Lesson 21 of 36 ยท Network and Log Forensics

Web Server Log Analysis

Learn to read Apache and Nginx access logs, parse them with a regular expression, and spot scanning, brute-force and suspicious requests during an investigation.

Web server logs are among the most useful records in an incident. Every request leaves a line: who asked, when, for what, and how the server answered. They survive when attackers wipe files, and they are easy to process with a short script. This lesson teaches you to read the common log format, parse it, and recognize the patterns that suggest scanning or abuse.

This lesson analyzes logs from servers you own or are authorized to investigate. The sample log is invented. Do not run scanners or probes against systems you do not control.

The combined log format

Apache and Nginx write the combined log format by default:

TEXT
203.0.113.9 - - [12/Nov/2026:14:03:11 +0000] "GET /index.php?id=5 HTTP/1.1" 200 5123 "-" "Mozilla/5.0"
Field Example Meaning
Client address 203.0.113.9 Who connected (maybe a proxy)
Identity, user - - Almost always empty; user is set under HTTP auth
Timestamp [12/Nov/2026:14:03:11 +0000] When the request finished, with offset
Request line "GET /index.php?id=5 HTTP/1.1" Method, path, protocol
Status 200 Server's answer (2xx ok, 3xx redirect, 4xx client error, 5xx server error)
Size 5123 Response bytes
Referer, user agent "-" "Mozilla/5.0" Claimed by the client; easily forged

Two cautions. Everything the client sends (path, referer, user agent) is attacker-controlled text, so never trust it blindly or paste it into a web page unescaped. And the address may be a load balancer or CDN; look for X-Forwarded-For logging.

Parsing with a regular expression

Python Parse access log lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the UNPARSED branch: in forensics, a line that does not match is itself a finding (a corrupted log, a different format, or an injection attempt). Count and report such lines instead of discarding them.

Patterns worth looking for

  • Scanning: many 404 responses from one address across many different paths (/wp-login.php, /admin, /.env).
  • Brute force: repeated POST to a login path with 401 or 403, followed by a 200.
  • Injection attempts: ', %27, UNION SELECT, ../, <script> in query strings.
  • Data theft: unusually large response sizes or many sequential requests to download endpoints.
  • Odd methods or agents: PUT, DELETE, TRACE, or user agents that name scanning tools.

A success after a run of failures is the most important pattern: it may mean an account was guessed.

Python Detect scanning and login brute force

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The thresholds (3 here) are arbitrary: real values depend on traffic. Tune them against a baseline, and prefer to rank suspects rather than decide automatically.

Practical cautions

  • Time zones: the log offset (+0000) is the server's; convert everything to UTC before comparing with other sources.
  • Log rotation: older entries sit in access.log.1, access.log.2.gz. Collect them all and hash them.
  • Truncation and tampering: an attacker with root can edit logs. Compare with copies on a remote log server, and look for gaps in time or request counters.
  • Proxy addresses: behind a CDN every client may share the CDN's address; use the forwarded-for header when present.

Related reading: authentication logs and timelines.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In a combined log line, which field is fully controlled by the client?

    The user agent and referer are sent by the client and are easily forged, so treat them as untrusted text.
  2. 2. What is the strongest sign of a guessed account in the logs?

    Repeated failures then a success on the login endpoint suggests password guessing that worked.
  3. 3. What should you do with lines your regex cannot parse?

    Unparsed lines can be corrupted or odd input, which is itself relevant. Never silently drop evidence.
  4. 4. Why convert log times to UTC?

    A common time base lets you merge logs from different systems into one timeline.

Summary

  • Combined logs record client, time, request, status, size, referer and user agent.
  • Parse with a regular expression and keep a counter of unparsed lines.
  • Look for scanning (many 404s), brute force (401s then 200) and injection strings.
  • Normalize to UTC, collect rotated logs, and compare with remote copies.

Next: Authentication Log Investigation