Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 31 ยท Web Application Security Concepts

Reading Web Server Logs for Attacks

Learn the access log format, the request patterns that point to scanning, injection and brute force, and how to summarize a log with a Python stand-in for blue-team detection.

Every web request leaves a trace. The server writes one line to an access log with who asked, what they asked for, and how the server answered. For a defender, that file is a free flight recorder: long before anyone notices a defaced page, the log has usually shown a stranger knocking on doors that no real visitor ever tries.

This lesson is the blue-team side of Injection and Input Validation: Defense. You will learn the common log format, the patterns that separate normal traffic from probing, and how to count and rank them. The log files and the tools (grep, awk, less) are real; here the data is a small made-up sample so every result is reproducible. Nothing in this lesson touches a real server.

What a log line answers

Who The client IP address.
When A timestamp with time zone.
What Method, path and query string.
Outcome Status code and bytes sent.

Four questions, one line per request.

1 / 2

The combined log format

Apache and Nginx both default to a layout called the combined log format. One request is one line:

TEXT
203.0.113.9 - - [12/Mar/2026:10:15:32 +0000] "GET /index.html HTTP/1.1" 200 5120 "https://example.org/" "Mozilla/5.0 (X11; Linux x86_64)"

The fields, in order:

Field Example Meaning
Client address 203.0.113.9 The IP that connected (behind a proxy, this may be the proxy)
Identity and user - - Almost always dashes; the user shows up if HTTP authentication is used
Time [12/Mar/2026:10:15:32 +0000] When the request arrived, with offset
Request line "GET /index.html HTTP/1.1" Method, target and protocol
Status 200 The server's answer code
Size 5120 Response body bytes
Referrer "https://example.org/" The page that linked here, if the client says so
User agent "Mozilla/5.0 ..." What the client calls itself

Two cautions shape everything that follows. First, every field except the server's own timestamp and status is chosen by the client. The user agent and referrer can say anything. Second, the address you see is only as trustworthy as the network path: attackers rent many addresses, and visitors behind a company or school network share one.

The status code is the quickest filter. 2xx means success, 3xx a redirect, 4xx a client mistake (404 not found, 403 forbidden, 401 unauthorized), and 5xx a server error. A real visitor produces mostly 200 and 304. A scanner produces a wall of 404.

Patterns worth a second look

No single line proves an attack. What you look for is a pattern: the same address, many lines, and a shape that normal browsing does not have.

Path guessing. Real users follow links, so almost every path they request exists. A client that asks for /admin, /backup.zip, /.git/config, /wp-login.php and /phpmyadmin/ within seconds, and gets 404 each time, is enumerating. If one of those guesses returns 200, you have found a problem: that file is exposed.

Suspicious characters in the request. Percent-encoding hides characters from casual reading, so decode first. Parts of a URL that should raise eyebrows include:

  • a single quote or %27, UNION SELECT, or OR 1=1 in a query string (a sign that someone is trying SQL injection);
  • <script> or %3Cscript%3E (cross-site scripting probes);
  • ../ or %2e%2e%2f (path traversal, trying to climb out of the web folder);
  • ;, | or backticks followed by command names (command injection probes).

A probe in the log is not a successful attack. If the application validates input and uses parameterized queries, the probe fails and you simply learn that someone tried. Look at the status and size of the answer: a 500 right after a quote character suggests the input reached code that choked on it, which deserves a developer's attention.

Brute force and credential stuffing. Many POST /login requests from one address, or one account hit from many addresses, with the answer size staying the same (same failure page) and then suddenly changing, which can mean one attempt worked.

Rate anomalies. A person cannot load 200 pages in a minute. Counting requests per address per minute is the simplest bot detector there is.

Tool signatures. Default user agents such as sqlmap, Nikto or curl are easy to spot, but they are also trivial to change. Treat them as a bonus clue, never as the test.

Logs record what the server saw, not what was intended. An empty referrer, an old browser version or a foreign address is not evidence of anything on its own. Combine at least two signals before you act.

Real commands you would run

On a Linux server these one-liners answer the usual questions. They are shown read-only because the site cannot read your logs.

Terminal
# Top client addresses
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# Count status codes
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

# Requests that returned 404 per client
awk '$9 == 404 {print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# Quote characters or UNION in the request
grep -Ei "%27|union.+select" /var/log/nginx/access.log

These are described in Command Line Essentials for Security Work. Tools such as fail2ban automate the response by reading the log and adding a firewall rule for addresses that fail too often (see Firewalls, IDS and Log Monitoring).

A stand-in: parse and summarize a log

The real tools above are text filters. The runnable version below does the same job in Python: it parses a sample log with a regular expression, decodes the URLs, and prints a summary. The log lines are invented, and the addresses come from the documentation ranges (203.0.113.0/24, 198.51.100.0/24) reserved for examples.

Python Parse a sample access log

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The regular expression names each part of the line ((?P<ip>...)), so every row becomes a dictionary. Lines that do not match are dropped here; in real work you should count them too, because a malformed line is itself interesting (it may be an attack on the log parser or on the server).

A stand-in: flag suspicious requests

Now add the detection rules. Each rule is a name plus a regular expression, applied to the decoded target. Decoding matters: %27 and ' must look the same to the rule.

Python Flag requests that match attack patterns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the status codes next to each hit. The script probe returned 200, which means the server accepted it. That does not prove a vulnerability (the page may encode the text safely), but it is the line you would investigate first. The 404 answers show guesses that missed. This is why detection rules should report the status alongside the pattern.

Rate anomalies

Pattern rules miss attacks that use normal-looking requests, such as password guessing. The tool for those is counting over time. Group requests by address and by minute, and compare the count to what a person could do.

Python Find clients that exceed a request rate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A real limit depends on the site. A page with 40 images may legitimately cause 40 requests in a second, so rate limits should apply to costly or sensitive paths (login, search, password reset) rather than to everything.

From detection to response

Finding a pattern is only half of the job. A sensible response ladder looks like this:

  1. Record and alert. Write the finding somewhere a person will look (a SIEM or a chat channel), as described in Firewalls, IDS and Log Monitoring.
  2. Slow down. Rate limits and temporary blocks stop the damage without deciding what the client is.
  3. Fix the cause. If a probe got a 500 or a 200 it should not have, find the code path and repair it (parameterized queries, output encoding, removing the exposed file).
  4. Block with care. Blocking an address can hit innocent users behind shared networks, and attackers change addresses easily.

Also protect the logs themselves. They contain personal data (addresses, sometimes tokens in URLs), so limit who can read them and how long you keep them. Never put passwords or session tokens in query strings, because the log keeps them. Ship logs to a separate machine: an intruder's first move is often to edit or delete the local file.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A client asks for /admin, /backup.zip and /.git/config within two seconds and gets 404 each time. What does this most likely show?

    Real visitors follow links to pages that exist. Quick requests for common sensitive paths that all miss are typical enumeration. The 404 answers show the guesses failed.
  2. 2. Why should detection rules decode the URL before matching?

    An attacker or tool may send %27 instead of a quote. If your rule only searches for the plain character, it misses the encoded one. Decode first, then match.
  3. 3. Which field in a combined log line can the client freely choose and fake?

    The server decides the status and the time. The user agent, referrer and request line come from the client, so they should never be trusted on their own.
  4. 4. A request containing a single quote returned status 500. What is the best reading?

    A probe is not a breach, but a server error right after special characters suggests the input was not handled safely. It is a lead for a code review.

Summary

  • An access log line records the client address, time, request, status, size, referrer and user agent. Only the time and status come from the server.
  • Look for patterns: many 404s, special characters, path traversal, repeated login attempts and high request rates.
  • Decode URLs before matching, and read the status code next to every hit.
  • A probe in the log is not a successful attack; combine at least two signals before reacting.
  • Respond in steps: record, slow down, fix the cause, and block carefully. Protect the logs themselves.
  • The commands (awk, grep, fail2ban) are real; the Python examples are small stand-ins that show the same logic on invented data.

Next: Wireless Security Concepts.