Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 31 ยท Projects

Project: Security Log Analyzer

Build a security log analyzer in Python: parse SSH and web server logs, count failures, flag brute-force patterns and print a clear summary for defenders.

Every attack leaves footprints, and most of them land in log files. A defender who can read logs quickly can answer the questions that matter after something odd happens: who tried to sign in, how often, from where, and did anyone get in? Tools such as Fail2ban, a SIEM or a simple shell pipeline all start from the same step, which is turning raw text lines into structured records you can count.

In this project you build a small security log analyzer in Python. It parses an SSH authentication log and a web server access log, counts failures, flags brute-force patterns with a sliding time window, and prints a summary a human can act on. It is a purely defensive tool: it only reads text you give it. If you want the background first, read Reading Web Server Logs for Attacks and Command Line Essentials for Security Work.

Real versus stand-in. On a real Kali or Debian machine the SSH log lives in /var/log/auth.log (or in the journal, read with journalctl -u ssh), and web logs in /var/log/nginx/access.log or /var/log/apache2/access.log. The browser cannot read those files, so every example here parses sample text stored in the program. The parsing logic is the same code you would point at a real file. All addresses come from the documentation ranges (203.0.113.0/24, 198.51.100.0/24, 192.0.2.0/24), which are reserved and never belong to real hosts.

What the finished tool does

One tool, four steps

Parse Turn log lines into records.
Count Tally failures per address and user.
Detect Flag bursts inside a time window.
Report Print findings in priority order.

Each stage works on its own, so you can test it alone.

1 / 2

Stage 1: parse SSH log lines

A log line is just a string with a predictable layout. A regular expression with named groups turns it into fields. We then store each event in a small dataclass, because named fields are easier to read than positions in a tuple. The sample log below mimics what OpenSSH writes: a month, day and time, the host name, the process, and a message.

Two message shapes matter to us: Failed password for ... and Accepted password for .... Everything else (session opened, connection closed) is ignored but counted, so you can see how much of the log you did not understand. That "ignored" number is a useful health check for any parser.

Python Stage 1: parse the authentication log

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice for ... else: the else branch runs only when the loop finished without a break, which here means "no pattern matched this message". The log has no year, so the parser takes it as an argument. Real logs sometimes cross New Year, and a careful tool would handle that; for a one-day log the simple version is fine.

Stage 2: count and flag brute force

Counting failures per address is the first useful signal. The second is timing: five failures in a year is normal typing mistakes, five in ten seconds is a script. A sliding window finds that. For each address we sort its failure times and check, for every failure, how many fall inside the following WINDOW seconds.

We also want the most important finding: a success that comes right after a burst. That pattern is the difference between "someone knocked on the door" and "someone may be inside", so it gets the highest severity.

Python Stage 2: sliding-window detection

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two thresholds define the behavior: WINDOW and THRESHOLD. Change them and re-run to see how the findings move. A short window with a high threshold catches only fast scripts. A long window with a low threshold also catches "low and slow" guessing, but starts to flag people who forgot a password. Tuning those two numbers against your real traffic is most of the work of running a detector. If you are curious about the defensive tooling that automates this, see Firewalls, IDS and Log Monitoring.

The sliding window here is O(n^2) per address in the worst case, which is fine for a sample. For millions of lines you would keep a collections.deque of recent timestamps and drop old ones as new lines arrive, so each line is handled once.

Stage 3: parse the web access log

Web servers log one line per request in the combined log format: client address, time, request line in quotes, status code, bytes, referrer and user agent. Attacks show up differently here than in SSH. Instead of failed passwords you look for many 404 Not Found answers (a client guessing paths), requests to sensitive paths such as /.env or /wp-login.php, and unusual user agents.

Python Stage 3: web log parsing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The 404 count is a heuristic. A broken link on your own site also produces 404 answers, and a crawler can legitimately miss pages. The word list in SENSITIVE raises or lowers the severity but never proves intent. Good reports say "this behavior matched the rule X" rather than "this person is an attacker".

Stage 4: one combined report

The last stage joins the pieces into the summary a person reads first: totals, then findings sorted by severity. Keeping the detection (functions that return data) separate from the printing (a function that formats it) is what lets you reuse the same findings later, for example by exporting JSON or sending an alert. The finding list below uses a plain dict per finding, the same shape you would use in Writing a Professional Security Report.

Python Stage 4: the combined summary

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice what the report does not contain: any decision to block anyone. Detection and response are separate jobs. A person (or an automation you trust and have tested) decides whether to block an address, force a password reset, or just keep watching. Blocking automatically on a heuristic can lock out your own users, and attackers can abuse that by spoofing sources.

Ideas to extend the project

  • Read from a real file with open("auth.log") when you run the code on your own machine, instead of from a string.
  • Add a rule for one user failing from many addresses (a distributed guess) alongside one address trying many users.
  • Add a "first seen" check that compares each successful login address with a known list of your team's networks.
  • Export the findings as JSON so another tool can ingest them.
  • Write unit tests with a three-line log for each rule. A detector you cannot test is a detector you cannot trust.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does the parser count the lines it ignored?

    A large ignored count means the format changed or the regex is wrong, so the findings may be incomplete. It is a cheap health check.
  2. 2. Which finding deserves the highest severity in this analyzer?

    A success after a burst suggests the guessing may have worked, so it is the case where the defender must act first.
  3. 3. Why is the 404 count only a heuristic?

    Rules like this raise suspicion but do not prove intent. A report should name the rule that matched, not accuse a person.
  4. 4. What is the benefit of separating detection functions from the printing function?

    Functions that return data are easy to test and to reuse. Formatting is a separate concern that can change without touching the rules.

Summary

  • A log analyzer has four steps: parse lines into records, count, detect patterns, and report.
  • Named regex groups and a dataclass give you readable, structured events.
  • A sliding time window separates a typo from a script; WINDOW and THRESHOLD need tuning against real traffic.
  • A success right after a burst of failures is the highest-priority pattern in an authentication log.
  • Web log signals include many 404 answers and requests to sensitive-looking paths, but they are heuristics, not proof.
  • Keep detection, formatting and response separate; never block automatically on a rule you have not tested.

Next project: Project: Network Exposure Auditor.