Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 38 ยท Projects and Practice

Project: A Log Analyzer

Build a log analyzer step by step with regular expressions. Parse web server access lines with named groups, count status codes, rank paths, find slow requests and flag suspicious clients.

Logs are the classic regex playground: every line follows a format, there are thousands of them, and the questions you ask ("which pages fail?", "who is hammering the login?") are exactly what pattern matching answers. In this project you will build a small analyzer in Node.js, one step at a time, using only the standard library.

By the end you will have a parser that is tolerant of bad lines, a stats collector, and a report. Each step is a runnable program that reuses the code from the earlier ones.

flowchart LR
  L["Raw log text"] --> P["Parse each line<br/>named-group regex"]
  P -->|"no match"| B["Collect bad lines"]
  P -->|"match"| N["Normalize<br/>numbers, dates, paths"]
  N --> S["Aggregate stats"]
  S --> R["Report"]

The pipeline: parse, normalize, aggregate, report.

Step 1: Parse one line

The format is the common "combined" access log style:

TEXT
203.0.113.9 - - [09/Mar/2025:10:15:32 +0000] "GET /products?id=7 HTTP/1.1" 200 5123 0.042

That is: client IP, two legacy fields, a timestamp in brackets, the quoted request line, the status code, the response size and the time in seconds. Write the pattern with x-style structure by building it from named pieces so it stays readable.

Node.js Step 1: a readable line parser

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Building a regex from named string pieces is a good habit for long patterns: each piece can be tested alone, and String.raw avoids doubling backslashes.

Step 2: Parse a whole log and keep bad lines

Never let one malformed line crash the run. Count it, remember a few examples, and carry on.

Node.js Step 2: parse many lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Notice the normalization rules in the loop: the query string is removed from the path with /\?.*$/ so /products?id=7 and /products?id=8 count as the same page, and a dash in the size field becomes zero.

Step 3: Aggregate

Statistics are just grouping and counting. A small helper countBy does most of the work.

Node.js Step 3: counts, rankings and slow requests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Step 4: Group by pattern with a path classifier

Raw paths are too specific for a useful report: /users/17 and /users/92 are the same route. Replace variable parts with placeholders using a small list of rules. Regex replace with ordered rules is a reusable technique.

Node.js Step 4: normalize paths into routes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Step 5: Put it together

Finally combine the parts into one function that takes raw text and returns a report string. Run it on the sample text.

Node.js Step 5: the complete analyzer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Ideas to extend it

  • Parse the timestamp into a Date and report requests per minute.
  • Add a second format (for example JSON lines or syslog) and pick the parser by testing the line start.
  • Detect scanners: paths matching /\.(?:php|env|git)\b|wp-login/ from the same IP.
  • Read a real file with node:fs and node:readline, so huge logs are processed line by line without loading them into memory.
  • Add a time limit and input length cap per line (see hardening): logs are untrusted input.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does the analyzer collect unparsable lines instead of throwing an error?

    Real logs contain truncated and corrupted lines. Counting and sampling them keeps the analysis going and shows data quality problems.
  2. 2. What does path.replace(/?.*$/, "") do?

    The pattern matches a question mark and everything after it to the end, which removes the query string.
  3. 3. Why normalize /users/17 and /users/92 into /users/:id?

    Statistics per concrete URL are noisy. Grouping by route shows which handlers are busy or slow.
  4. 4. What is the advantage of building a long regex from named string pieces with String.raw?

    Splitting a long pattern into labelled parts makes it maintainable, and String.raw keeps backslashes as written.

Summary

  • A named-group regex turns each log line into an object; a null result marks a bad line.
  • Normalize while parsing: strip query strings, convert numbers, replace ids with placeholders.
  • Aggregation is counting, grouping and sorting; the regex work is mostly at the edges.
  • Build long patterns from named parts, test them alone and always handle unparsable input.

Next: Project: Markdown to HTML Converter