Skip to content
Tutorials Center
My learning

Topics

Lesson 67 of 133 ยท Data and Storage

CSV, Logs and Text Processing

Parse CSV by hand with quotes and escapes, read big inputs with readline, extract data with regular expressions, aggregate logs and print aligned reports using Intl in Node.js.

A huge share of real-world programming is not databases or web servers. It is turning text into answers: a CSV export from a spreadsheet, a server log with a million lines, a report that someone wants as neat columns. Node.js is a very good tool for this work. Strings, regular expressions and streams are all built in, and the code reads almost like a description of the task.

In this lesson you will write a CSV parser that copes with quoted fields, read input line by line with readline, pull values out of log lines with regular expressions, aggregate the result, and print an aligned report. Each piece is small, and together they cover most "scripting" jobs you will meet.

Why not just split on commas?

The first idea is line.split(','). It works until it does not. CSV (comma-separated values) is a loose format with a few rules that break that approach:

  • A field may be wrapped in double quotes, and then it may contain commas: "Smith, Ada".
  • A double quote inside a quoted field is written twice: "She said ""hi""".
  • A quoted field may even contain a newline, so one record can span several lines.
  • Different files use different separators (; is common in Europe, tabs make a TSV file).
Node.js Why split(',') fails

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The naive split cuts the name in two and leaves stray quote characters. We need a parser that reads one character at a time and remembers whether it is currently inside quotes.

A small CSV parser

The trick is a tiny state machine: a loop over the characters with a flag inQuotes. Outside quotes a comma ends a field and a newline ends a record. Inside quotes everything is plain text, except a doubled "", which means one literal quote character.

Node.js parseCsv handles quotes, escapes and newlines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

This parser is about 35 lines and handles the cases above. Real files add more quirks: a leading BOM character (๏ปฟ), blank lines, and ragged rows with too few or too many columns. For production, battle-tested libraries such as csv-parse exist (read-only mention, since our editor has no npm packages). The principle stays the same, and it is worth understanding what such a library does.

From rows to objects

Arrays of strings are awkward. Most programs want the first row as a header and each later row as an object. Remember that every value is still a string; convert numbers and dates explicitly.

Node.js Header row to objects with typed values

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Careful with Number(''), which is 0 and not an error, and with floating point money. For serious money calculations keep integer cents or use a decimal library; 0.1 + 0.2 is not 0.3 in any JavaScript runtime.

Writing CSV safely

Writing is simpler, but one rule matters: quote any field that contains the delimiter, a double quote or a line break, and double the quotes inside. Otherwise your own file will not parse. There is a second, nastier reason to be careful: a field that starts with =, +, - or @ can be run as a formula when someone opens the file in a spreadsheet ("CSV injection"). If the data comes from users, prefix such fields with a single quote or a space.

Node.js toCsv with correct quoting

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Reading big inputs line by line

For small data you can read the whole file and split it. For large inputs you read a stream of lines, so memory stays flat. The node:readline module turns any readable stream (a file, a socket, or process.stdin) into an async iterable of lines. File streams are covered in File Streams, and interactive use in Interactive CLIs with readline.

The example reads from standard input. In our editor, the Input box feeds that stream. Piped input works the same way on your machine: cat access.log | node report.mjs.

Node.js Counting lines and words from standard input

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Two habits matter here. Set crlfDelay: Infinity so a Windows \r\n pair is always one line break, and never await slow work for each line unless you must; for heavy per-line work consider batching. Sorting with a tie-breaker (|| a[0].localeCompare(b[0])) also makes the output deterministic, which helps when you test it.

Extracting data with regular expressions

Log lines usually follow a pattern, and a regular expression can pull the pieces out. Named capture groups ((?<name>...)) make the result readable: the match then has a .groups object with the names you chose.

Consider lines like 2026-03-05T10:15:02Z GET /api/users 200 12ms. One pattern captures everything:

Node.js Parsing a log line with named groups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Some regex advice from experience:

  • Anchor with ^ and $ when you parse whole lines, so a half-match does not slip through.
  • Prefer specific classes (\d+, [^ ]+) over .*. Greedy wildcards cause surprises and slow matching.
  • Beware of patterns with nested repetition such as (a+)+$. On unlucky input they take exponential time, an attack called ReDoS (see Web Application Security).
  • Reuse a compiled regex outside the loop. Without the g or y flag a regex has no state, so sharing it is safe. With g, exec and test remember lastIndex, which causes famous bugs when the same object is reused.
  • Always handle the "no match" result (null). Real logs contain garbage.

Aggregating a log

Now combine the pieces. We parse each line, skip the bad ones while counting them, and group the numbers: requests per status class, slowest paths, average time per path. Map and Object.groupBy are the right tools for "group by".

Node.js Summarizing a small access log

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Notice the key normalization: /api/users/7 and /api/users/99 become one group, /api/users/:id. Without it, every id would be its own row and the report would be useless. Choosing what to group by is usually the hardest part of log analysis.

Formatting a report

Numbers in a report should line up and be readable. padStart and padEnd pad strings to a fixed width (use padStart for numbers so the digits align on the right). For thousands separators and decimals use Intl.NumberFormat or toLocaleString, and give the locale explicitly ('en-US'), otherwise the output changes with the user's computer settings.

Node.js An aligned table with Intl.NumberFormat

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

console.table is handy while you work but is not meant for final reports. Pad widths count UTF-16 units, so wide characters such as CJK text or emoji can misalign columns; for those use Intl.Segmenter and a width library. For machine-readable output, print JSON or CSV instead, as in JSON and Data Handling.

Putting it together: a streaming CSV summary

You can combine everything into one pipeline that never holds the whole file. We cannot ship a huge file in the editor, so the example uses stdin with a few lines. For records that span multiple lines, join continuation lines before parsing; here the data has none.

Node.js Sum a CSV column from standard input

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Memory use is proportional to the number of distinct cities, not the number of rows. That is the property to look for when you design a text-processing job. Later lessons on streams and processing large data go much deeper.

Common pitfalls

  • Encoding. Read files as UTF-8 and strip a leading BOM with text.replace(/^๏ปฟ/, ''); Excel often writes one.
  • Line endings. Windows files use \r\n. Use split(/\r?\n/) or readline.
  • Locale. Some regions write 1.234,56. Never guess: parse with a known format.
  • Trusting input. Validate row length and number conversions (Number.isNaN), and count rejected rows.
  • Loading everything. readFile plus split on a multi-gigabyte log can crash the process. Stream it.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does line.split(',') fail for general CSV?

    A field such as "Smith, Ada" contains the delimiter. A parser has to track whether it is inside quotes.
  2. 2. How is a literal double quote written inside a quoted CSV field?

    The CSV convention doubles the quote: "She said ""hi""" represents She said "hi".
  3. 3. What do you get from the number column of a parsed CSV file?

    CSV has no types. Every cell is text until you call Number() or parse it in another way.
  4. 4. Which setting makes readline treat \r\n as a single line break?

    With crlfDelay set to Infinity, a \r followed by \n is always one line break, whatever the delay between the two characters.
  5. 5. Why is it good to use named capture groups when parsing log lines?

    match.groups.status is clearer than match[4] and survives changes to the pattern. They do not change speed or safety.

Summary

  • CSV is not simply "split on commas": handle quoted fields, doubled quotes, embedded newlines and different delimiters with a small state machine.
  • Every parsed value is a string. Convert types explicitly and count rows you reject.
  • When you write CSV, quote fields that contain delimiters, quotes or newlines, and defuse values that begin with =, +, - or @.
  • Use readline with crlfDelay: Infinity to process large input line by line in constant memory.
  • Regular expressions with named groups, anchors and a null check turn log lines into objects. Avoid nested repetition.
  • Map and Object.groupBy aggregate data; normalize keys (like /:id) before grouping.
  • padStart, padEnd and Intl.NumberFormat with an explicit locale produce stable, aligned reports.

Next: Caching in Memory