Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 36 ยท Projects and Practice

Project: A Word Frequency Counter in Nim

Build a word frequency counter in Nim using strings, tables and sorting, then follow the same algorithm in a runnable C version with a hash table.

In this project you will count how often each word appears in a text and print the most frequent ones. It uses ideas from the whole course: strings, a hash table, sorting, and a clean split between a reusable function and a main part.

The Nim compiler is not available on this site, so the Nim program is read-only: type it in with your own Nim install and run nim r wordcount.nim. A runnable C version of the same algorithm follows, so you can see the whole thing work here.

The plan

flowchart LR
  T["Text"] --> S["Split into words"]
  S --> N["Lowercase and trim"]
  N --> C["Count in a table"]
  C --> R["Sort by count"]
  R --> P["Print top N"]

Five small steps: each one is its own proc in the Nim version.

The Nim program

NIM
import std/[tables, strutils, algorithm, sequtils]

proc words(text: string): seq[string] =
  ## Split on anything that is not a letter or an apostrophe.
  var current = ""
  for ch in text:
    if ch.isAlphaAscii or ch == '\'':
      current.add ch.toLowerAscii
    elif current.len > 0:
      result.add current
      current.setLen 0
  if current.len > 0:
    result.add current

proc countWords(ws: seq[string]): CountTable[string] =
  for w in ws:
    result.inc w

proc top(counts: CountTable[string], n: int): seq[(string, int)] =
  result = counts.pairs.toSeq
  result.sort(proc (a, b: (string, int)): int =
    if a[1] != b[1]: cmp(b[1], a[1])     # larger counts first
    else: cmp(a[0], b[0]))               # ties: alphabetical
  if result.len > n:
    result.setLen n

when isMainModule:
  let text = "the cat and the hat and the bat"
  for (w, c) in top(countWords(words(text)), 3):
    echo w, ": ", c

Points worth noticing:

  • CountTable[string] is a generic table that maps a key to a count; inc adds one.
  • The sort comparator returns negative, zero or positive, like cmp; sorting ties alphabetically makes output deterministic.
  • when isMainModule: runs the demo only when the file is the program, not when another file imports it.
  • current.setLen 0 reuses the string's buffer instead of allocating a new one.

For this text the expected output is the: 3, and: 2 and then bat: 1 (ties are alphabetical: bat, cat, hat).

The same algorithm in C

The stand-in uses a small open-addressing hash table, which is what CountTable does conceptually. Run it to see the result.

C C stand-in: word counter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

Extending the project

  1. Read a file with readFile or lines from stdin with stdin.lines.
  2. Add a --top option with the parseopt module or the cligen package (see Nimble and Packages).
  3. Ignore common words with a HashSet[string].
  4. Write tests with std/unittest: empty text, punctuation, mixed case.
  5. Handle UTF-8 letters with std/unicode (isAlpha(rune)) instead of ASCII only.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which type counts occurrences of keys in Nim's standard library?

    CountTable maps each key to an integer count with inc.
  2. 2. Why sort ties alphabetically?

    Hash table order varies, so a tie-breaker gives stable output.
  3. 3. What does when isMainModule: do?

    It keeps demo code out of imported modules.
  4. 4. Why lowercase each letter?

    Normalizing case treats variants as the same word.

Summary

  • Break a task into small procs: split, normalize, count, sort, print.
  • CountTable is the idiomatic frequency counter.
  • A deterministic tie-breaker keeps output testable.
  • when isMainModule: separates a library from its demo.

Next: Project: Tokenizer and Calculator