Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 38 ยท Projects and Practice

Project: Build a Mini Test Framework with Macros

Build a small unit-test framework in Common Lisp step by step, using defmacro, gensym, special variables and conditions, then run a stand-in version in JavaScript.

Test frameworks are the classic first macro project: they need new syntax (deftest, check), control over evaluation (to print the unevaluated expression that failed) and a registry of tests. In this project you build a mini framework in about 60 lines of Common Lisp, in four steps.

No Common Lisp implementation runs on this site, so the Lisp code is read-only and I show only outputs I am certain about. The JavaScript stand-in at each step follows the same design so you can run and modify it here.

Goal

LISP
(deftest test-arithmetic
  (check (= 4 (+ 2 2))
         (= 7 (* 2 3))
         (equal '(1 2) (list 1 2))))

(run-tests)

Expected report: one line per check form with pass or FAIL, the expression text, and a final summary.

Step 1: the check macro

A function would receive the value NIL, losing the expression. A macro keeps the form and can print it:

LISP
(defvar *results* nil "List of (passed-p . expression) for the current run.")

(defmacro check-one (form)
  `(let ((ok (ignore-errors ,form)))
     (push (cons (and ok t) ',form) *results*)
     ok))

',form inserts the form quoted, so the report can print (= 4 (+ 2 2)) as text. ignore-errors turns an error in the test expression into a failed check. There is a subtle variable-capture risk with ok: if the user's form mentions a variable called ok, it would see the macro's binding. The capture lesson tells us to use a gensym:

LISP
(defmacro check-one (form)
  (let ((ok (gensym "OK")))
    `(let ((,ok (ignore-errors ,form)))
       (push (cons (and ,ok t) ',form) *results*)
       ,ok)))

Now check accepts several forms by expanding into one check-one per form:

LISP
(defmacro check (&body forms)
  `(progn ,@(mapcar (lambda (f) `(check-one ,f)) forms)))

Check the expansion with (macroexpand-1 '(check (= 1 1) (= 2 2))), as in the debugging lesson: you should see a progn containing two check-one forms.

JS Stand-in step 1: check records the expression text

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Step 2: the registry and deftest

Tests are named functions registered in a list:

LISP
(defvar *tests* nil "Names of registered tests, oldest first.")

(defmacro deftest (name &body body)
  `(progn
     (defun ,name () ,@body)
     (pushnew ',name *tests*)
     ',name))

pushnew avoids duplicates when you re-evaluate a definition in the REPL, an important interactive habit. Because deftest expands to defun, each test is an ordinary function you can call alone: (test-arithmetic).

Add a name to each result so the report can group them. Use a special variable bound by the runner:

LISP
(defvar *current-test* nil)

(defmacro check-one (form)
  (let ((ok (gensym "OK")))
    `(let ((,ok (ignore-errors ,form)))
       (push (list *current-test* (and ,ok t) ',form) *results*)
       ,ok)))

Step 3: the runner

LISP
(defun run-tests (&optional (tests (reverse *tests*)))
  (setf *results* nil)
  (dolist (name tests)
    (let ((*current-test* name))
      (handler-case (funcall name)
        (error (e)
          (push (list name nil (format nil "ERROR: ~a" e)) *results*)))))
  (report (reverse *results*)))

(defun report (results)
  (let ((pass 0) (fail 0))
    (dolist (r results)
      (destructuring-bind (name ok form) r
        (if ok (incf pass) (incf fail))
        (format t "~:[FAIL~;pass~]  ~a: ~s~%" ok name form)))
    (format t "~d passed, ~d failed~%" pass fail)
    (zerop fail)))

*current-test* is a dynamically bound special variable, so check-one can read it without the user passing the name. The format directive ~:[FAIL~;pass~] prints the first text if its argument is nil, otherwise the second. handler-case keeps one crashing test from stopping the run, and report returns true when everything passed, which a CI script can turn into an exit code.

JS Stand-in step 3: registry, dynamic test name, report

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Step 4: nicer features

Extend the framework with:

  1. A signals check that expects a condition class:
LISP
(defmacro check-signals (condition-type form)
  (let ((hit (gensym "HIT")))
    `(let ((,hit nil))
       (handler-case ,form
         (,condition-type () (setf ,hit t)))
       (push (list *current-test* ,hit '(signals ,condition-type ,form)) *results*)
       ,hit)))
  1. Test groups. Make deftest accept a list (name suite), or keep a hash table from suite to names.
  2. Fixtures. A with-fixture macro using unwind-protect (see cleanup) to create and destroy resources around the body.
  3. Failure details. Evaluate (= a b)-style forms by hand to show both values: use destructuring-bind (op a b) form at macroexpansion time and bind gensyms for each argument so they print.
  4. Exit codes for CI. (sb-ext:exit :code (if (run-tests) 0 1)).

What the project teaches

  • Macros that need the unevaluated form (to print it), the textbook case for defmacro.
  • gensym for hygiene and pushnew for REPL-friendly registration.
  • Dynamic variables (*current-test*) as implicit context.
  • Conditions to keep one bad test from killing the run.
  • A real library has the same skeleton: FiveAM, Parachute and Rove all add registries, suites and reporting around this core.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why must check be a macro and not a function?

    The macro receives the unevaluated form, so it can both evaluate it and quote it for the report.
  2. 2. Why does deftest use pushnew?

    Interactive redefinition is normal in Lisp; pushnew keeps the registry free of duplicates.
  3. 3. What is *current-test* and why bind it with let in run-tests?

    Special variable bindings last for the dynamic extent of the form, so check-one can read the current test name.
  4. 4. What prevents one crashing test from stopping the whole run?

    The error is converted into a recorded failure and the loop continues.

Summary

  • check is a macro because it needs the form's text as well as its value.
  • deftest expands into a defun plus a registry entry using pushnew.
  • A dynamically bound *current-test* carries context without extra arguments.
  • handler-case isolates failures; report returns a boolean for CI.
  • The JavaScript versions are stand-ins that use thunks and labels where Lisp macros capture the code.

Next: Project: A Bank Account System with CLOS and Conditions