Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 40 ยท Tooling and Libraries

Testing and Debugging OCaml: Alcotest, Expect Tests and Tools

Test OCaml code with dune test, Alcotest unit tests, ppx_expect snapshot tests and QCheck properties, then debug with Printf, backtraces, the type checker, ocamldebug and warnings.

A strong type system rules out whole classes of mistakes, but not wrong logic. OCaml projects therefore combine the compiler's checks with three kinds of tests: unit tests (a few chosen inputs), expect tests (snapshots of printed output) and property tests (random inputs checked against a rule). Dune runs them all with dune test.

The commands and libraries below run on your machine; the site cannot run OCaml, so the ocaml and lisp blocks are read-only. The Python stand-ins show the same three testing styles.

Unit tests with Alcotest

Alcotest is the most popular test framework. Add it as a test dependency, then write a test executable.

LISP
; test/dune
(test
 (name test_hello)
 (libraries hello alcotest))
OCAML
(* test/test_hello.ml *)
let test_add () =
  Alcotest.(check int) "same ints" 5 (Hello.add 2 3)

let test_parse () =
  Alcotest.(check (option int)) "valid" (Some 42) (Hello.parse "42");
  Alcotest.(check (option int)) "invalid" None (Hello.parse "x")

let test_raises () =
  Alcotest.check_raises "div by zero" Division_by_zero (fun () -> ignore (Hello.div 1 0))

let () =
  Alcotest.run "hello"
    [ ("arith", [ Alcotest.test_case "add" `Quick test_add;
                  Alcotest.test_case "div" `Quick test_raises ]);
      ("parse", [ Alcotest.test_case "parse" `Quick test_parse ]) ]

Alcotest.(check int) builds a checker from a testable: a value describing how to compare and print a type. Built-ins exist for int, string, bool, float, list, option, pair; for your own types you define one with Alcotest.testable pp equal:

OCAML
let shape = Alcotest.testable Shape.pp Shape.equal
let () = Alcotest.(check shape) "area" (Circle 1.0) (Shape.make "circle" 1.0)

Run dune test. On failure, Alcotest prints the expected and actual values side by side.

Expect tests with ppx_expect

Expect tests record what a piece of code prints and compare it to a stored snapshot. You write the code, run it, and accept the output:

LISP
(library
 (name hello)
 (inline_tests)
 (preprocess (pps ppx_expect)))
OCAML
let%expect_test "greeting" =
  print_endline "hello";
  [%expect {| hello |}]

let%expect_test "sum" =
  Printf.printf "%d\n" (List.fold_left ( + ) 0 [ 1; 2; 3 ]);
  [%expect {| 6 |}]

When the output changes, dune test shows a diff, and dune promote accepts the new output into the source file. This is excellent for printers, parsers and anything with textual output, and keeps examples in your documentation honest.

Property-based tests with QCheck

Instead of choosing inputs, describe a property that must hold for all inputs and let the library generate hundreds of random ones, then shrink a failing case to a minimal example:

OCAML
let rev_twice =
  QCheck.Test.make ~name:"reverse twice is identity"
    QCheck.(list int)
    (fun xs -> List.rev (List.rev xs) = xs)

let () = QCheck_runner.run_tests_main [ rev_twice ]

Good properties: round trips (parse (print x) = x), idempotence (sort (sort xs) = sort xs), invariants (a tree stays balanced after insert), and comparison with a slow but obviously correct version.

flowchart LR
    A["unit tests<br/>few chosen inputs"] --> D["dune test"]
    B["expect tests<br/>output snapshots"] --> D
    C["property tests<br/>random inputs"] --> D

Figure: three complementary kinds of tests, one command.

Stand-in: all three styles in Python

Python Stand-in: unit, snapshot and property-style tests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Debugging

OCaml's type checker removes many bugs before you run anything, so the first debugging tool is reading the compiler error. Types are always inferred left to right, so the real mistake is often a few tokens before the position the compiler highlights.

Printf and the toplevel

The quickest approach is still printing. Printf.eprintf writes to stderr, which stays unbuffered; leave your debug prints on stderr so they do not mix with program output. In utop you can call any function directly and see its inferred type and value.

Backtraces

Uncaught exceptions end the program with a message such as Fatal error: exception Not_found. Ask for a backtrace:

Terminal
OCAMLRUNPARAM=b dune exec ./bin/main.exe

Backtraces need the code compiled with debug info (dune's dev profile does this) and show which line raised. Inside a program, Printexc.record_backtrace true and Printexc.print_backtrace stderr do the same.

Warnings as friends

Keep dune's default warnings enabled. Unused variable (26, 27), non-exhaustive match (8), unused match arm (11) and fragile match (4, off by default) point at real bugs.

ocamldebug and others

ocamldebug is a time-travelling debugger for bytecode programs: it can step backwards. For native code, gdb or lldb work; perf and landmarks help with profiling; OCaml 5 ships the runtime_events tracing system.

Tracing in the toplevel

In ocaml or utop, #trace f;; prints every call and return of f, which makes recursion easy to understand:

OCAML
# let rec fact n = if n <= 1 then 1 else n * fact (n - 1);;
# #trace fact;;
# fact 3;;
fact <-- 3
fact <-- 2
fact <-- 1
fact --> 1
fact --> 2
fact --> 6
- : int = 6

Common mistakes

  • Testing only the happy path. Include empty lists, zero, negative numbers and the failing branch of every result.
  • Snapshot tests whose output contains timestamps or addresses, which change on every run.
  • Properties that restate the implementation. A property should be simpler than the code.
  • Keeping Printf debug lines in released code.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which command runs the tests of a dune project?

    dune test builds and runs every test stanza and expect test.
  2. 2. What does an expect test compare?

    [%expect {| ... |}] holds the expected output, and dune promote updates it after a deliberate change.
  3. 3. Which of these is a good property for property-based testing of sort?

    Idempotence holds for every list, so random generation can check it on many inputs.
  4. 4. How do you see where an uncaught exception was raised?

    The b flag turns on backtraces.

Summary

  • dune test runs unit tests (Alcotest), expect tests (ppx_expect) and property tests (QCheck).
  • Alcotest checkers come from testables. Expect tests are promoted with dune promote.
  • Read compiler errors carefully. Use OCAMLRUNPARAM=b, #trace, Printf on stderr and warnings to debug.
  • Test edge cases and failure paths, not just the happy path.

Next: Libraries: Core, Lwt, Eio, ppx and the Ecosystem