Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 40 ยท Tooling and Libraries

Libraries: Stdlib, Base and Core, Lwt, Eio, ppx and More

Tour the OCaml ecosystem. Compare Stdlib with Base and Core, learn concurrency with Domains, Lwt and Eio, use Yojson, Cmdliner, ppx deriving, and decide which library fits each job.

OCaml's standard library is deliberately small. Real programs add libraries from opam for JSON, command lines, concurrency, parsing, databases and web servers. This lesson is a map of the ecosystem: what exists, what each library is for, and how it plugs into dune, so you can pick tools with confidence.

Libraries are installed with opam on your machine; the site cannot run OCaml, so all ocaml blocks are read-only. The Python stand-in shows the idea behind the ppx deriving mechanism.

Stdlib, Base, Core

Three layers cover most of what you need:

Library What it is Use it when
Stdlib ships with the compiler; modules List, Array, String, Hashtbl, Map, Option, Result, Seq, In_channel small tools, learning, minimal dependencies
Base Jane Street's alternative standard library with safer defaults (for example List.hd returns an option) and consistent naming you prefer stricter, more uniform APIs
Core Base plus the "batteries": Command for command lines, Time, Sexp, Hashtbl with extra operations larger applications in the Jane Street style

Using Base:

OCAML
open Base

let () =
  let xs = [ 3; 1; 2 ] in
  let sorted = List.sort xs ~compare:Int.compare in
  Stdio.printf "%s\n" (String.concat ~sep:"," (List.map sorted ~f:Int.to_string))
(* prints: 1,2,3 *)

Notice labelled arguments (~compare, ~f, ~sep) everywhere. They make calls self-documenting at the price of a style that differs from the standard library. You can mix: use Stdlib for most code and call into Base or Core where they have something you want. Avoid opening both in one file, which shadows names confusingly.

Concurrency and parallelism

OCaml 5 introduced two separate tools, plus a long-standing library:

Tool Purpose
Domains (Domain.spawn) true parallelism on several cores
Effects the runtime mechanism behind lightweight schedulers
Lwt cooperative concurrency with promises ('a Lwt.t); mature, widely used
Eio newer direct-style concurrency built on effects: no monads, it reads like ordinary code
Mutex, Condition, Atomic synchronization primitives

A first look at parallelism with domains:

OCAML
let sum_range lo hi =
  let s = ref 0 in
  for i = lo to hi do s := !s + i done;
  !s

let () =
  let d = Domain.spawn (fun () -> sum_range 1 500_000) in
  let b = sum_range 500_001 1_000_000 in
  let a = Domain.join d in
  Printf.printf "%d\n" (a + b)
(* prints: 500000500000 *)

Each domain has its own minor heap, and values are shared through the major heap. Do not share a ref between domains without protection: use Atomic.t for counters and Mutex.t for larger critical sections.

Lwt in a nutshell:

OCAML
let ( let* ) = Lwt.bind

let main () =
  let* () = Lwt_io.printl "start" in
  let* () = Lwt_unix.sleep 0.1 in
  Lwt_io.printl "done"

let () = Lwt_main.run (main ())

With Eio, the same program is straight-line code: Eio.Time.sleep clock 0.1; traceln "done", and Eio.Fiber.both runs two functions concurrently. For a new project on OCaml 5, Eio is worth a look. Lwt has far more existing libraries, such as Cohttp and Dream.

Data formats and text

Task Library
JSON yojson (read, write, a simple variant type), ezjsonm
Command-line cmdliner (declarative options, man pages), or stdlib Arg
Regular expressions re (pure OCaml), str (bundled with the compiler)
Parsing menhir (LR parser generator), angstrom (combinators), sedlex (Unicode lexers)
S-expressions sexplib
CSV csv
Pretty printing fmt (builds on Format)
Logging logs

Using Yojson:

OCAML
let () =
  let json = Yojson.Safe.from_string {|{"name":"Ada","langs":["OCaml","C"]}|} in
  let open Yojson.Safe.Util in
  let name = json |> member "name" |> to_string in
  let langs = json |> member "langs" |> to_list |> List.map to_string in
  Printf.printf "%s knows %d languages\n" name (List.length langs)
(* prints: Ada knows 2 languages *)

Web, databases and numbers

  • Dream and Opium are web frameworks; Cohttp is an HTTP client and server toolkit.
  • Caqti gives typed database access (PostgreSQL, SQLite). Irmin is a versioned store.
  • Owl provides scientific computing (matrices, statistics); Zarith gives arbitrary-precision integers.
  • js_of_ocaml and Melange compile OCaml to JavaScript; ReScript is a related syntax.
  • ocamlfind and findlib names are what dune's (libraries ...) uses.

ppx: code generation at compile time

A ppx is a rewriter that transforms the syntax tree during compilation. The most used one derives functions from type definitions:

LISP
(library
 (name shapes)
 (preprocess (pps ppx_deriving.show ppx_deriving.eq ppx_compare)))
OCAML
type shape = Circle of float | Rect of float * float [@@deriving show, eq]

let () =
  print_endline (show_shape (Rect (2., 3.)));
  Printf.printf "%b\n" (equal_shape (Circle 1.) (Circle 1.))
(* prints: Rect (2., 3.)  then  true *)

[@@deriving show, eq] generates show_shape and equal_shape for you. Other ppx libraries derive JSON converters (ppx_yojson_conv), comparison, hashing, S-expressions (ppx_sexp_conv), inline tests and benchmarks. They remove the boilerplate that would otherwise make serializers tedious.

Stand-in: deriving by hand in Python

Python's dataclass decorator derives __repr__ and __eq__ from the fields, the same convenience:

Python Stand-in: derived show and equality

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Choosing

flowchart TD
    Q["What do you need?"] --> A["A small tool"] --> A1["Stdlib + dune"]
    Q --> B["Many cores"] --> B1["Domains + Atomic/Mutex"]
    Q --> C["Many sockets or I/O"] --> C1["Lwt or Eio"]
    Q --> D["Structured data"] --> D1["yojson + ppx_deriving"]
    Q --> E["Command line"] --> E1["cmdliner"]

Figure: a quick guide to the ecosystem.

Before adding a dependency, check the opam page for recent releases, the number of reverse dependencies and its licence. A short list of well-maintained libraries is easier to live with than a long list of clever ones.

Common mistakes

  • open Core and open Stdlib together, causing shadowed names.
  • Sharing mutable state between domains without Atomic or Mutex.
  • Mixing Lwt and Eio in one program without an adapter.
  • Forgetting to add the ppx to (preprocess (pps ...)), which gives "unbound value show_shape".

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the main difference between Domains and Lwt?

    Domains run on separate cores, whereas Lwt interleaves promises within a single thread.
  2. 2. What does [@@deriving show] do?

    ppx rewriters generate code from type definitions during compilation.
  3. 3. Which library is the usual choice for declarative command-line parsing?

    Cmdliner builds argument parsers and man pages from declarations.
  4. 4. Where do you list a library such as yojson for a program built with dune?

    dune links the libraries named in the libraries field.

Summary

  • Stdlib is small and always available. Base and Core offer stricter, richer alternatives.
  • OCaml 5 offers Domains for parallelism. Lwt and Eio provide concurrent I/O.
  • Yojson, Cmdliner, Menhir, Fmt and Logs cover everyday needs, and ppx libraries derive boilerplate.
  • Add dependencies deliberately and declare them in dune and dune-project.

Next: Project: An Expression Evaluator