Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 36 ยท Projects and Practice

Project: a supervised key-value store

Build a key-value store as a real OTP application: a gen_server with TTL expiry, ETS-backed reads, a supervisor, tests and a runnable JavaScript model.

This project assembles the whole chapter: a gen_server that stores values with optional expiry, ETS for fast reads, a supervisor that restarts it, an application that starts it, and tests. The Erlang code is read-only here (the site cannot run Erlang); a JavaScript model lets you experiment with the logic, and the file layout matches what rebar3 new app kvstore gives you.

What we are building

  • kvstore:put(Key, Value) and kvstore:put(Key, Value, TtlMs).
  • kvstore:get(Key) returns {ok, Value} or error, and expired entries count as missing.
  • Reads go straight to an ETS table and never wait on the server, so they scale with CPU cores.
  • Writes go through the server process, so they are serialised and safe.
  • A periodic sweep deletes expired entries.
  • The server is supervised: if it crashes, the supervisor starts a new one with an empty store.
flowchart TD
    A["kvstore_app"] --> S["kvstore_sup (one_for_one)"]
    S --> W["kvstore_server (gen_server)"]
    W -. "owns" .-> T[("ETS table kvstore")]
    C["any process"] -- "get: direct read" --> T
    C -- "put: call" --> W

Only the server writes; anyone may read.

The ETS table

ETS (Erlang Term Storage) holds tuples in memory, shared by all processes of a node, with very fast access and no copying of the table itself. The table belongs to the process that creates it, and vanishes when it dies, which is why the server creates it in init/1.

ERLANG
ets:new(kvstore, [named_table, set, protected, {read_concurrency, true}]).

protected means everybody can read but only the owner writes. named_table lets code refer to it by name.

The server

ERLANG
-module(kvstore_server).
-behaviour(gen_server).

-export([start_link/0, put/3]).
-export([init/1, handle_call/3, handle_cast/2, handle_info/2]).

-define(TABLE, kvstore).
-define(SWEEP_MS, 1000).

start_link() ->
    gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).

put(Key, Value, TtlMs) ->
    gen_server:call(?MODULE, {put, Key, Value, TtlMs}).

init([]) ->
    ets:new(?TABLE, [named_table, set, protected, {read_concurrency, true}]),
    erlang:send_after(?SWEEP_MS, self(), sweep),
    {ok, #{}}.

handle_call({put, Key, Value, TtlMs}, _From, State) ->
    ExpiresAt = expiry(TtlMs),
    true = ets:insert(?TABLE, {Key, Value, ExpiresAt}),
    {reply, ok, State};
handle_call(_Other, _From, State) ->
    {reply, {error, unknown_call}, State}.

handle_cast(_Msg, State) ->
    {noreply, State}.

handle_info(sweep, State) ->
    Now = erlang:monotonic_time(millisecond),
    %% delete every entry whose expiry time is below Now
    ets:select_delete(?TABLE, [{{'_', '_', '$1'}, [{is_integer, '$1'}, {'<', '$1', Now}], [true]}]),
    erlang:send_after(?SWEEP_MS, self(), sweep),
    {noreply, State};
handle_info(_Info, State) ->
    {noreply, State}.

expiry(infinity) -> infinity;
expiry(TtlMs)    -> erlang:monotonic_time(millisecond) + TtlMs.

Three Erlang points worth studying:

  • erlang:send_after/3 sends a message to self() later, a clean timer that fits handle_info. Re-arm it after each sweep.
  • ets:select_delete/2 uses a match specification: {{'_', '_', '$1'}, Guards, [true]} matches tuples of three elements, binds the third to $1, filters by guards and returns true for each deletion. The guards {is_integer, '$1'} skip entries with infinity.
  • Monotonic time never jumps backwards when the system clock changes, which is the right clock for timeouts.

The public API, with direct reads

ERLANG
-module(kvstore).
-export([put/2, put/3, get/1]).

put(Key, Value) -> put(Key, Value, infinity).
put(Key, Value, TtlMs) -> kvstore_server:put(Key, Value, TtlMs).

get(Key) ->
    case ets:lookup(kvstore, Key) of
        [{Key, Value, infinity}] ->
            {ok, Value};
        [{Key, Value, ExpiresAt}] ->
            case erlang:monotonic_time(millisecond) < ExpiresAt of
                true  -> {ok, Value};
                false -> error
            end;
        [] ->
            error
    end.

Reading checks the expiry itself, so an entry that has expired but not been swept yet is still reported as missing. If the server has crashed and no table exists, ets:lookup raises badarg; during the brief restart, callers may catch it or retry.

Supervisor and application

ERLANG
-module(kvstore_sup).
-behaviour(supervisor).
-export([start_link/0, init/1]).

start_link() -> supervisor:start_link({local, ?MODULE}, ?MODULE, []).

init([]) ->
    {ok, {#{strategy => one_for_one, intensity => 5, period => 10},
          [#{id => kvstore_server, start => {kvstore_server, start_link, []}}]}}.
ERLANG
-module(kvstore_app).
-behaviour(application).
-export([start/2, stop/1]).

start(_Type, _Args) -> kvstore_sup:start_link().
stop(_State) -> ok.

with {mod, {kvstore_app, []}} in kvstore.app.src (applications and releases). Killing the server (exit(whereis(kvstore_server), kill)) makes the supervisor start a new one; the ETS table is created again, empty. If that is unacceptable, the next step would be an heir process or a persistent store such as DETS or Mnesia.

Tests

ERLANG
-module(kvstore_tests).
-include_lib("eunit/include/eunit.hrl").

kvstore_test_() ->
    {setup,
     fun() -> {ok, P} = kvstore_sup:start_link(), P end,
     fun(P) -> unlink(P), exit(P, shutdown) end,
     [?_assertEqual(error, kvstore:get(missing)),
      ?_assertEqual(ok, kvstore:put(a, 1)),
      ?_assertEqual({ok, 1}, kvstore:get(a)),
      fun() ->
          ok = kvstore:put(b, 2, 20),
          ?assertEqual({ok, 2}, kvstore:get(b)),
          timer:sleep(40),
          ?assertEqual(error, kvstore:get(b))
      end]}.

Run with rebar3 eunit (testing).

Run the model

This JavaScript stand-in is not Erlang. It models the table, TTL with a virtual clock (so the output is deterministic), the sweep and the supervisor restart that loses the data.

JS Stand-in: TTL store with a restart

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Ideas for extension

  1. Store a separate counter per key with ets:update_counter/3.
  2. Replace the single sweep with a per-entry timer (erlang:start_timer/3) and compare the cost.
  3. Add a kvstore:keys/0 using ets:select/2.
  4. Add an heir ({heir, Pid, Data}) so the table survives a server crash.
  5. Persist to disk with dets or file:write_file/2 and reload in init/1.
  6. Use gen_server:cast for writes and measure the difference in mailbox length.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do reads bypass the gen_server and go directly to ETS?

    A protected ETS table can be read by any process, so reads do not serialise through one process.
  2. 2. What happens to the ETS table when the owner process dies?

    A table lives and dies with its owner unless an heir is configured.
  3. 3. Why use erlang:monotonic_time/1 for expiry?

    Timeouts need a clock that only moves forward.
  4. 4. What does erlang:send_after(Ms, self(), sweep) do?

    The message arrives in the mailbox later and is handled by handle_info/2.

Summary

  • A supervised gen_server serialises writes; a protected ETS table lets everyone read concurrently.
  • send_after plus handle_info makes a periodic job; match specifications delete in bulk.
  • A supervisor restores the service after a crash, and the lost state shows why persistence or an heir may be needed.
  • Tests run with EUnit fixtures; the application and supervisor files follow the OTP template.

Next: Practice exercises