Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 48 ยท Concurrency and OTP

ETS, Registry and Application State

Learn when to keep data in ETS tables, how to find processes with Registry, what persistent_term is for, and when a plain process is still the better choice.

A GenServer serializes everything: one message at a time. That is perfect for protecting state, but it becomes a bottleneck when thousands of processes want to read the same data, such as a cache, a configuration table or a counter. The BEAM ships with tools for this: ETS (in-memory tables), Registry (a process directory) and persistent_term (global values that are written rarely and read constantly).

This lesson explains each one, shows the access modes and the main functions, and gives a rule of thumb for choosing between a process and a table. You should know GenServer and supervisors first.

Elixir and Erlang are not installed on this site, so every Elixir snippet below is read-only. The runnable example at the end is a JavaScript stand-in that mimics a table with owners and access rules.

ETS: Erlang Term Storage

An ETS table is a mutable, in-memory table of tuples that lives outside any process heap. Many processes can read it at the same time without sending messages. You create one with :ets.new/2 (it is an Erlang module, so the name starts with a colon):

ELIXIR
table = :ets.new(:users, [:set, :public, :named_table])

:ets.insert(table, {1, "Ada", :admin})
:ets.insert(table, {2, "Linus", :member})

:ets.lookup(:users, 1)
# [{1, "Ada", :admin}]

:ets.lookup(:users, 99)
# []

Points to notice:

  • Rows are tuples. By default the first element is the key.
  • lookup always returns a list, because some table types allow several rows per key. A missing key is [], not nil.
  • With :named_table you refer to the table by its atom name; otherwise you use the reference that new returns.
  • Reading a row copies it into the calling process. ETS is fast, but it is not zero-copy for large terms.

Table types

Type Behavior
:set (default) One row per key. Inserting the same key replaces the row.
:ordered_set One row per key, kept sorted by key. Good for range scans.
:bag Several rows per key, but never two identical rows.
:duplicate_bag Several rows per key, duplicates allowed.

Access modes and the owner

Every table has an owner, the process that created it. The access mode decides who else may touch it:

  • :protected (default): anyone can read, only the owner can write.
  • :public: anyone can read and write.
  • :private: only the owner can read or write.

When the owner process dies, the table is deleted. This is the most common surprise. If a cache table must outlive restarts of the code that uses it, create it in a small, stable process that does nothing else (often one started early in your supervision tree), and let other processes use it.

Pick an access mode

public Everyone reads and writes.
protected Everyone reads. Only the owner writes.
private Only the owner touches it.

The default is protected.

1 / 2

Updating and querying

Counters are the classic ETS use case, and update_counter/3 is atomic, so no lost updates:

ELIXIR
:ets.new(:hits, [:set, :public, :named_table, write_concurrency: true])

:ets.insert(:hits, {"/home", 0})
:ets.update_counter(:hits, "/home", 1)
# 1
:ets.update_counter(:hits, "/home", 1)
# 2

:ets.insert_new(:hits, {"/home", 100})   # false: key already exists
:ets.delete(:hits, "/home")

For searching beyond the key you can use :ets.match/2 or :ets.select/2 with match specifications, and :ets.tab2list/1 to dump the table. A full-table scan is O(n), so design the key around your most common lookup.

Two options tune concurrency: read_concurrency: true for tables that are read much more than written, and write_concurrency: true when many processes write different keys.

A read-through cache

A typical pattern is a cache where a miss computes the value and stores it:

ELIXIR
defmodule Cache do
  @table :app_cache

  def init_table do
    :ets.new(@table, [:set, :public, :named_table, read_concurrency: true])
  end

  def fetch(key, fun) do
    case :ets.lookup(@table, key) do
      [{^key, value}] ->
        value

      [] ->
        value = fun.()
        :ets.insert(@table, {key, value})
        value
    end
  end
end

Be aware of one limitation: two processes that miss at the same moment will both compute the value. For a cache that is fine, since the results are the same. If the computation must happen only once, route writes through a single process.

Registry: finding processes by name

When you start many processes (one per chat room, one per user session), you need a way to find them without inventing an atom for each. Never create atoms from user input (see the note on atom exhaustion in Security Basics). A Registry maps any term to a process.

ELIXIR
# In your supervision tree:
children = [
  {Registry, keys: :unique, name: MyApp.Registry}
]

A process registers itself, or is started with a :via name:

ELIXIR
defmodule Room do
  use GenServer

  def start_link(room_id) do
    GenServer.start_link(__MODULE__, room_id, name: via(room_id))
  end

  def post(room_id, text), do: GenServer.cast(via(room_id), {:post, text})

  defp via(room_id), do: {:via, Registry, {MyApp.Registry, room_id}}

  @impl true
  def init(room_id), do: {:ok, %{id: room_id, messages: []}}

  @impl true
  def handle_cast({:post, text}, state) do
    {:noreply, %{state | messages: [text | state.messages]}}
  end
end

Now Room.post("elixir-fans", "hello") finds the right process by its string id. You can also look one up directly:

ELIXIR
Registry.lookup(MyApp.Registry, "elixir-fans")
# [{#PID<0.250.0>, nil}]

The result is a list of {pid, value} pairs, empty when nothing is registered. When a registered process exits, its entry is removed automatically.

With keys: :duplicate, many processes can register under one key, which turns the Registry into a lightweight publish/subscribe tool: Registry.dispatch/3 runs a function over all processes for a key.

ELIXIR
Registry.dispatch(MyApp.Registry, "news", fn entries ->
  for {pid, _value} <- entries, do: send(pid, {:headline, "Elixir 2.0?"})
end)

Registry itself is built on ETS tables, partitioned for speed. You will use this in the chat-room project later.

persistent_term: global, read-mostly values

:persistent_term stores a term so that reading it is extremely fast and does not copy it. That makes it good for configuration or lookup data that is computed once at startup.

ELIXIR
:persistent_term.put({MyApp, :country_codes}, %{"DE" => "Germany", "FR" => "France"})

:persistent_term.get({MyApp, :country_codes})
# %{"DE" => "Germany", "FR" => "France"}

:persistent_term.get({MyApp, :missing}, :default)
# :default

The price is on the write side. Replacing or deleting a value can trigger a global garbage collection that scans every process, so use it only for data that almost never changes. If values change often, use ETS.

Process or table?

Need Reach for
State changes must be applied one by one, with validation A GenServer or Agent
Many readers, simple reads and writes by key ETS
Counters updated from many processes ETS with update_counter
Find a process by id or name Registry
Rarely changing global data persistent_term

ETS is not a database. It has no transactions across tables, no persistence, and no distribution; the data is gone when the node stops. Do not hide complex business rules behind a public table that anyone can write to. A common design is a GenServer that owns and writes the table (protected mode) while everyone else reads directly.

A table model in JavaScript

JavaScript has no shared-memory tables between workers like this, but a class can illustrate owners and access rules. It is a stand-in, not ETS.

JS A tiny ETS-like table with access modes

Press Run (or Ctrl + Enter) to see the console output here.

No output.

JS A registry that drops dead entries

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does :ets.lookup(:users, 99) return when no row has key 99?

    lookup always returns a list of matching rows, so a missing key gives an empty list.
  2. 2. What happens to an ETS table when the process that created it exits?

    A table lives as long as its owner. Create shared tables in a stable process that does little else.
  3. 3. Which access mode lets any process read the table but only the owner write to it?

    protected is the default: reads are open, writes are limited to the owner.
  4. 4. Why is persistent_term a poor choice for data that changes every second?

    Reads are very fast, but replacing or deleting values can make the runtime scan all processes. Use it for rarely changing data.
  5. 5. What does Registry.lookup/2 return?

    The result is a list, empty if nothing is registered, because a duplicate registry can hold several processes per key.

Summary

  • ETS tables hold tuples outside process heaps, so many processes can read without messaging one owner.
  • Table types are :set, :ordered_set, :bag and :duplicate_bag; access is :public, :protected (default) or :private.
  • lookup returns a list, update_counter is atomic, and the table dies with its owner.
  • Registry maps any term to a process and cleans up when processes exit; :via tuples name GenServers by it.
  • :persistent_term suits global data that is read constantly and written almost never.
  • Use a process to guard rules, and ETS for shared reads and counters. ETS is not a database.

Next: writing tests for all of this in Testing with ExUnit.