Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 48 ยท Real-World Elixir

Files, JSON and Serialization

Read and write files with Elixir's File and Path modules, stream large files line by line, encode JSON with Jason, and serialize terms with term_to_binary.

Programs eventually need to talk to the outside world: read a configuration file, write a report, send data to a web client or save a term for later. Elixir handles all of this with a few small modules. File and Path deal with the file system, File.stream!/1 reads big files without loading them, JSON libraries convert between maps and text, and :erlang.term_to_binary/1 saves any Elixir value in a compact binary format.

You will learn the {:ok, _} / {:error, _} convention of file functions, safe path handling, streaming, JSON with Jason (and the built-in JSON module in recent Elixir releases), and when to use Erlang's binary format. You should know error handling and streams.

Elixir cannot run on this site, and a browser sandbox has no real file system, so every Elixir snippet is read-only. The JavaScript TryIts are stand-ins that show the same data transformations.

Reading and writing files

Most File functions come in two forms. The plain one returns a tuple, and the one ending in ! raises on failure:

ELIXIR
File.write("notes.txt", "first line\nsecond line\n")
# :ok

File.read("notes.txt")
# {:ok, "first line\nsecond line\n"}

File.read("missing.txt")
# {:error, :enoent}

File.read!("missing.txt")
# ** (File.Error) could not read file "missing.txt": no such file or directory

The error reasons are atoms from the operating system: :enoent (does not exist), :eacces (no permission), :eisdir (it is a directory). Use the tuple version when failure is normal and you want to pattern match on it, and the bang version in scripts or when a missing file is a bug.

ELIXIR
case File.read("config.txt") do
  {:ok, text} -> parse(text)
  {:error, :enoent} -> default_config()
  {:error, reason} -> raise "cannot read config: #{:file.format_error(reason)}"
end

Other everyday functions:

ELIXIR
File.write("log.txt", "started\n", [:append])   # add to the end
File.exists?("log.txt")                          # true
File.mkdir_p("data/reports/2026")                # creates every missing level
File.ls!("data")                                 # list names in a directory
File.cp!("a.txt", "b.txt")                       # copy
File.rename!("b.txt", "c.txt")                   # move
File.rm("c.txt")                                 # delete a file
File.rm_rf("data/old")                           # delete a directory tree (careful)

File.write/3 replaces the whole file unless you pass :append. To avoid half-written files after a crash, write to a temporary name and then File.rename/2 it, because a rename on the same disk is atomic on common systems.

Building paths with Path

Never build paths by gluing strings with "/". The Path module does it correctly and works on Windows too:

ELIXIR
Path.join("data", "reports")           # "data/reports"
Path.join(["data", "2026", "a.csv"])   # "data/2026/a.csv"
Path.basename("data/2026/a.csv")       # "a.csv"
Path.extname("data/2026/a.csv")        # ".csv"
Path.dirname("data/2026/a.csv")        # "data/2026"
Path.rootname("a.csv")                 # "a"
Path.expand("~/notes")                 # absolute path of your home folder
Path.wildcard("data/**/*.csv")         # all matching files

When part of a path comes from a user, remember that "../../etc/passwd" is a valid input. Path.safe_relative/1 (Elixir 1.14 and later) returns {:ok, path} only if the result stays inside the base directory and :error otherwise. Use it before touching any user-controlled name. This ties into Security Basics.

Four tools for four jobs

File Read, write, copy and delete.
Path Build and take apart paths safely.
Streams Process big files line by line.
Encoders Turn terms into JSON or binaries.
1 / 2

Streaming large files

File.read!/1 puts the whole file in memory. For a file of several gigabytes, use File.stream!/1, which yields one line at a time as a lazy stream:

ELIXIR
"logs/app.log"
|> File.stream!()
|> Stream.map(&String.trim_trailing/1)
|> Stream.filter(&String.contains?(&1, "ERROR"))
|> Stream.take(10)
|> Enum.to_list()

Because everything before Enum.to_list/1 is lazy, reading stops after ten matches, and memory use stays tiny. A stream can also be written out into another file:

ELIXIR
"input.csv"
|> File.stream!()
|> Stream.map(&String.upcase/1)
|> Stream.into(File.stream!("output.csv"))
|> Stream.run()

Lines keep their trailing newline, so trim it when you compare or split. For a quick check of a file you know is small, File.read!/1 |> String.split("\n") is fine.

The same idea in JavaScript

This stand-in processes lines of a text lazily with a generator, stopping early like Stream.take/2:

JS Lazy line processing

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Only the first four lines were read, because two errors were enough.

JSON

Elixir's standard library did not include a JSON codec for many years, so almost every project uses Jason, a dependency you add to mix.exs ({:jason, "~> 1.4"}). Recent Elixir releases (1.18 and later) also include a built-in JSON module with a similar API (JSON.encode!/1, JSON.decode/1); check your version before choosing. The Jason examples below also read well with JSON.

ELIXIR
Jason.encode!(%{name: "Ada", langs: ["Elixir", "Erlang"], active: true})
# "{\"active\":true,\"langs\":[\"Elixir\",\"Erlang\"],\"name\":\"Ada\"}"

Jason.decode!("{\"name\": \"Ada\", \"age\": 36}")
# %{"age" => 36, "name" => "Ada"}

Jason.decode("not json")
# {:error, %Jason.DecodeError{...}}

Details to remember:

  • Decoded keys are strings. %{"name" => "Ada"}, not %{name: "Ada"}.
  • keys: :atoms converts keys to atoms, but never use it with untrusted input: atoms are never garbage collected, so an attacker could exhaust the atom table. keys: :atoms! only uses atoms that already exist, which is the safer choice.
  • Encoding turns atoms into strings (:ok becomes "ok") and nil into null. Decoding cannot restore atoms or tuples, so a round trip changes the data.
  • Tuples, PIDs and functions cannot be encoded; Jason raises. Convert them first.
  • Jason.encode!(map, pretty: true) produces indented output.

For your own structs you must say which fields to encode:

ELIXIR
defmodule User do
  @derive {Jason.Encoder, only: [:id, :name]}
  defstruct [:id, :name, :password_hash]
end

Jason.encode!(%User{id: 1, name: "Ada", password_hash: "secret"})
# "{\"id\":1,\"name\":\"Ada\"}"

Using only: is the safe default, because it keeps secrets like password_hash out of API responses.

JS A JSON round trip changes the data

Press Run (or Ctrl + Enter) to see the console output here.

No output.

In Elixir the same kind of loss happens: atoms, tuples, dates and structs all need explicit conversion.

Saving any term: term_to_binary

When the data never leaves the BEAM world (a cache file, a snapshot of state, a message between nodes), you can skip JSON. Erlang's external term format stores every term, including atoms, tuples, maps and structs, exactly:

ELIXIR
data = %{user: {:ok, "Ada"}, scores: [1, 2, 3], at: ~D[2026-11-06]}

binary = :erlang.term_to_binary(data)
:erlang.binary_to_term(binary) == data
# true

File.write!("snapshot.bin", binary)
"snapshot.bin" |> File.read!() |> :erlang.binary_to_term() |> Kernel.==(data)
# true

:erlang.term_to_binary(term, [:compressed]) shrinks large terms. There is one serious rule: never call binary_to_term/1 on untrusted data. It can create atoms (exhausting the atom table) and even build function references. Use :erlang.binary_to_term(binary, [:safe]) when the source is not fully under your control, or choose JSON. The format is Elixir/Erlang only, so other languages cannot read it, and the shape of your structs must stay compatible between versions of your code.

Choosing a format

Need Choice
Talk to browsers, APIs, other languages JSON
Config for humans A config file read at startup (see releases and deployment)
Snapshot of internal state term_to_binary (trusted source only)
Huge text data File.stream! plus line-based processing

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does File.read("missing.txt") return?

    Functions without a bang return tuples. :enoent means the file does not exist. File.read! raises instead.
  2. 2. Why is File.stream!/1 better than File.read!/1 for a 5 GB log?

    A stream yields lines on demand and can stop early, while read! loads the whole file into memory.
  3. 3. What type are the keys of a map returned by Jason.decode!/1 by default?

    JSON keys are text, so they are decoded as strings. Converting them to atoms with untrusted input risks exhausting the atom table.
  4. 4. What is the safe way to deserialize a binary from an outside source with Erlang's format?

    The :safe option refuses to create new atoms. For untrusted data, a plain format such as JSON is better still.
  5. 5. Which module builds paths portably?

    Path.join/2 and friends handle separators correctly, and Path.safe_relative/1 helps keep user input inside a folder.

Summary

  • File functions return :ok/{:ok, value}/{:error, reason}; the ! versions raise.
  • Build paths with Path.join/2; check user-provided names with Path.safe_relative/1.
  • File.stream!/1 plus Stream functions processes huge files lazily.
  • Jason (or the built-in JSON in recent Elixir) converts maps and JSON text; decoded keys are strings, and atoms, tuples and structs need explicit handling.
  • :erlang.term_to_binary/1 keeps any term exactly, but only decode it from trusted sources, or with [:safe].

Next: storing data in a database with Ecto: Databases and Changesets.