Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 36 ยท Data Structures

Binaries and Bit Syntax

Build and parse binary data in Erlang with bit syntax: segments, sizes, types, endianness, pattern matching network packets, bitstrings and binary functions.

A binary is a compact sequence of bytes. It is the type you use for network packets, file contents, encoded text and any raw data. Erlang's bit syntax is one of its signature features: the same notation builds binaries and takes them apart with pattern matching, which makes protocol code remarkably short.

Creating binaries

ERLANG
1> <<1, 2, 3>>.
<<1,2,3>>
2> <<"hello">>.
<<"hello">>
3> <<72, 105>>.
<<"Hi">>
4> byte_size(<<1, 2, 3>>).
3
5> <<"abc", 100>>.
<<"abcd">>

Between << and >> you write segments separated by commas. Each segment defaults to one byte (an integer 0 to 255, larger values wrap) or a string, which is expanded to its bytes. The shell prints a binary of printable bytes as text.

Segment specifications

A segment has the general form Value:Size/TypeSpecifierList:

ERLANG
1> <<258:16>>.
<<1,2>>
2> <<258:8>>.
<<2>>
3> <<1:4, 15:4>>.
<<31>>
4> <<3.14:32/float>>.
<<64,72,245,195>>
5> <<"รฉ"/utf8>>.
<<195,169>>
  • Size is in bits by default (or in units set by unit): 258:16 is two bytes.
  • Type can be integer (default), float, binary, bytes, bitstring, bits, utf8, utf16, utf32.
  • Signedness: signed or unsigned (default).
  • Endianness: big (default, network order), little or native.
ERLANG
1> <<1:16/little>>.
<<1,0>>
2> <<1:16/big>>.
<<0,1>>
3> <<-1:8/signed>>.
<<255>>

Pattern matching on binaries

This is where bit syntax pays off. Match fixed-size fields first, then capture the remainder as a binary:

ERLANG
1> <<A, B, Rest/binary>> = <<1, 2, 3, 4, 5>>.
<<1,2,3,4,5>>
2> A.
1
3> B.
2
4> Rest.
<<3,4,5>>

Rules: a segment of type binary without a size must be the last one. A size can be a variable bound earlier in the same pattern, which is the secret of length-prefixed formats:

ERLANG
parse_message(<<Len:8, Body:Len/binary, Rest/binary>>) ->
    {Body, Rest}.
ERLANG
1> parse_message(<<5, "hello world">>).
{<<"hello">>,<<" world">>}

Parsing a packet header

Suppose a tiny packet has a 4-bit version, a 4-bit type, a 16-bit length and then the payload:

ERLANG
-module(packet).
-export([parse/1, build/3]).

parse(<<Version:4, Type:4, Length:16, Payload:Length/binary, _/binary>>) ->
    #{version => Version, type => Type, payload => Payload}.

build(Version, Type, Payload) ->
    Length = byte_size(Payload),
    <<Version:4, Type:4, Length:16, Payload/binary>>.
ERLANG
1> Pkt = packet:build(1, 7, <<"data">>).
<<23,0,4,100,97,116,97>>
2> packet:parse(Pkt).
#{payload => <<"data">>,type => 7,version => 1}

The same notation that builds the packet takes it apart. Compare that with shifting and masking by hand in other languages.

Matching text protocols

ERLANG
route(<<"GET ", Path/binary>>) -> {get, Path};
route(<<"POST ", Path/binary>>) -> {post, Path};
route(_) -> unknown.

Because a binary pattern like <<"GET ", ...>> compares bytes directly, it is fast.

Bitstrings

A binary is a bitstring whose length is a multiple of 8. If the length is not, it is a general bitstring:

ERLANG
1> <<5:3>>.
<<5:3>>
2> is_bitstring(<<5:3>>).
true
3> is_binary(<<5:3>>).
false
4> bit_size(<<5:3>>).
3

Bitstrings let you pack flags and fields at bit level, such as in compression or encoding code.

Binary functions

ERLANG
1> binary:part(<<"abcdef">>, 2, 3).
<<"cde">>
2> binary:split(<<"a,b,c">>, <<",">>, [global]).
[<<"a">>,<<"b">>,<<"c">>]
3> binary:match(<<"hello world">>, <<"world">>).
{6,5}
4> binary:replace(<<"aXbX">>, <<"X">>, <<"-">>, [global]).
<<"a-b-">>
5> binary_to_list(<<1, 2>>).
[1,2]
6> list_to_binary([1, 2, <<3>>, "ab"]).
<<1,2,3,97,98>>
7> binary:copy(<<"ab">>, 3).
<<"ababab">>
8> iolist_to_binary(["abc", [<<"de">>, $f]]).
<<"abcdef">>
  • binary_to_term/1 and term_to_binary/1 turn any term into a binary and back, used for storage and messaging.
  • binary:split/3, binary:match/2, binary:replace/4 and binary:part/3 replace most manual loops.
  • Large binaries (over 64 bytes) live outside the process heap and are reference-counted, so passing them between processes is cheap. Slices made by matching share memory with the original.

Prefer binaries over strings (lists) for large or network text. A list uses a cell per character, a binary one byte. Convert to a list only when you need list operations.

Stand-ins in JavaScript

DataView and Uint8Array give similar tools. These samples are not Erlang.

JS Stand-in: building and parsing a packet (JavaScript, not Erlang)

Press Run (or Ctrl + Enter) to see the console output here.

No output.

JS Stand-in: a length-prefixed message (JavaScript, not Erlang)

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does <<258:16>> produce?

    258 in 16 bits big-endian is 0x0102, the bytes 1 and 2.
  2. 2. In <<Len:8, Body:Len/binary, Rest/binary>>, what makes it work for length-prefixed data?

    A size may refer to a variable bound earlier in the same pattern.
  3. 3. When is a bitstring not a binary?

    A binary is a bitstring whose bit size is divisible by 8.
  4. 4. Which segment type specifier selects little-endian order?

    Use /little, /big (default) or /native for byte order.

Summary

  • <<...>> builds binaries from segments written Value:Size/Type; the default segment is one byte.
  • Specifiers cover integer, float, binary, utf8, signedness and endianness.
  • Pattern match binaries to parse them; a size may use an earlier variable, and a trailing Rest/binary captures the remainder.
  • Bitstrings need not be whole bytes; binary module functions handle splitting, matching and replacing.
  • Prefer binaries over strings for large data; term_to_binary/1 serializes any term.

Next: Processes and spawn