Skip to content
Tutorials Center
My learning

Topics

Go Reference ยท Encoding and Data Formats

The Go encoding/binary package

Reference for Go's encoding/binary package: big and little endian, Read and Write for fixed-size structs, Append helpers, varints and what types are allowed.

The encoding/binary package converts numbers to and from byte sequences. It is what you use to read or write file headers, network protocols and any other format where each field has a fixed width and a defined byte order.

Byte order matters: big endian stores the most significant byte first (network order), little endian stores it last. The package exposes both as binary.BigEndian and binary.LittleEndian, and offers a second layer, Read and Write, that handles whole structs through reflection.

Go
b := binary.BigEndian.AppendUint32(nil, 0x01020304) // [1 2 3 4]

Try it

Go Byte order and Append helpers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Go Write and Read a struct, plus varints

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Syntax

Go
var BigEndian bigEndian
var LittleEndian littleEndian
var NativeEndian nativeEndian

// ByteOrder methods (BigEndian and LittleEndian)
Uint16(b []byte) uint16      PutUint16(b []byte, v uint16)      AppendUint16(b []byte, v uint16) []byte
Uint32(b []byte) uint32      PutUint32(b []byte, v uint32)      AppendUint32(b []byte, v uint32) []byte
Uint64(b []byte) uint64      PutUint64(b []byte, v uint64)      AppendUint64(b []byte, v uint64) []byte

func Read(r io.Reader, order ByteOrder, data any) error
func Write(w io.Writer, order ByteOrder, data any) error
func Size(v any) int
func Append(buf []byte, order ByteOrder, data any) ([]byte, error)
func Encode(buf []byte, order ByteOrder, data any) (int, error)
func Decode(buf []byte, order ByteOrder, data any) (int, error)

func PutUvarint(buf []byte, x uint64) int
func Uvarint(buf []byte) (uint64, int)
func PutVarint(buf []byte, x int64) int
func Varint(buf []byte) (int64, int)
func AppendUvarint(buf []byte, x uint64) []byte
func AppendVarint(buf []byte, x int64) []byte
func ReadUvarint(r io.ByteReader) (uint64, error)
func ReadVarint(r io.ByteReader) (int64, error)
Name Description
order binary.BigEndian, binary.LittleEndian or binary.NativeEndian.
data For Write: a fixed-size value, a pointer to one, or a slice of fixed-size values. For Read: a pointer to a fixed-size value or a slice.

Allowed types are bool, the sized integers (int8 to int64, uint8 to uint64), float32, float64, complex64, complex128, and arrays and structs made only of these. Plain int, uint, string, slices inside structs, maps and pointers are not fixed-size and make Write/Read return an error.

Errors and edge cases:

  • Uint32(b) and friends panic if b is shorter than needed; PutUint32 panics if there is not enough room. Check the length first.
  • Read returns io.EOF if no bytes could be read and io.ErrUnexpectedEOF if the data ends in the middle of the value.
  • Uvarint returns n == 0 when the buffer is too small and n < 0 on overflow (more than 64 bits).
  • Blank fields (_) in a struct are skipped on reads and written as zeros.
  • Append, Encode, Decode and NativeEndian need Go 1.23.

Usage notes

  • Struct layout is exactly the field list. There is no padding between fields, unlike a C struct or Go's in-memory layout. Size(h) above is 12, the sum of the field sizes.
  • Speed: Read and Write use reflection. For hot paths call ByteOrder methods on a byte slice directly.
  • Varints take 1 to 10 bytes, with small numbers using fewer. Varint uses zig-zag encoding so small negative numbers stay small. Protocol Buffers use the same scheme.
  • NativeEndian is for sharing memory with code on the same machine. Do not write it to files or networks.
  • Network protocols use big endian by convention, which is why it is also called network order.
  • Bit-level data: this package works in whole bytes. Use shifts and masks from bitwise operators for fields smaller than a byte.