Skip to content
Tutorials Center
My learning

Topics

Node.js Reference · Buffers and Encoding

Node.js Buffer encodings

Guide to the Node.js Buffer encodings: utf8, utf16le, latin1, ascii, hex, base64 and base64url, how each converts text and bytes, and the pitfalls of lossy and lenient decoding.

An encoding is a rule that maps between text and bytes. Node.js Buffer supports seven of them, and each is used by name in Buffer.from(string, encoding), buf.toString(encoding), buf.write(), buf.fill(), buf.indexOf(), Buffer.byteLength() and in stream and file APIs such as fs.readFile(path, "utf8") or stream.setEncoding("hex").

The encodings fall into two groups. Character encodings (utf8, utf16le, latin1, ascii) turn text into bytes. Binary-to-text encodings (hex, base64, base64url) turn arbitrary bytes into a safe, printable string, so they are used for hashes, keys, tokens and data that must travel inside JSON, URLs or e-mail.

NODE
const buf = Buffer.from("hi");
buf.toString("hex");       // "6869"
buf.toString("base64");    // "aGk="

The names

Encoding Aliases What it is
utf8 utf-8 Unicode, 1 to 4 bytes per character. The default almost everywhere.
utf16le utf-16le, ucs2, ucs-2 Unicode, 2 or 4 bytes per character, little-endian.
latin1 binary One byte per character, code points 0 to 255 only.
ascii 7-bit text; on write it behaves as Latin-1, on read it clears the high bit.
hex Two hexadecimal digits per byte.
base64 Standard Base64 with +, / and = padding.
base64url URL-safe Base64 with - and _, no padding on output.

Names are case-insensitive. Buffer.isEncoding(name) tells you whether a name is valid; an unknown name throws ERR_UNKNOWN_ENCODING. There is no UTF-32, UTF-16BE, Windows-1252 or Shift_JIS: for those, use TextDecoder or buffer.transcode() where it applies.

Try it

Node.js The same text in every encoding

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Node.js Lossy, lenient and URL-safe

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

How each one behaves

utf8 is the default. Characters take one byte (ASCII), two, three or four bytes. Invalid byte sequences decode to the replacement character U+FFFD, and unpaired surrogates in a string are encoded as U+FFFD too, so a round trip of broken text is not exact. A leading byte order mark (EF BB BF) is not removed by toString(); it stays as the character U+FEFF.

utf16le writes each UTF-16 code unit as two bytes, low byte first, so characters beyond U+FFFF become four bytes. toString("utf16le") on an odd number of bytes ignores the last byte. A BOM FF FE is likewise kept as U+FEFF.

latin1 maps each byte to the code point of the same value, so it can carry any byte sequence losslessly through a string: Buffer.from(s, "latin1").toString("latin1") === s whenever all characters are at most U+00FF. When writing, characters above that lose their high byte (a euro sign U+20AC becomes byte 0xAC).

ascii is the same as latin1 for writing. For reading, the high bit of each byte is cleared, so a byte 0xE9 shows as i. It is almost never what you want; use utf8 or latin1.

hex pairs the digits (upper or lower case is accepted). Output is always lowercase.

base64 and base64url accept both alphabets and missing padding when decoding. Output from base64 includes = padding, while base64url output has none. Decoding stops at the first character that is not part of the alphabet, apart from whitespace.

Usage notes

  • Characters are not bytes. "héllo".length is 5 but the UTF-8 byte length is 6. Use Buffer.byteLength() to size buffers, never string.length.
  • Pick utf8 for text and base64 or hex for binary. Never put binary data through utf8 decoding: invalid sequences are replaced and the data is destroyed.
  • Decoding never fails. buf.toString() replaces bad bytes silently. Check with isUtf8() or use TextDecoder with fatal: true when you need errors.
  • Encoding input is lenient too: a bad hex or Base64 string gives a shorter buffer without any error. Validate the string with a regular expression when it comes from users.
  • Streams: setEncoding("utf8") or StringDecoder handles characters split across chunks, a bare chunk.toString() does not.
  • base64url is the right choice for tokens in URLs and file names, and it is the encoding of JWT segments.
  • binary is an old name for latin1, from the days when Node used strings as byte containers. Prefer latin1 and buffers.
  • Case: Buffer.from("x", "UTF8") works, but write names in lowercase for consistency.