Skip to content
Tutorials Center
My learning

Topics

Node.js Reference Β· Buffers and Encoding

The StringDecoder class

Reference for string_decoder.StringDecoder in Node.js: write(), end(), multi-byte characters split across chunks, and how it differs from toString().

A StringDecoder turns a stream of byte chunks into strings without breaking characters that are split across chunk boundaries. A character like € takes three bytes in UTF-8. If a network packet or file chunk ends after the second byte, calling chunk.toString() on it produces a replacement character, and the next chunk starts with a stray byte that decodes badly as well.

The decoder remembers the incomplete tail of each chunk and prepends it to the next one. It is what readable.setEncoding() uses internally.

NODE
import { StringDecoder } from 'node:string_decoder';
const decoder = new StringDecoder('utf8');

Try it

Node.js A character split across two chunks

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Node.js Other encodings and a chunked file

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Syntax

NODE
import { StringDecoder } from 'node:string_decoder';

new StringDecoder(encoding = 'utf8')
decoder.write(buffer: Buffer | TypedArray | DataView | string): string
decoder.end(buffer?: Buffer | TypedArray | DataView): string
decoder.encoding   // string
Name Description
encoding One of utf8, utf16le (alias ucs2), latin1, ascii, hex, base64, base64url. Defaults to utf8.
write(buffer) Decodes as much as forms complete characters and keeps the rest. Returns the decoded string, possibly empty.
end([buffer]) Optionally decodes one last chunk, then returns anything still buffered and resets the decoder.

Returns: a string from write() and end().

Errors: an unknown encoding throws ERR_UNKNOWN_ENCODING; passing something that is not a buffer-like value to write() throws ERR_INVALID_ARG_TYPE. Passing a string to write() returns it unchanged.

Usage notes

  • Always call end(). A truncated sequence stays in the decoder until end() runs, then comes out as U+FFFD. Forgetting it silently loses the last bytes.
  • Invalid bytes (not just incomplete ones) become U+FFFD immediately, as with buf.toString().
  • One decoder per stream. The decoder holds state, so never share an instance between two independent streams.
  • Single buffers do not need it. If you already have all the bytes, buf.toString() or new TextDecoder().decode() is simpler. TextDecoder supports the same idea through decode(chunk, { stream: true }).
  • Streams do it for you. readable.setEncoding('utf8') makes the stream emit strings decoded with a StringDecoder, so a data handler never sees half a character.
  • Binary-to-text encodings. hex, base64 and base64url also need state: base64 encodes three bytes at a time, and the decoder holds back the remainder.
  • Concatenating buffers first with Buffer.concat(chunks).toString() also works, but keeps the whole input in memory.