Skip to content
Tutorials Center
My learning

Topics

Node.js Reference Β· Streams and readline

Node.js TextDecoderStream and TextEncoderStream

Reference for TextDecoderStream and TextEncoderStream in Node.js: turn byte streams into strings and back, handle split characters, BOMs and bad input.

TextDecoderStream is a transform stream that converts chunks of bytes into strings. TextEncoderStream does the opposite: strings in, UTF-8 Uint8Array chunks out. They are globals in Node.js, and they wrap TextDecoder and TextEncoder with streaming behavior.

The important feature is correct handling of characters that straddle chunk boundaries. A multi-byte UTF-8 character can arrive split across two chunks, and decoding each chunk separately would produce garbage. The decoder stream keeps the incomplete bytes and finishes the character when the next chunk arrives.

NODE
const text = response.body.pipeThrough(new TextDecoderStream());

Try it

Node.js Characters split across chunks

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Node.js Encoding, options and invalid bytes

Press Run (or Ctrl + Enter) to see the result here.

Program input Β· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Syntax

NODE
new TextDecoderStream(label = 'utf-8', options?)
new TextEncoderStream()
decoderStream.encoding   // string
decoderStream.fatal      // boolean
decoderStream.ignoreBOM  // boolean
stream.readable
stream.writable
Name Description
label Encoding name such as 'utf-8', 'utf-16le' or 'latin1'. Supported labels depend on the ICU build of your Node.js.
options.fatal When true, invalid input errors the stream with a TypeError. Default false: invalid bytes become U+FFFD.
options.ignoreBOM When true, a leading byte order mark stays in the output. Default false: it is stripped.

Returns: a transform stream pair. The decoder's writable side accepts ArrayBuffer, typed arrays and Buffers; its readable side emits strings. The encoder's writable side accepts strings; its readable side emits Uint8Array.

Errors: an unsupported label throws a RangeError (ERR_ENCODING_NOT_SUPPORTED). With fatal: true, bad bytes produce ERR_ENCODING_INVALID_ENCODED_DATA.

Usage notes

  • Never call toString() per chunk. It cannot know a character continues in the next chunk. The decoder stream, or StringDecoder for Node streams, can.
  • TextEncoderStream is UTF-8 only. It takes no label. Lone surrogates in input are replaced by U+FFFD.
  • Lines are not chunks. Decoding gives you text pieces with arbitrary boundaries; use a TransformStream to split lines.
  • The BOM is stripped once. Only a BOM at the very start of the stream is removed.
  • Pairs well with fetch. response.body.pipeThrough(new TextDecoderStream()) streams a response as text.
  • Added in Node.js 18.