Skip to content
Tutorials Center
My learning

Topics

Node.js Reference · Buffers and Encoding

Node.js buffer.transcode() function

Reference for Node.js buffer.transcode(): convert a Buffer between utf8, utf16le, latin1 and ascii, understand replacement with '?', and know which encodings it supports.

buffer.transcode(source, fromEnc, toEnc) converts the bytes of a buffer from one text encoding to another and returns a new Buffer. It is imported from node:buffer. Unlike buf.toString() followed by Buffer.from(), it works on bytes directly, without creating a JavaScript string in between.

Only a small set of encodings is supported: utf8, ucs2 (also utf16le), latin1 (also binary) and ascii. Characters that the target encoding cannot represent are not an error; they are replaced by a question mark.

NODE
import { transcode } from "node:buffer";

transcode(Buffer.from("€"), "utf8", "ucs2"); // <Buffer ac 20>

Try it

Node.js Between UTF-8, UTF-16 and Latin-1

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Node.js Unrepresentable characters and errors

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Syntax

NODE
import { transcode } from "node:buffer";

transcode(source, fromEnc, toEnc)
// returns Buffer
Name Description
source A Buffer or Uint8Array holding text in the encoding fromEnc.
fromEnc Current encoding: "utf8", "ucs2", "utf16le", "latin1", "binary" or "ascii".
toEnc Target encoding, from the same set.

Returns: a new Buffer with the converted bytes.

Errors and edge cases:

  • A source that is not a Buffer or Uint8Array (for example a string) throws ERR_INVALID_ARG_TYPE.
  • An unsupported encoding, such as "hex", "base64" or "utf-32", throws an error whose code is an ICU name such as U_ILLEGAL_ARGUMENT_ERROR.
  • Input that is not valid in fromEnc (for example a stray 0xff byte in UTF-8) can throw U_INVALID_CHAR_FOUND.
  • An unrepresentable character is written as ? (byte 0x3f) and counts as one character, not an error.
  • The function needs Node.js to be built with ICU. The official builds are; if it is missing, transcode is undefined.

Usage notes

  • Use the Buffer spellings: write utf8, ucs2, latin1 and ascii in lowercase, exactly as shown above.
  • Lossy in one direction: going to latin1 or ascii can destroy characters, and going back does not restore them. Convert to UTF-8 or UTF-16 when you must not lose text.
  • ascii as target produces ? for every byte above 127, including accented letters that Latin-1 can hold.
  • For other encodings (Windows-1252, Shift_JIS, GBK, ISO-8859-x) use new TextDecoder("windows-1252").decode(buf) to read them, since TextDecoder supports the WHATWG label list. TextEncoder can only produce UTF-8.
  • Decode and encode is often clearer: Buffer.from(buf.toString("latin1"), "utf8") does the same for the supported pairs and handles Base64 or hex too; use transcode() when you want to avoid the intermediate string for large data.
  • UTF-16 is little-endian only. Use swap16() after transcoding for big-endian output.