Node.js Reference · Buffers and Encoding
Node.js buffer.transcode() function
Reference for Node.js buffer.transcode(): convert a Buffer between utf8, utf16le, latin1 and ascii, understand replacement with '?', and know which encodings it supports.
buffer.transcode(source, fromEnc, toEnc) converts the bytes of a buffer from one text encoding to another and returns a new Buffer. It is imported from node:buffer. Unlike buf.toString() followed by Buffer.from(), it works on bytes directly, without creating a JavaScript string in between.
Only a small set of encodings is supported: utf8, ucs2 (also utf16le), latin1 (also binary) and ascii. Characters that the target encoding cannot represent are not an error; they are replaced by a question mark.
import { transcode } from "node:buffer"; transcode(Buffer.from("€"), "utf8", "ucs2"); // <Buffer ac 20>
Try it
Press Run (or Ctrl + Enter) to see the result here.
Program input · what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.
Press Run (or Ctrl + Enter) to see the result here.
Program input · what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.
Syntax
import { transcode } from "node:buffer"; transcode(source, fromEnc, toEnc) // returns Buffer
| Name | Description |
|---|---|
source |
A Buffer or Uint8Array holding text in the encoding fromEnc. |
fromEnc |
Current encoding: "utf8", "ucs2", "utf16le", "latin1", "binary" or "ascii". |
toEnc |
Target encoding, from the same set. |
Returns: a new Buffer with the converted bytes.
Errors and edge cases:
- A
sourcethat is not aBufferorUint8Array(for example a string) throwsERR_INVALID_ARG_TYPE. - An unsupported encoding, such as
"hex","base64"or"utf-32", throws an error whosecodeis an ICU name such asU_ILLEGAL_ARGUMENT_ERROR. - Input that is not valid in
fromEnc(for example a stray0xffbyte in UTF-8) can throwU_INVALID_CHAR_FOUND. - An unrepresentable character is written as
?(byte0x3f) and counts as one character, not an error. - The function needs Node.js to be built with ICU. The official builds are; if it is missing,
transcodeisundefined.
Usage notes
- Use the Buffer spellings: write
utf8,ucs2,latin1andasciiin lowercase, exactly as shown above. - Lossy in one direction: going to
latin1orasciican destroy characters, and going back does not restore them. Convert to UTF-8 or UTF-16 when you must not lose text. asciias target produces?for every byte above 127, including accented letters that Latin-1 can hold.- For other encodings (Windows-1252, Shift_JIS, GBK, ISO-8859-x) use
new TextDecoder("windows-1252").decode(buf)to read them, sinceTextDecodersupports the WHATWG label list.TextEncodercan only produce UTF-8. - Decode and encode is often clearer:
Buffer.from(buf.toString("latin1"), "utf8")does the same for the supported pairs and handles Base64 or hex too; usetranscode()when you want to avoid the intermediate string for large data. - UTF-16 is little-endian only. Use
swap16()after transcoding for big-endian output.