Linux Command Reference Β· Text Processing
Linux iconv command
Reference for the Linux iconv command: convert files between UTF-8, Latin-1 and other encodings, transliterate or skip bad characters, and list supported encodings.
iconv reads text in one character encoding and writes it in another. A file saved as Latin-1 (ISO-8859-1) from an old system, a Windows export in CP1252, or UTF-16 from a spreadsheet can all be turned into plain UTF-8 with it, which is what nearly every modern Linux tool expects.
The tool only converts bytes to bytes; it does not guess the source encoding. You must say what the input is with -f and what you want with -t. If a character cannot be represented in the target encoding, or the input contains invalid bytes, iconv stops with an error unless you tell it to transliterate or discard.
iconv -f FROM -t TO [FILE]...
Try it
Press Run (or Ctrl + Enter) to see the result here.
Program input Β· what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.
Press Run (or Ctrl + Enter) to see the result here.
Program input Β· what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.
Syntax
iconv [-c] [-s] -f FROM-ENCODING -t TO-ENCODING[//SUFFIX] [-o OUTFILE] [FILE]... iconv -l
| Option | Description |
|---|---|
-f, --from-code=NAME |
Encoding of the input. Defaults to the current locale's encoding. |
-t, --to-code=NAME |
Encoding of the output. Defaults to the locale's encoding. |
-c |
Omit characters that cannot be converted instead of failing. |
-s |
Silent: do not report conversion problems on standard error. |
-o FILE |
Write to FILE instead of standard output. |
-l, --list |
List the supported encoding names. |
//TRANSLIT |
Suffix on the target name: replace unrepresentable characters with look-alikes. |
//IGNORE |
Suffix on the target name: silently drop unrepresentable characters. |
Returns: 0 on success, 1 if input could not be converted (with -c, older glibc returns 0 and newer versions return 1).
Errors: cannot convert for an unsupported pair of names, illegal input sequence for bytes that are not valid in the source encoding, and conversion stopped due to problem in writing the output when the target cannot hold a character.
Usage notes
- You must know the source encoding.
file -igives a hint, but it is a guess. Converting with the wrong-fdoes not fail for single-byte encodings like Latin-1; it simply produces garbage. //TRANSLITis approximate.e-acutebecomeseor?depending on the locale, so setLC_ALLto a UTF-8 locale for predictable results. It is good for building ASCII file names or slugs, not for lossless storage.- Check the exit status of
-con your system. The glibc 2.28 used here returns 0 after dropping characters, while newer glibc versions return 1, so a script usingset -ecan behave differently on different machines. - Same-encoding conversion is a validator.
iconv -f UTF-8 -t UTF-8 file > /dev/nullfails on the first invalid byte, a quick check that a file is clean UTF-8. - Do not overwrite the input.
iconv -f A -t B file > filetruncatesfilebefore reading. Use-o, a temporary file, orsponge. - Byte order marks.
-t UTF-16writes a BOM and uses the machine's byte order; useUTF-16LEorUTF-16BEfor a fixed order without a BOM. A UTF-8 BOM from Windows tools stays in the data as three bytes, whichsed '1s/^\xEF\xBB\xBF//'removes. - Line endings are not touched. CRLF to LF is a job for
tr -d '\r'orsed. - Encoding names are case-insensitive and have aliases (
UTF8,utf-8,LATIN1,CP1252,WINDOWS-1252).iconv -lprints all of them for your C library. - Different implementations (glibc, musl, BSD) support different name sets and suffix rules;
//TRANSLITbehaves differently on musl.
Related
file,od,wc,strings- Lessons: Viewing Text, Binary and Hex Tools