Skip to content
Tutorials Center
My learning

Topics

Linux Command Reference ยท Text Processing

Linux uniq command

Reference for the Linux uniq command: collapse adjacent duplicate lines, count them, show only duplicates or only unique lines, and skip fields or characters.

uniq filters out adjacent repeated lines. It reads a file or standard input, and for each run of identical lines writes just one. With options it can prefix each line with how many times it occurred, print only the lines that were repeated, or print only those that never repeated.

The key word is adjacent: uniq does not remember lines it saw earlier, which is why it is almost always used after sort. The classic frequency table is sort | uniq -c | sort -rn.

Terminal
uniq [OPTION]... [INPUT [OUTPUT]]

Try it

Bash Adjacent lines only

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Bash Frequency table and comparing parts of lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Syntax

Terminal
uniq [-cdDiu] [-f N] [-s N] [-w N] [-z] [INPUT [OUTPUT]]
Option Description
-c Prefix each output line with the number of occurrences in its group.
-d Print one copy of each group that has more than one line.
-D Print all lines of every repeated group.
-u Print only lines that are not repeated.
-i Ignore case when comparing.
-f N Skip the first N fields (blank-separated) when comparing.
-s N Skip the first N characters when comparing.
-w N Compare no more than N characters per line.
-z Lines end in NUL instead of newline.
--group Separate groups with an empty line; accepts separate, prepend, append, both.
INPUT, OUTPUT Input file (default standard input) and output file (default standard output).

Returns: 0 on success, non-zero on error. It does not signal whether duplicates were found.

Usage notes

  • Sort first. uniq only compares each line to the previous one, so a b a stays a b a. For deduplicating unsorted data without reordering, use awk '!seen[$0]++'.
  • sort -u is the shortcut when you only want distinct lines and no counts. uniq earns its place when you need -c, -d or -u.
  • -c pads the count to seven columns followed by a space. Pipe through sed 's/^ *//' or awk '{ print $1 }' if you need to parse it.
  • -f and -s combine: fields are skipped first, then characters. Field separators are runs of blanks, and the blanks before a field count as part of it.
  • Whole-line options: -d and -u look at the same groups, so every group is printed by exactly one of -d or -u.
  • The output file argument is a rarely used second positional argument. A mistaken extra file name silently overwrites that file, so quote carefully when scripting.
  • Locale: the comparison is bytewise equality (or case-folded with -i), so a locale never changes which lines are equal, but it does change how sort orders them.
  • Large inputs work in constant memory because uniq keeps only one line at a time.