Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 38 ยท One-liners

AWK One-liners: A Practical Collection

A tested collection of AWK one-liners with explanations: print columns, sum and average, filter by field, number and reverse lines, dedupe, transpose and more.

AWK one-liners are small programs you type and throw away: print the second column, add up a column, show lines longer than 80 characters, remove duplicates. They are the fastest way to answer questions about a text file. This lesson is a cookbook; each recipe is short, tested and explained, and grouped by purpose so you can find the right one quickly.

As elsewhere, the examples stick to features that all common AWKs share. Each editor builds its own input, so change it and re-run to experiment.

Fields and columns

Bash Selecting and rearranging columns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The "drop the first column" recipe works by blanking $1 (which rebuilds the line with a leading separator) and then removing that leading space. It is safe for any number of fields.

Filtering rows

Bash Conditions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

awk 'END { print }' works because $0 keeps the last record after the input ends in most AWKs (POSIX specifies it). If you need to be extra safe, store the line yourself: { last = $0 } END { print last }.

Counting and arithmetic

Bash Totals, averages, extremes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The n + 0 in the last line prints 0 instead of an empty string when nothing matched (an unset variable is empty in string context).

Strings and text

Bash Working on text

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Duplicates, ordering, joins

Bash Dedupe and reshape

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

seen[$0]++ == 1 is true exactly on the second occurrence, so each repeated line is printed once. awk 'ORS = NR % 2 ? " " : "\n"' is a beloved trick: the assignment is the pattern (its value is non-empty, so it is true), and it sets the output record separator to a space after odd lines and a newline after even ones.

Reading the shell's world

Bash Mixing with the shell

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

How to build your own

  1. Say it in words: for each line where field 3 is above 100, add field 2 to a total, and at the end print it.
  2. Translate: pattern $3 > 100, action { s += $2 }, end END { print s }.
  3. Test on a tiny input whose answer you know, then on the real data.
  4. Add printf once the logic is right.

Common mistakes

  • Using print $1, $2 when you meant print $1 $2 (comma adds OFS).
  • Comparing numbers that look like strings: $1 > "9" is a text comparison; use $1 + 0 > 9.
  • Starting a max/min search with m = 0: negative numbers will never win; initialise on NR == 1.
  • Forgetting the single quotes around the program.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does awk 'NR == 1 || $1 > m { m = $1 } END { print m }' compute?

    It starts with the first value and replaces m whenever a larger one appears.
  2. 2. What does awk '!seen[$0]++' print?

    The post-increment returns the previous count, which is 0 (false) on first sight, so the negation passes only the first copy.
  3. 3. How can you print all fields except the first?

    Blanking $1 rebuilds the record with a leading separator, which sub then removes.
  4. 4. Why write END { print n + 0 } rather than END { print n }?

    An unset variable is the empty string in string context; adding 0 forces a number.

Summary

  • Most one-liners are pattern { action } plus an END block for totals.
  • $NF, $(NF-1), NR, NF and length cover column and line questions.
  • Accumulate with s += $n, track extremes with NR == 1 || ....
  • !seen[$0]++ dedupes; arrays plus END loops reverse, transpose and join.
  • Say the task in words first, then translate to pattern and action.

Next: Text Surgery One-liners: sed and AWK Together