Linux Command Reference ยท Text Processing
Linux awk command
Reference for the Linux awk command: pattern-action programs, fields, built-in variables, BEGIN and END, arrays, printf, user functions and mawk versus gawk.
awk is a small programming language for processing text records. It reads input one record (by default a line) at a time, splits it into fields, and runs your program on each record. A program is a list of pattern { action } pairs: if the pattern matches the record, the action runs. A missing pattern matches every record; a missing action prints the record.
It is the standard tool for column-oriented data such as logs, CSV-like tables and the output of other commands: sums, averages, filters, reformatting and simple reports fit in one line.
awk [OPTION]... 'PROGRAM' [FILE]...
Try it
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.
Syntax
awk [-F FS] [-v VAR=VALUE]... 'PROGRAM' [FILE]... awk [-F FS] [-v VAR=VALUE]... -f PROGFILE [FILE]...
A program is pattern { action } items separated by newlines or semicolons. Special patterns are BEGIN (before input) and END (after input). A range pattern p1, p2 selects from a record matching p1 through one matching p2.
| Variable | Meaning |
|---|---|
$0, $1, $2, ... |
The whole record, and the fields. $NF is the last field. |
NF |
Number of fields in the record. |
NR, FNR |
Record number overall, and within the current file. |
FS, OFS |
Input and output field separator (default: runs of blanks / one space). |
RS, ORS |
Input and output record separator (default: newline). |
FILENAME |
Name of the current input file. |
| Statement / function | Description |
|---|---|
print a, b |
Print values separated by OFS, ended by ORS. |
printf fmt, ... |
Formatted output like C; no newline unless you write \n. |
if, while, for, do, break, next, exit |
Control flow; next skips to the next record, exit goes to END. |
length(s), substr(s, start, len), index(s, t) |
Length, substring (1-based), position of t in s. |
split(s, arr, sep), sub(re, repl, target), gsub(...) |
Split into an array; replace first / all matches (target defaults to $0). |
match(s, re) |
Sets RSTART and RLENGTH. |
tolower, toupper, sprintf, int, sqrt, sin, cos, exp, log, rand, srand |
Common functions. |
arr[key], key in arr, delete arr[key], for (k in arr) |
Associative arrays. |
getline, system(cmd), close(cmd) |
Read another line; run a command. |
Returns: exit status 0 normally, or the value given to exit; 2 for a syntax error.
Usage notes
- Quote the program with single quotes so the shell does not expand
$1. For shell values pass them with-v name=value, not by splicing into the program text. - Default field splitting treats any run of spaces and tabs as one separator and ignores leading blanks. With
-F,every comma separates, so empty fields appear. A single space-F' 'is the default behavior. - Comparisons: fields that look like numbers compare numerically, otherwise as strings.
$1 == "10"forces a string comparison;$1 + 0 == 10forces a number. - Uninitialized variables are
0and"", which is whysum += $2works without a setup step. - Array order is unspecified.
for (k in a)may list keys in any order, and the order differs between awk implementations. Pipe throughsortwhen the output must be stable. - Assigning to a field (
$3 = "") or to$0rebuilds the record usingOFS. Assigning toNFtruncates or extends it. - mawk vs gawk. The default
awkon Debian-like systems may bemawk, which is fast but lacks gawk-only features:gensub,strftime,asort, andPROCINFO. Scripts that must run anywhere should stick to POSIX features. Check withawk -W versionorawk --version. - Common mistakes: using
$before a plain variable ($sumis a field!), forgetting that string indexes start at 1, and printingNRexpecting a count when the file is empty (ENDruns,NRis 0). - When to use something else: simple text replacement is quicker with
sed, picking columns withcut, and searching withgrep.
Related
sed,grep,cut,paste,join- Lessons: awk, Regular expressions, printf and date