Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 38 ยท AWK Basics

Your First AWK Program: Patterns, Actions and BEGIN/END

Write your first AWK programs: the pattern-action model, running from the command line or a file, BEGIN and END blocks, the built-in variables NR and NF, and comments.

sed edits text line by line. AWK reads text line by line too, but it treats each line as a record of fields and gives you a real programming language to work with them. The core idea is only one sentence:

An AWK program is a list of pattern { action } rules. For every input line, AWK runs the action of each rule whose pattern matches.

In this lesson you will write programs in that form, learn the two special patterns BEGIN and END, and meet the first built-in variables. Everything runs in the browser on the real AWK.

Running AWK

There are three ways to run a program:

TEXT
awk 'program' file1 file2 ...      # program on the command line (most common)
awk -f program.awk file ...        # program in a file
command | awk 'program'            # input from a pipe

The program is almost always in single quotes, so the shell does not touch $1.

Bash The shortest AWK programs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Two defaults to remember:

  • No pattern means "every line".
  • No action means { print } (print the whole line).

So awk '/grace/' behaves like grep grace, and awk 'NR==2' prints line 2.

Fields: $0, $1, $2 ...

Each line is split into whitespace-separated fields. $1 is the first, $2 the second, $NF the last. $0 is the whole line. print $1, $3 prints two fields separated by a space; there is more on separators in the next lesson.

The built-in variables NR and NF

Variable Meaning
NR number of records read so far (the current line number)
NF number of fields in the current record
FILENAME name of the current input file
FNR record number in the current file (restarts for each file)
Bash NR, NF and conditions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The condition $3 > 100 compares numerically because the field looks like a number. How AWK decides between number and string comparison is the topic of AWK variables.

BEGIN and END

Two special patterns have no input line behind them:

  • BEGIN { ... } runs once before any input is read. Use it to set variables (separators, counters) and print headings.
  • END { ... } runs once after the last line. Use it to print totals. In END, $0 and NF keep the values of the last record, and NR is the total number of lines.
flowchart TD
    A["BEGIN { ... } runs once"] --> B["read next record"]
    B --> C["for each rule: if pattern matches, run action"]
    C --> D{"more input?"}
    D -->|"yes"| B
    D -->|"no"| E["END { ... } runs once"]

AWK's main loop. The BEGIN and END blocks surround it.

Bash A report with a heading and totals

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

total += $3 adds the third field to total. Uninitialized variables start as 0 (and as the empty string in text context), so you do not need to declare anything. The middle rule has no pattern, so it runs for every line.

A program with only a BEGIN rule never reads input; AWK exits after running it. That makes AWK a calculator:

Bash AWK as a calculator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Several rules in one program

A program may contain many rules. For each line, AWK checks them in order and runs every matching action; you do not need else. Use next to skip the remaining rules for the current line.

Bash Multiple rules and next

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

next stops processing the current line and reads the next, so a line counted as an error is not counted again as "other". Without next, a line matching two patterns runs both actions.

Layout, semicolons and comments

AWK is forgiving about layout:

  • A rule's opening brace must be on the same line as the pattern (/x/ {), otherwise the pattern and the action become separate rules.
  • Statements are separated by newlines or semicolons.
  • # starts a comment that runs to the end of the line.
  • You can split a long line after ,, {, && or ||, or end a line with a backslash.

For longer programs, use a file with -f. The first line may be #!/usr/bin/awk -f:

Bash A program in a file

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Try it: some quick one-liners

Goal Program
Count lines awk 'END { print NR }' file
Print the last line awk 'END { print $0 }' file
Print non-empty lines awk 'NF' file
Print lines longer than 20 chars awk 'length($0) > 20' file
Print line numbers and lines awk '{ print NR ": " $0 }' file

NF as a pattern means "non-zero", so awk 'NF' drops blank lines. You will find many more in the one-liners chapter.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does awk '/grace/' do?

    A rule with a pattern but no action prints the matching lines.
  2. 2. When does an END block run?

    END runs once after all input is read; NR then holds the total number of records.
  3. 3. What does next do?

    next skips the remaining rules for the current line and moves on to the next line.
  4. 4. What does awk 'NF' file print?

    NF is zero for an empty line, which is false; any other value is true and selects the line.

Summary

  • An AWK program is a list of pattern { action } rules run for every record.
  • A missing pattern means every line; a missing action means print.
  • BEGIN runs once before input, END once after it; NR and NF count records and fields.
  • next skips the remaining rules for the current line; -f file runs a program from a file.

Next: Fields and Separators