Skip to content
Tutorials Center
My learning

Topics

Lesson 16 of 38 ยท AWK Basics

AWK Patterns: Regex, Expressions and Range Patterns

Choose which lines an AWK action runs on: regex patterns, comparison and match operators, combining with && || !, field-specific matches, range patterns and the BEGIN/END patterns.

The part of a rule in front of the braces is the pattern. It decides which lines the action runs on. AWK patterns are far more flexible than sed addresses because they can be any expression: a regex, a comparison, a test on a field, a combination with and, or, not, or a range between two events. This lesson covers them all.

The kinds of pattern

Pattern Example Selects lines where...
regex /error/ the line matches the regex
expression $3 > 100 the expression is true (non-zero, non-empty)
combined /a/ && $2 == 5 both parts are true
range /start/,/end/ between a start and an end line
special BEGIN, END before / after the input
empty (nothing) every line

An expression is true if it is a non-zero number or a non-empty string. So NF selects lines with at least one field, and $1 selects lines whose first field is not empty and not 0.

Regex patterns

/regex/ is shorthand for $0 ~ /regex/. The regex is ERE: + ? | ( ) are special as they are (see Regular Expressions).

Bash Regex patterns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

For case-insensitive matching in portable AWK, convert first: tolower($0) ~ /error/.

Matching a specific field with ~ and !~

A bare regex searches the whole line. To test one field, use ~ (matches) or !~ (does not match):

Bash Field-specific matching

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The anchors ^ and $ make the match exact for that field. $1 ~ /GET/ would also match a field like FORGET. If you need an exact string, == is simpler: $1 == "POST".

Comparisons: numbers or strings?

Comparison operators are < <= == != >= >. AWK decides how to compare based on the types:

  • If both operands are numbers (or fields/values that look like numbers), the comparison is numeric.
  • Otherwise it is a string comparison, by character codes.
  • A string constant like "10" is always a string.
Bash Numeric versus string comparison

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

In the first command abc is also printed: it does not look numeric, so it is compared as a string with 9 and a sorts after 9. In the second command "10" > "9" is false as strings (since 1 < 9). When mixing, force the type with +0 (number) or "" (string) to be explicit. The next lesson explains the rules in depth.

Combining patterns: && || !

Use && (and), || (or), ! (not) and parentheses:

Bash Logical operators in patterns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

! in front of a regex also works: !/^#/ selects every line that is not a comment, and NR % 2 selects odd lines (NR % 2 == 1).

Bash Skip comments and blank lines in one pattern

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Range patterns

pattern1, pattern2 selects from a line where pattern1 is true to the next line where pattern2 is true, inclusive. Like sed's ranges, it can match multiple times, and it runs to the end of the input if the end never matches. One difference: AWK tests pattern2 on the same line that matched pattern1, so a one-line range is possible.

Bash Range patterns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

For fancier cases (the end excluded, nested ranges, state that carries into other rules) use a flag variable, which is easier to reason about than a range:

Bash A flag instead of a range

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The order of the three rules is the trick: END turns the flag off before the print rule can see the marker line, and BEGIN turns the flag on after it, so neither marker is printed.

BEGIN and END as patterns

They are patterns too and combine only with an action. You can have several BEGIN or END blocks; AWK runs them in order. BEGIN and END cannot be combined with other patterns (no BEGIN && x, no ranges).

The ternary operator and short-circuiting

&& and || stop as soon as the answer is known (short-circuit). That lets you guard an expression: NF >= 3 && $3 > 100 never reads $3 when it is missing (though reading a missing field is harmless in AWK; it is just empty). For choosing a value inside an action, use condition ? a : b.

Bash Short-circuit and ternary

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does awk '$3 > 100' print?

    With no action the default print runs on every line for which the pattern is true.
  2. 2. How do you test whether field 2 does NOT match the regex ^a?

    !~ is the negated match operator.
  3. 3. What does awk '/BEGIN/,/END/' select?

    A range pattern includes both ends and can repeat for several blocks.
  4. 4. Why does awk '$1 > "9"' treat the value 10 as smaller than 9?

    When one operand is a string constant, the comparison is done as strings, character by character.
  5. 5. Which pattern selects non-comment, non-empty lines?

    Both conditions must hold: not starting with #, and at least one field.

Summary

  • A pattern can be a regex, any expression, a combination with &&, ||, !, a range, or BEGIN/END.
  • ~ and !~ test a regex against a specific field; == is for exact comparison.
  • Fields that look like numbers compare numerically; string constants force string comparison.
  • Range patterns are inclusive and repeat; flag variables handle trickier block logic.

Next: Variables, Types and Operators