Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 38 ยท AWK Programs

User-defined Functions and Recursion in AWK

Write your own AWK functions: parameters, return values, local variables, passing arrays by reference, recursion, helper libraries and loading code with -f.

Once an AWK program grows past a screenful, repeating the same few lines (trim a string, convert units, format a date) becomes a burden. AWK lets you define functions, so a program can read like a small script with named building blocks. They also make recursion possible, which suits tree-shaped text such as nested lists.

The syntax is close to C and JavaScript, with two twists that surprise newcomers: local variables are declared as extra parameters, and arrays are always passed by reference.

Defining and calling a function

AWK
function name(param1, param2) {
  ...
  return value
}

The keyword may be written func in some versions, but function is the portable spelling. Definitions can appear anywhere at the top level of the program (before or after the code that calls them). A call must have no space between the name and the opening parenthesis: max(a, b), not max (a, b).

Bash Small helper functions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Scalars are passed by value, so trim(s) can change its own copy of s freely and the caller's variable is untouched. The function returns its result with return; falling off the end returns an empty value.

Local variables: the extra-parameter trick

AWK has no local keyword. Any variable you use inside a function is global unless it is a parameter. So the convention is to add the locals to the parameter list, separated from the real parameters by extra spaces:

AWK
function repeat(s, n,   out, i) { ... }

Callers pass only s and n; out and i start empty on every call and vanish at the end. Forgetting this is the number one source of bugs: a loop variable i inside a function silently overwrites the i the caller was looping with.

Bash The global-variable trap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The loop with bad does not run three times as written: the call resets i to 3 and the loop leaves early. The second loop with properly declared locals behaves.

Passing arrays

An array parameter is passed by reference: changes made in the function are visible to the caller. An array local to a function is created simply by using an unused parameter as an array. A scalar parameter can never become an array on the fly if the caller passed a scalar, but an untouched local can.

Bash Arrays as arguments

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Because there is no portable length(array), functions take the element count n as an argument.

Recursion

A function may call itself. Each call gets its own parameters and locals, so recursion works as expected. Keep a base case, and remember that AWK has a finite call stack (thousands of levels is usually fine, millions is not).

Bash Factorial, Fibonacci and a binary string

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

fib as written is exponentially slow; for large n store results in an array (memoization):

AWK
function fib(n) { return (n in memo) ? memo[n] : (memo[n] = n < 2 ? n : fib(n - 1) + fib(n - 2)) }

A recursive example on text

Recursion shines on nested data. This function prints a tree of indented names by treating each line child parent as an edge.

Bash Printing a tree recursively

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The %*s with an empty string is a neat way to print depth * 2 spaces.

Organising code: -f and reusable libraries

Long programs belong in files. awk -f prog.awk data.txt loads the program, and you can pass several -f options: awk -f lib.awk -f main.awk data. That lets you keep a library of helpers (trim, join, max) in one file and reuse it everywhere. A first line of #!/usr/bin/awk -f makes the file directly executable on Unix.

Bash A library file plus a main program

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Common mistakes

  • A space between the function name and ( in a call.
  • Forgetting to declare locals, so a loop counter clashes with the caller's.
  • Expecting an array to be copied when passed: it is shared.
  • Returning an array: not possible; fill an array parameter instead.
  • Recursion without a base case.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How do you create a local variable in an AWK function?

    AWK has no local keyword: extra parameters are local to the call, and any other variable is global.
  2. 2. What happens when an array is passed to a function and modified there?

    Arrays are passed by reference; scalars are passed by value.
  3. 3. Which call is syntactically correct?

    In a function call there must be no space between the name and the parenthesis (a space means string concatenation with max).
  4. 4. How do you load a library of functions plus a main program?

    You may give -f several times; the files are concatenated into one program.

Summary

  • function name(params) { ... return v } defines a function; call it without a space before (.
  • Locals are extra parameters; all other variables are global.
  • Scalars pass by value, arrays by reference; pass the count explicitly.
  • Recursion works and each call has its own parameters and locals.
  • Use -f (repeatable) to keep reusable functions in library files.

Next: Pipelines with grep, sort, uniq and AWK