Skip to content
Tutorials Center
My learning

Topics

Lesson 1 of 38 ยท Getting Started

What Are AWK and sed?

Meet sed and AWK, the two classic Unix text tools: what each one does, how they differ, when to pick which, and how to run every example in this course.

Every day, programs produce text: log files, CSV exports, configuration files, command output. Sooner or later you need to change that text, pull a few columns out of it, or count something in it. You could open an editor, or write a Python script. But for a huge number of jobs there is a faster way: two small tools that have shipped with Unix for fifty years, sed and AWK.

In this lesson you will learn what each tool is for, how they differ, and how to decide between them. You will also run your first commands. The whole course runs in your browser, so you do not need to install anything.

sed: the stream editor

sed stands for stream editor. It reads text one line at a time, applies the edit commands you give it, and writes the result. The classic job is search and replace:

Bash Your first sed command

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

s/cats/dogs/ means substitute cats by dogs. Without the trailing g only the first match on each line changes; with g (global) every match does. You will study this command in sed Substitute.

sed does more than substitution. It can delete lines, print only some lines, insert text before a line, join lines, and even run small loops. But its mental model stays simple: one line in, edit, one line out.

AWK: a tiny language for columns

AWK (named after its authors Aho, Weinberger and Kernighan) is a complete little programming language built around one idea: a program is a list of pattern { action } rules. AWK reads each line, splits it into fields ($1, $2, ...), and runs every action whose pattern matches.

Bash Your first AWK program

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The first program prints the first field of every line. The second only acts on lines whose second field is at least 80. The third adds up the second field and, after the last line (END), prints the average. AWK has variables, arithmetic, strings, arrays, functions and loops, so it can do far more than sed when the data is structured.

Which one should I use?

A rough rule: sed edits text, AWK analyzes it.

You want to... Reach for
Replace one string with another sed
Delete or print lines by number or pattern sed (or AWK)
Rewrite part of every line using a regex sed
Pick columns, reorder or reformat them AWK
Sum, count, average, group by a key AWK
Apply logic ("if the third field is larger than ...") AWK
Edit many files in place sed -i

They are not rivals. A real command line often chains them with other tools: sed cleans the input, awk summarises it, sort orders it. That is the topic of the pipelines chapter.

flowchart LR
    A["input text<br/>(file or pipe)"] --> B["sed<br/>edit lines"]
    B --> C["awk<br/>fields and totals"]
    C --> D["sort / uniq / head"]
    D --> E["result"]

A typical pipeline: each tool does the one thing it is best at.

A very short history

sed was written by Lee McMahon at Bell Labs around 1973-74. AWK followed in 1977, written by Alfred Aho, Peter Weinberger and Brian Kernighan, and was greatly extended in 1985 (user functions, multiple input streams). The POSIX standard describes both, which is why scripts written for them keep working on Linux, macOS, BSD and even Windows (through Git Bash or WSL) decades later.

Several AWK programs exist: the original one-true-awk (used on macOS and BSD), gawk (GNU AWK, with many extras) and mawk (small and fast, the default on Debian and Ubuntu). This course teaches the POSIX core that all of them share, and points out GNU-only extras clearly. The examples on this site run on mawk and GNU sed, so every runnable block sticks to portable features.

How to read the examples

  • Examples inside a <TryIt> are runnable: press Run and the real tools execute on our server. Each example creates its own sample files first, because every run starts in an empty directory.
  • Plain code blocks are read-only, for things that cannot or should not run here (for example editing a real file in your home directory).
  • A line like $ sed -n '2p' file shows what you would type in a terminal. The $ is the prompt, not part of the command.

If you want a refresher on the shell itself, see Linux Command Line and the Linux lessons on sed and awk. This course goes much deeper.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does sed do with its input?

    sed is a stream editor: it processes the input line by line, runs your commands on each line and prints the outcome, so it works on files of any size.
  2. 2. In awk '$2 >= 80 { print $1 }', what is the part before the braces?

    AWK programs are made of pattern { action } rules. The pattern decides which lines the action runs on.
  3. 3. Which task suits AWK better than sed?

    AWK splits lines into fields and has variables and arithmetic, so totals and column work are its strength. sed is best for replacing and deleting text.
  4. 4. What does the g flag do in s/a/b/g?

    g means global: all matches on each line are replaced.

Summary

  • sed is a stream editor: one line in, edit, one line out. It excels at search and replace, deleting and printing lines.
  • AWK is a small language of pattern { action } rules over fields. It excels at columns, totals and logic.
  • They combine well in pipelines with sort, uniq and friends.
  • This course teaches portable POSIX behaviour and flags GNU-only extras.

Next: How Text Streams Work