Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 38 ยท sed Scripting

sed Branching, Loops and Multi-line Editing

Write real sed programs: labels with b and t, loops that repeat a substitution, and multi-line editing with n, N, D and P, including joining lines and commas in numbers.

Everything so far has run top to bottom, once per line. Two features break that pattern and make sed a (small) programming language: branching, which jumps to a label so you can build loops and conditions, and the multi-line commands n, N, D, P, which let a cycle work with more than one line at a time. They are where sed gets powerful and where it gets cryptic. We will go slowly and use examples you can run.

Labels and b

A label is a name followed by a colon at the start of a line: :again. The command b label jumps to it. Without a label, b jumps to the end of the script (the cycle's end, where auto-print happens), skipping the remaining commands.

Bash b skips the rest of the script

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

/^skip/b reads "on lines starting with skip, branch to the end". That is the sed equivalent of if ... continue.

t: branch if a substitution succeeded

t label jumps to the label only if a substitution has succeeded since the last input line was read or since the last t. That is the key to loops: "keep substituting until there is nothing left to substitute".

TEXT
:a
s/pattern/replacement/
ta

Each time the s succeeds, ta jumps back to :a and tries again; when s fails, ta does not jump and the script continues. GNU sed also has T label, which jumps if no substitution succeeded, but it is not POSIX.

Example: put commas in numbers

Insert a comma before each group of three digits, working from the right. The loop applies the same substitution until no more digit groups remain:

Bash 1234567 becomes 1,234,567

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

On 1234567: first 567 at the end gets a comma before it (1234,567); the loop runs again and the 234, group gets one (1,234,567); no further match, so sed leaves the loop. The pattern ($|,) anchors the three digits to the end or to a following comma.

Example: squeeze repeated spaces without g

Loops can express what a single s cannot. A simple one: remove all doubled characters in a line one at a time.

Bash A loop that shrinks runs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The first command collapses aaabbbccc to abc: each pass removes one repeated character. The second is the simpler way to squeeze spaces; a loop is overkill when a flag does the job.

n and N: working with more than one line

  • n: print the pattern space (unless -n), then replace it with the next input line.
  • N: append a newline and the next input line to the pattern space.

If there is no next line, n ends the script (GNU prints the pattern space first), and N does the same (GNU sed prints the pattern space; POSIX says to quit without printing).

Bash N joins lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

$!ba is the key to join everything: "unless this is the last line, branch to a". Each pass appends another line with N. After the last line, the whole file sits in the pattern space and the final s///g converts all newlines at once.

In a multi-line pattern space, ^ and $ match the start and end of the whole space, not of each line (GNU has the M flag for the other behaviour). And . also matches the newline character. If you need "not a newline", write [^\n] (GNU) or [^ followed by a real newline in strictly portable code.

Example: join continuation lines

Lines ending in a backslash continue on the next line, as in shell scripts and Makefiles. Join them:

Bash Joining lines that end in a backslash

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

When the line ends with \, N pulls in the next line, s removes the backslash, the newline and the next line's leading blanks (the blank before the backslash stays as the separator), and ta repeats because the joined line may end with another backslash.

D and P: sliding windows

  • P: print the pattern space up to the first newline.
  • D: delete the pattern space up to the first newline and restart the cycle without reading new input if anything is left.

Together with N they let you keep a window of lines. The idiom $!N; ...; P; D is the sed way to look at two consecutive lines:

Bash Remove duplicate consecutive lines (uniq)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Walk through it: $!N appends the next line, so the pattern space is line1\nline2. The regex ^\(.*\)\n\1$ matches when both lines are identical; !P prints the first line only if they differ. D removes the first line and restarts with the second line still in place, so every line gets compared with its successor. This script is a classic.

Bash Print the last two lines (tail -2)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The window holds two lines; $!D drops the older one unless we are at the end, leaving the last two.

Putting a decision into a script

You can build if/else by combining t and b:

Bash if / else with t and b

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The first matching substitution jumps to done, skipping the fallback s/.*/other/. This is a classic lookup table pattern. Be careful: t tests the flag since the last input line or since the last t, so each t resets it.

Debugging tips

  • Print the pattern space with l (shows newlines as \n) while developing.
  • GNU sed has --debug, which prints the canonical form of the script and each step. Try it on a small input.
  • Keep scripts in files with comments (see Script files and recipes).
  • If you need more than a couple of branches, strongly consider AWK instead: it has if, while and variables.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does b without a label do?

    An empty branch target means the end of the script; the pattern space is still auto-printed.
  2. 2. When does t label jump?

    t tests the substitution flag, and resets it when it jumps.
  3. 3. What does sed ':a;N;$!ba;s/\n/,/g' do?

    The loop appends every line to the pattern space with N; the final substitution replaces all newlines.
  4. 4. What is the difference between D and d?

    D is the multi-line version: it removes the first line of the pattern space and loops again with the rest.

Summary

  • :label, b label and t label give sed jumps and loops; b alone goes to the end of the script.
  • n replaces the pattern space with the next line; N appends it with a newline.
  • P prints and D deletes up to the first newline, enabling sliding-window scripts.
  • $!ba after N joins a whole file; t loops until a substitution stops matching.
  • Use l or --debug to understand what a script does; switch to AWK when it grows too tricky.

Next: Script Files and sed Recipes