Skip to content
Tutorials Center
My learning

Topics

Lesson 9 of 38 ยท sed Basics

sed Capture Groups and Backreferences

Rearrange text with sed: capture groups, \1 to \9 in the replacement, backreferences inside patterns to find repeats, and GNU case conversion with \U, \L and \u.

Up to now you replaced a match with fixed text or the whole match (&). Real editing jobs need more: swap two words, reformat a date, move an extension, double a letter. For these you capture parts of the match in groups and reuse them in the replacement. This is the most powerful feature of s, and the one beginners find hardest. With a few worked examples it becomes routine.

Groups and numbered references

In ERE (sed -E) parentheses form a group; in BRE write \( \). Groups are numbered by the position of their opening parenthesis, left to right, from 1 to 9. In the replacement, \1 is the text of the first group, \2 of the second, and so on. \0 or & is the whole match.

TEXT
sed -E 's/(first) (second)/\2 \1/'
             \1      \2
Bash Swapping two words

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The third command is the same as the second in BRE: the parentheses need backslashes. In ERE you must escape them to match literal parentheses. Compare how much easier ERE is to read.

Reformatting dates

A group captures whatever it matched. Here three groups hold year, month and day, and the replacement reorders them.

Bash ISO date to European date

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

In the second command & is the whole match and \/ is an escaped slash, since the slash is also the delimiter. Using | as delimiter avoids that: s|...|& -> \3/\2/\1|.

Groups that do not take part

A group inside an optional or alternative part may match nothing; then its reference is an empty string. That is useful for optional prefixes:

Bash Optional parts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

(https?://)? is an optional group; the host is always \2. Without matching the scheme, \1 would be empty and no error occurs.

Backreferences inside the pattern

You can use \1 in the pattern too, to find repeated text. This works even in BRE and is a regex feature that goes beyond "regular" languages.

Bash Find doubled words and double letters

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

\b (GNU) is a word boundary. ([a-z]+) \1 means "a word, a space, and the same word again". ([a-z])\1 finds any letter immediately repeated. Backreferences inside patterns are slow on huge inputs but fine for everyday work.

Case conversion (GNU)

GNU sed has escape sequences in the replacement for changing case:

Escape Effect
\U upper-case everything until \E or the end
\L lower-case everything until \E or the end
\u upper-case the next character
\l lower-case the next character
\E stop \U or \L
Bash Capitalizing with \\u, \\U and \\L

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The first capitalizes each word, the fourth converts snake_case_name into snakeCaseName (camelCase), and the last normalizes first LAST into First Last by lowering the group and capitalizing its first letter (write \L\u, in that order). These are GNU extensions: on macOS use awk with toupper/substr or perl.

How many groups can I use?

Nine, \1 to \9. If you need more, simplify with non-capturing logic or move to AWK, which can split fields and handle any number of them. Also note that nested groups are numbered by their opening parenthesis: in ((a)(b)) the outer group is \1, (a) is \2 and (b) is \3.

Bash Nested groups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

A mini reference: swapping, wrapping, trimming

TEXT
sed -E 's/^[[:space:]]+|[[:space:]]+$//g'      trim both ends
sed -E 's/(.*)/(\1)/'                          wrap each line in parentheses
sed -E 's/^(.*)\.(txt)$/\1.md/'                change an extension
sed -E 's/([^,]*),([^,]*)/\2,\1/'              swap the first two CSV fields

Test each of them before trusting it; CSV with quoted commas, for example, defeats the last pattern (see CSV tasks later).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does echo 'a b' | sed -E 's/(a) (b)/\2 \1/' print?

    \1 is the first group (a) and \2 the second (b); the replacement swaps them.
  2. 2. How do you write a group in BRE (plain sed)?

    In BRE, groups use backslashed parentheses. With -E they are plain parentheses.
  3. 3. What does sed -nE '/([a-z])\1/p' find?

    \1 inside the pattern repeats whatever the group matched.
  4. 4. In ((a)(b)), which reference is the group (b)?

    Groups are numbered by opening parenthesis: the outer one is 1, (a) is 2 and (b) is 3.
  5. 5. Which GNU escape upper-cases the whole next group until the end of the replacement?

    \U switches to upper case until \E; \u only affects the next character.

Summary

  • Parentheses capture parts of a match (BRE: \( \), ERE: ( )), numbered 1-9 by their opening parenthesis.
  • \1...\9 in the replacement reinsert them; & is the whole match.
  • \1 inside the pattern matches a repeat of the same text.
  • GNU \U \L \u \l \E convert case.

Next: Append, Insert, Change and Transliterate