Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 43 ยท Regular Expressions

Substitution and Transliteration

Rewrite text in Perl with s///, the /g /e /r modifiers, tr/// character mapping and counting, split and join with regex, plus real-world cleanups.

Matching tells you what is in a string. Substitution changes it. Perl's s/// operator finds text with a pattern and replaces it, and its simpler cousin tr/// swaps or counts individual characters. Together with split and join, they handle most text cleanup you will ever need: trimming, reformatting, slugs, number formatting and templates.

This lesson assumes you know patterns and captures from Captures and Modifiers.

Perl is not installed on this site, so the Perl code is read-only and its output is shown below each snippet. The interactive examples are Python stand-ins (re.sub, str.translate, re.split) that do the same jobs. They are close cousins, not copies, and the differences are noted.

The s/// operator

The shape is s/pattern/replacement/modifiers. By default it works on $_; bind it to a variable with =~. It changes the variable in place, so the variable must be one you can assign to.

PERL
use v5.36;

my $text = "cat and cat";
$text =~ s/cat/dog/;      # first match only
say $text;                # dog and cat

$text =~ s/cat/bird/g;    # /g: every match
say $text;                # dog and bird

s/// returns the number of replacements it made (or an empty string, which is false, when nothing matched). That makes counting easy:

PERL
my $note = "cat and cat";
my $n = ($note =~ s/cat/dog/g);
say $n;                   # 2

The replacement part behaves like a double-quoted string, so you can use variables and the capture variables $1, $2:

PERL
my $name = "Lovelace, Ada";
$name =~ s/^(\w+), (\w+)$/$2 $1/;
say $name;                # Ada Lovelace

Inside the replacement, escapes \U (uppercase), \L (lowercase), \u (uppercase next letter) and \E (end) are available. \u\L$1 means "lowercase the whole word, then capitalize its first letter":

PERL
my $title = "tHE qUICK fox";
$title =~ s/(\w+)/\u\L$1/g;
say $title;               # The Quick Fox

When the pattern contains slashes, pick other delimiters so you do not need backslashes. Any non-space punctuation works, and bracket pairs may be mixed:

PERL
my $path = "/usr/local/bin";
$path =~ s{/usr/local}{/opt};
say $path;                # /opt/bin

/r returns a new string

Normally s/// modifies its target, which is awkward when you want to keep the original or work on a function result. The /r modifier (non-destructive) leaves the original alone and returns the new string. It also lets you chain steps.

PERL
my $original = "  hello   world  ";
my $clean = $original =~ s/^\s+|\s+$//gr =~ s/\s+/ /gr;
say "[$clean]";           # [hello world]
say "[$original]";        # [  hello   world  ]

Before /r existed, programmers wrote (my $copy = $original) =~ s/.../.../;, and you will still see that idiom in older code.

/e evaluates code

With /e, the replacement is not a string but Perl code, and its value is inserted. This is how you compute replacements:

PERL
my $prices = "apple 3, pear 5";
$prices =~ s/(\d+)/$1 * 2/ge;
say $prices;              # apple 6, pear 10

A common use is a tiny template engine: look each placeholder up in a hash.

PERL
my %v = (name => "Ana", n => 3);
my $tpl = "Hi {name}, you have {n} new {thing}.";
$tpl =~ s{\{(\w+)\}}{ $v{$1} // "?" }ge;
say $tpl;                 # Hi Ana, you have 3 new ?.

/ee evaluates the replacement and then evaluates the result as Perl code. Never use it on text that came from users, a file or the network: it runs whatever that text contains. Plain /e is safe, because it only runs code you wrote in the program.

s/// at a glance

Find A regex pattern, with captures if needed.
Replace A string with $1, or code with /e.
/g Replace every match, not just the first.
/r Return a new string, keep the original.

The return value is the number of replacements, or false for none.

1 / 2
Python Substitution with re.sub (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Transliteration with tr///

tr/search/replace/ maps characters from the first list to the second, one to one. It does not use regular expressions: ranges like a-z are allowed, but there are no quantifiers, no classes and no variables inside it. In exchange it is very fast.

PERL
my $word = "perl";
(my $shout = $word) =~ tr/a-z/A-Z/;
say $shout;               # PERL

y/// is a synonym for tr///. The classic example is ROT13, which moves each letter 13 places:

PERL
my $secret = "Hello" =~ tr/A-Za-z/N-ZA-Mn-za-m/r;
say $secret;              # Uryyb

Counting characters

The return value of tr/// is the number of characters matched. If the replacement list is empty, the string is not changed, which makes tr the quickest way to count characters:

PERL
my $dna = "AACGTTTA";
my $gc  = ($dna =~ tr/GC//);   # count G and C
my $as  = ($dna =~ tr/A//);    # count A
say "$gc $as";                 # 2 3

my $rev = reverse $dna;
my $rc  = $rev =~ tr/ACGT/TGCA/r;   # reverse complement
say $rc;                       # TAAACGTT

The modifiers /d, /c, /s and /r

Modifier Meaning
/d Delete characters that have no counterpart in the replacement list
/c Complement: use all characters not in the search list
/s Squeeze runs of the same translated character into one
/r Return the new string instead of changing the original
PERL
say "a1-b2 c3" =~ tr/a-zA-Z//cdr;     # abc   (delete non-letters)
say "aaabbb  cc" =~ tr/a-z//sr;       # ab  c (squeeze letter runs)
Python Transliteration (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

split and join

split is the opposite of matching: the pattern describes the separators, and you get the pieces between them. join glues pieces back together with a separator.

PERL
my @parts = split /,\s*/, "a, b,c,,d";
say scalar @parts;               # 5
say join "|", @parts;            # a|b|c||d

A few rules are worth knowing:

  • split ' ', $str (a single space string, not a regex) is special: it skips leading whitespace and splits on runs of whitespace. This is what you want for words.
  • split //, $str splits between every character.
  • A third argument limits the number of pieces: split /=/, "key=a=b", 2 gives ("key", "a=b").
  • Trailing empty fields are removed unless you pass a negative limit: split /,/, "a,b,,," returns ("a", "b"), but split /,/, "a,b,,,", -1 returns five items.
  • If the pattern contains capture groups, the separators are included in the result: split /(,)/, "a,b" gives ("a", ",", "b").
PERL
say join "|", split ' ', "  many   spaces here ";   # many|spaces|here
say join "|", split //, "abc";                      # a|b|c
my @short = split /,/, "a,b,,,";
my @full  = split /,/, "a,b,,,", -1;
say scalar(@short), " ", scalar(@full);            # 2 5

Real-world cleanups

Substitution shines in small, practical jobs. These are the patterns you will reuse again and again.

Trim and collapse whitespace:

PERL
$s =~ s/^\s+|\s+$//g;     # trim both ends
$s =~ s/\s+/ /g;          # collapse runs of whitespace

Make a URL slug (lowercase, replace runs of non-alphanumerics with a hyphen, trim hyphens):

PERL
my $slug = lc("Perl: Hello, World!") =~ s/[^a-z0-9]+/-/gr =~ s/^-|-$//gr;
say $slug;                # perl-hello-world

Add thousands separators:

PERL
my $n = 1234567;
1 while $n =~ s/^(\d+)(\d{3})/$1,$2/;
say $n;                   # 1,234,567

That loop keeps inserting a comma before the last three digits of the leading digit run until the pattern no longer matches. A lookaround version does it in one pass: s/(?<=\d)(?=(?:\d{3})+$)/,/g (see Advanced Regex).

Strip a file extension and rename:

PERL
(my $base = "photo.final.jpg") =~ s/\.[^.]+$//;
say $base;                # photo.final
Python split, join and cleanups (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does s/cat/dog/g return when it replaced two matches?

    s/// returns the number of substitutions it made, or a false empty string when none happened. The /r modifier is what makes it return the new string.
  2. 2. Which modifier makes s/// return the changed string and leave the original variable alone?

    /r is the non-destructive modifier. /g replaces every match, /e evaluates the replacement as code, and /x allows whitespace and comments in the pattern.
  3. 3. What does my $count = ($text =~ tr/a-z//); do?

    With an empty replacement list and no /d, tr leaves the string alone and returns the number of matched characters, which makes it a fast counter.
  4. 4. What does split /,/, "a,b,,," return?

    By default split drops trailing empty fields. Passing a negative limit, as in split /,/, $str, -1, keeps them.
  5. 5. Why is s/(\w+)/$1 * 2/e different from the same pattern without /e?

    /e treats the replacement as an expression to evaluate. Without it, the text $1 * 2 would be inserted as a string.

Summary

  • s/pattern/replacement/ changes a string in place; /g replaces all matches and the return value is the count.
  • /r returns a new string and keeps the original, which allows chaining.
  • /e runs the replacement as code. It is perfect for calculations and templates; never use /ee on untrusted input.
  • tr/// maps characters one to one, counts when the replacement is empty, and supports /d, /c, /s and /r. It does not use regex syntax.
  • split takes a pattern for the separators; trailing empty fields vanish unless you pass a negative limit. join is its inverse.
  • Trim, slug, thousands separators and extension stripping are one-line substitutions.

Next, look at the powerful regex features that go beyond the basics in Advanced Regex.