Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 43 ยท Tools and Quality

Testing with Test::More

Write Perl tests with Test::More: ok, is, is_deeply, plans, subtests, .t files, prove, mocking with local, a look at Test2, and coverage with Devel::Cover.

Perl has one of the oldest and best testing cultures in programming. Almost every module on CPAN ships with a t/ directory, and the tools to run those tests come with Perl itself. Writing a test means writing a small script that prints ok or not ok for every check; a runner reads that output and tells you whether the whole suite passed.

This lesson shows how to write tests with Test::More, how the output format works, how to organize .t files and run them with prove, how to replace a dependency while testing, what Test2 is, and how to measure which lines your tests actually run.

Perl is not installed on this site, so the Perl code below is read-only. The interactive examples are Python stand-ins: one builds a tiny test harness that prints the same TAP output so you can see how it works, and another shows temporary replacement of a function (mocking). The Perl output shown in the text comes from real runs of the code.

Your first test file

A test file ends in .t and is an ordinary Perl script. Suppose you have a small module Calc.pm with an add and an average function. Here is a test for it:

PERL
use v5.36;
use lib '.';
use Test::More;
use Calc qw(add average);

ok(defined &add, 'add is exported');
is(add(2, 3), 5, 'add adds');
is(average(2, 4, 9), 5, 'average of three numbers');
isnt(add(1, 1), 3, 'one plus one is not three');
like(average(1, 2), qr/^1\.5$/, 'fractions work');
is_deeply([ map { $_ * 2 } 1 .. 3 ], [2, 4, 6], 'doubling a list');

done_testing;

Running perl t/calc.t prints:

TEXT
ok 1 - add is exported
ok 2 - add adds
ok 3 - average of three numbers
ok 4 - one plus one is not three
ok 5 - fractions work
ok 6 - doubling a list
1..6

This format is called TAP, the Test Anything Protocol. Each line is one test with a number and the description you gave it. The last line, 1..6, is the plan: how many tests ran. A runner uses the plan to notice a test script that died halfway.

You can declare the plan in two ways:

  • use Test::More tests => 6; states the number up front. If the script stops early or runs extra tests, the run fails.
  • done_testing; at the end counts for you, which is easier while the file is still growing.

The core checks

ok Passes when the value is true.
is / isnt Compare as strings, with a clear diff.
like Match the value against a regex.
is_deeply Compare nested data structures.

Always give a test a short description.

1 / 2

The comparison functions

Prefer the most specific function, because it prints the most helpful failure message.

Function What it checks
ok($value, $name) The value is true
is($got, $expected, $name) Equal as strings (eq); undef is handled sensibly
isnt($got, $unexpected, $name) Not equal
like($got, qr/.../, $name) The string matches the pattern; unlike for the opposite
is_deeply($got, $expected, $name) Two data structures have the same shape and values
cmp_ok($got, '<', $limit, $name) Any comparison operator, with both values in the message
can_ok($class, @methods) / isa_ok($obj, $class) The class has the methods; the object is of the class
pass($name) / fail($name) Record a result you decided yourself

Compare what happens when a test fails. This script:

PERL
use Test::More tests => 2;
is(1 + 1, 3, 'math works');
is_deeply({ a => [1, 2] }, { a => [1, 3] }, 'nested data');

prints:

TEXT
1..2
not ok 1 - math works
#   Failed test 'math works'
#   at t/fail.t line 3.
#          got: '2'
#     expected: '3'
not ok 2 - nested data
#   Failed test 'nested data'
#   at t/fail.t line 4.
#     Structures begin differing at:
#          $got->{a}[1] = '2'
#     $expected->{a}[1] = '3'
# Looks like you failed 2 tests of 2.

The lines starting with # are diagnostics for humans. A plain ok($x == 3) would only tell you it failed; is shows both values and is_deeply points to the exact element that differs. Use diag("text") to print your own message and note("text") for information that only appears in verbose runs.

Remember that is compares with eq. For floating-point results, is(0.1 + 0.2, 0.3) fails; use cmp_ok(abs($got - $want), '<', 1e-9) instead.

To see how this output is produced, here is a small harness in Python that prints the same format. It is a stand-in to explain TAP, not a Perl feature:

Python A tiny TAP harness (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Subtests and skipping

A subtest groups related checks. It has its own plan, and the group counts as one test in the outer file:

PERL
subtest 'edge cases' => sub {
    plan tests => 2;
    is(add(0, 0), 0, 'zero');
    is(add(-1, 1), 0, 'negatives');
};

The output indents the inner results and then reports a single ok N - edge cases line for the whole group.

For tests that cannot run everywhere, use SKIP blocks and TODO blocks:

PERL
SKIP: {
    skip 'needs a Unix shell', 1 if $^O eq 'MSWin32';
    is(`echo hi`, "hi\n", 'shell works');
}

TODO: {
    local $TODO = 'rounding bug, fix planned';
    is(round_price(2.675), 2.68, 'rounds half up');
}

A skipped test is reported as passed with a note. A TODO test is expected to fail, so its failure does not break the suite, and an unexpected pass is flagged so you remember to remove the marker.

Organizing tests and running them with prove

By convention a project looks like this:

TEXT
MyProject/
  lib/MyProject/Calc.pm
  t/00-load.t
  t/calc.t
  t/report.t

Files in t/ run in alphabetical order, which is why numeric prefixes such as 00-load.t are popular for a basic "does it compile" test. The prove command runs them all:

Terminal
prove -l t/            # -l adds lib/ to @INC
prove -lv t/calc.t     # -v shows every ok line
prove -lr t/           # -r also searches subdirectories
prove -j4 -l t/        # run four files in parallel

A passing run looks like this (the timings on the last lines vary):

TEXT
t/00-load.t .. ok
t/calc.t ..... ok
t/report.t ... ok
All tests successful.
Files=3, Tests=14,  0 wallclock secs
Result: PASS

For a distribution with a Makefile.PL or Build.PL, make test or ./Build test calls the same machinery. Keep tests independent: no file should depend on another having run first, which also lets prove -j run them in parallel.

flowchart LR
    T["t/*.t files"] --> P["prove"]
    P --> H["TAP output<br/>ok / not ok"]
    H --> R["Summary: PASS or FAIL"]

prove runs every .t file, reads the TAP they print, and reports one result.

Replacing dependencies: mocking

Good tests should not depend on the clock, the network or a real database. Perl lets you replace any named subroutine temporarily by assigning to its glob with local. Here the code under test asks for the current hour:

PERL
package Clock;
sub now_hour { return (localtime)[2] }
sub greeting { return now_hour() < 12 ? 'Good morning' : 'Good afternoon' }

The test swaps now_hour for a fixed answer. local restores the original when the block ends:

PERL
{
    no warnings 'redefine';
    local *Clock::now_hour = sub { 9 };
    is(Clock::greeting(), 'Good morning', 'morning');
}
{
    no warnings 'redefine';
    local *Clock::now_hour = sub { 15 };
    is(Clock::greeting(), 'Good afternoon', 'afternoon');
}
TEXT
ok 1 - morning
ok 2 - afternoon
1..2

The CPAN module Test::MockModule wraps the same trick in a friendlier interface ($mock->mock(now_hour => sub { 9 })) and restores everything when the object goes out of scope. For HTTP and databases there are dedicated modules; the principle is always to substitute the outside world with something predictable.

The same idea in Python uses a context manager that puts the original back, shown here with plain assignment so you can see each step:

Python Replace a function temporarily (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Test2 and Test::More today

Since Perl 5.26, Test::More is built on Test2, a newer testing framework that handles nested results, threads and extensions more reliably. Your existing Test::More scripts keep working unchanged. If you want its richer style, the Test2::V0 bundle (a CPAN module) offers a single import with a smarter is that compares structures, like for partial matches, dies { ... } and warnings { ... } helpers, and mock support:

PERL
use Test2::V0;

is({ name => 'Ana', tags => [1, 2] }, { name => 'Ana', tags => [1, 2] }, 'same data');
like({ name => 'Ana', age => 30 }, { name => 'Ana' }, 'has the name');
ok(dies { die "boom\n" }, 'it dies');

done_testing;

You do not need to switch to write good tests; Test::More is stable, ubiquitous and enough for most projects.

Measuring coverage

A passing suite says nothing about code the tests never reached. Devel::Cover, from CPAN, records which statements, branches and conditions ran:

Terminal
cover -delete
HARNESS_PERL_SWITCHES=-MDevel::Cover prove -l t/
cover

The last command prints a table and writes an HTML report to cover_db/coverage.html. Treat the numbers as a map, not a target: 100% line coverage does not mean every behavior is checked, but a red line shows code with no test at all. Branch and condition coverage often reveal untested else paths that line coverage hides.

Habits worth keeping

  • Write the test first for each bug you fix, so it cannot return unnoticed.
  • Give every test a description, and make failures self-explanatory.
  • Test behavior (inputs and outputs) rather than internal details.
  • Keep tests fast and free of the network, the clock and shared files, or isolate those with mocks and File::Temp.
  • Run the suite before every commit, and in continuous integration.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does the line 1..8 at the end of a TAP test output mean?

    The plan tells the test runner how many tests to expect. If the script dies early, the count does not match and the run fails.
  2. 2. Which function is best for comparing two nested data structures?

    is_deeply walks both structures and, on failure, shows the first place where they differ.
  3. 3. What does prove -l t/ do?

    prove runs the test files, reads their TAP output and prints a summary. The -l flag puts lib/ on the module search path.
  4. 4. How do you replace a subroutine only for the duration of a block in a test?

    Assigning to the glob with local swaps the code and restores the original when the enclosing block ends. Add no warnings 'redefine' to silence the redefinition warning.
  5. 5. Why is is(0.1 + 0.2, 0.3) a poor test?

    Floating-point arithmetic is slightly inexact, so the values differ in tiny digits. Compare with a tolerance instead, for example with cmp_ok on the absolute difference.

Summary

  • A Perl test is a .t script that prints TAP: numbered ok and not ok lines plus a plan.
  • Use is, like and is_deeply for clear failure messages; end with done_testing or declare a plan up front.
  • subtest, SKIP and TODO group tests and handle platform differences.
  • prove -l t/ runs a whole suite; keep tests independent so they can run in parallel.
  • Replace outside dependencies with local *Package::name = sub {...} or Test::MockModule.
  • Test::More runs on Test2 today; Test2::V0 offers a richer style if you want it.
  • Devel::Cover shows which code the tests never touched.

Next, switch from careful testing to quick text surgery with Perl One-Liners.