Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 36 ยท Patterns and Extensions

Listeners and pre-run modifiers

Hook into a Robot Framework run with listeners and change a suite before it runs with pre-run modifiers: events, the listener API v3, examples and stand-ins.

Sometimes you want to react to what Robot does without touching every test: send a chat message when a test fails, record timings, take a screenshot on every failure, or skip tests that belong to a known bug. Robot Framework has two extension points for this: listeners, which receive events as a run progresses, and pre-run modifiers, which change the suite structure before it runs.

Both are written in Python and passed on the command line. The real run cannot happen on this site, so the Robot-side code is read-only; the runnable examples are Python stand-ins that simulate the event flow, so you can see exactly when each hook fires.

Listeners

A listener is a Python class (or module) whose methods are called at events: a suite starts, a test ends, a keyword fails. You attach it with --listener:

Terminal
robot --listener MyListener.py tests/
robot --listener MyListener.py:slack_url=https://hooks.example.test/x tests/

The text after the file name, separated by colons, becomes constructor arguments. There are two APIs. Listener API 3 (ROBOT_LISTENER_API_VERSION = 3) passes rich objects you can even modify; API 2 passes names and dictionaries. New code should use version 3.

Python
class FailureReporter:
    ROBOT_LISTENER_API_VERSION = 3

    def __init__(self):
        self.failed = []

    def start_suite(self, data, result):
        print(f"Suite started: {data.name}")

    def end_test(self, data, result):
        if result.status == "FAIL":
            self.failed.append((data.name, result.message))

    def close(self):
        for name, message in self.failed:
            print(f"FAILED: {name}: {message}")

The common hooks:

Method Called when
start_suite, end_suite a suite starts or ends
start_test, end_test a test starts or ends
start_keyword, end_keyword a keyword starts or ends
log_message, message a message is logged
output_file, log_file, report_file a result file is written
close the run is over

In end_test you may change result.status or result.message; that is how a listener can turn a failure into a skip for a known problem. Use that power sparingly: hidden status changes confuse people.

When listener hooks fire

start_suite Before the first test.
start_test Before each test.
end_test Status is known.
close The run is done.

Keyword hooks fire between start_test and end_test.

1 / 2

Library listeners

A library can also act as its own listener by setting the attribute inside the class. This is how libraries such as SeleniumLibrary-based helpers capture a screenshot after every failed keyword: they listen for end_keyword. Because it lives in the library, nobody has to remember to add a teardown.

Python
class ScreenshotOnFail:
    ROBOT_LIBRARY_SCOPE = "GLOBAL"
    ROBOT_LISTENER_API_VERSION = 3

    def __init__(self):
        self.ROBOT_LIBRARY_LISTENER = self

    def end_keyword(self, data, result):
        if result.status == "FAIL":
            print(f"(would capture a screenshot after: {data.name})")

Simulating the event flow

The first stand-in does not use Robot but plays the same role: a tiny "runner" calls listener methods at the right times, and a listener collects failures and durations. Running it shows the order of events.

Python Stand-in: a listener receives events

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pre-run modifiers

A pre-run modifier sees the whole suite tree after parsing and before execution, and may change it: remove tests, add tags, change names. You pass it with --prerunmodifier. It is built on the visitor pattern: the class gets a method call for every suite and test.

Python
from robot.api import SuiteVisitor

class SkipKnownBugs(SuiteVisitor):
    def __init__(self, ids="BUG-101,BUG-202"):
        self.bugs = set(ids.split(","))

    def start_suite(self, suite):
        suite.tests = [
            t for t in suite.tests
            if not self.bugs & {tag.upper() for tag in t.tags}
        ]

Run it like robot --prerunmodifier SkipKnownBugs:BUG-101 tests/. Tests tagged with a known bug id are removed from the run, so the build is not red for a defect that is already tracked. Be careful: a hidden removal is easy to forget. Log what you drop, or better, tag the tests robot:skip so the report shows them as skipped.

Other uses: add a slow tag to tests whose name contains "export", repeat each test with different browsers, or shuffle order deterministically.

Simulating a modifier

This stand-in builds a tiny suite structure and a "visitor" that changes it before the run, which is exactly the shape of a pre-run modifier.

Python Stand-in: a pre-run modifier

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pre-rebot modifiers and result visitors

--prerebotmodifier works the same way but acts on results after the run, before reports are created. It can add a message, change statuses or remove noisy details. ResultVisitor (from robot.api) is the base class. Teams use it, for example, to rename tests to include a ticket number or to mark known failures.

Be careful with extensions

  • A listener that raises an exception can hide or break results: guard network calls with try/except and timeouts.
  • Do not slow down the run: start_keyword fires thousands of times.
  • Keep listeners observers. Changing results silently makes reports untrustworthy.
  • Document extensions in the repository README so a new colleague knows why a test vanished.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is a listener for?

    Listeners are notified of events such as end_test, so they can report, log or react.
  2. 2. Which option attaches a pre-run modifier?

    --prerunmodifier loads a SuiteVisitor class that may change the suite before running.
  3. 3. Which listener API should new code use?

    API 3 passes rich objects and is the recommended choice.
  4. 4. Why be careful when a modifier removes tests tagged with a bug id?

    Silent removal is easy to forget; marking tests skipped keeps them visible in the report.

Summary

  • Listeners receive run events; use API 3 and attach with --listener.
  • A library can be its own listener, for example to take screenshots on failure.
  • Pre-run modifiers (SuiteVisitor) reshape the suite before running; pre-rebot modifiers change results.
  • Keep extensions fast, safe and documented, and prefer observing over rewriting results.

Next: BDD and Gherkin style