Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 36 ยท Reporting and CI

Reading results, log.html and output.xml

Understand the three files every Robot Framework run writes: output.xml, log.html and report.html. Log levels, rebot, screenshots and a runnable output parser.

A test that fails is only useful if you can find out why quickly. Robot Framework writes its results in three files after every run, and learning to read them is half of working with the tool. This lesson explains what each file contains, how log levels work, how to combine results with rebot, and how to attach screenshots and messages.

The real robot command cannot run on this site, so command lines and file contents are shown read-only. The runnable example is a Python stand-in that parses a small output file the way tools like rebot do.

The three files

Every run produces:

File What it is Who uses it
output.xml The raw machine-readable result of everything that ran Tools, CI, rebot
log.html The detailed, expandable log of every keyword and message Anyone debugging a failure
report.html A summary: pass/fail totals, statistics by tag and suite Managers, dashboards

output.xml is the source of truth. The other two are generated from it, which means you can rebuild them later (or merge several runs) without running the tests again.

One run, three files

output.xml Everything, as data.
log.html Every keyword, expandable.
report.html Totals by suite and tag.

The two HTML files are built from output.xml.

1 / 2

Reading log.html

log.html shows a tree: suite, test, keyword, nested keyword, and the messages each wrote. Failed keywords are red and auto-expanded, so the first thing you see is the place that went wrong. Click the arrows to open passed keywords. For a failing assertion the message looks like Page should have contained text 'Welcome' but did not, plus any screenshot you took.

Log levels

Messages have levels: TRACE, DEBUG, INFO, WARN, ERROR (and FAIL). By default the log keeps INFO and above. Raise or lower the threshold with --loglevel:

Terminal
robot --loglevel DEBUG tests/
robot --loglevel INFO:DEBUG tests/

The second form says "record INFO, but keep DEBUG messages in the output so they appear only when a test fails"; a quick way to get detail on failures without bloating every log. In your own keywords write messages with the built-in Log keyword:

TEXT
Log    Starting checkout for ${user}
Log    Cart: ${cart}    level=DEBUG
Log    Slow response, ${seconds}s    level=WARN

Warnings show up in a special block at the top of the report, so they are hard to miss. Log To Console prints to the terminal while the test runs; it is useful for watching progress but does not go into the log.

Screenshots and evidence

For web tests, take a screenshot when a test fails. A common teardown:

TEXT
*** Settings ***
Test Teardown    Run Keyword If Test Failed    Capture Page Screenshot

The image is saved next to the report and linked from the log at the failing step. For API tests, log the response body (Log ${response.text}) so you can see what the server returned.

Names, documentation and metadata

Results are easier to read when suites describe themselves. Use Documentation and Metadata in the settings:

TEXT
*** Settings ***
Documentation    Checkout tests for the shop.
Metadata         Version      2.4.1
Metadata         Environment  staging

Metadata appears at the top of the report, so six months later you still know which build and environment the numbers came from.

rebot: combine and rebuild

rebot is the tool that turns output.xml files into logs and reports. It is installed with Robot Framework:

Terminal
# regenerate the HTML after the run
rebot --name Shop output.xml

# merge two runs (for example Chrome and Firefox)
rebot --name Combined --output combined.xml chrome/output.xml firefox/output.xml

# merge a re-run of failed tests into the original (see the next lesson)
rebot --merge --output merged.xml first.xml rerun.xml

Useful options: --removekeywords PASSED shrinks huge outputs by dropping details of passed keywords; --flattenkeywords hides nested detail; --splitlog splits big logs into pieces so the browser stays fast.

A tiny output parser

The runnable stand-in below builds a miniature XML file shaped like a simplified output.xml (the real one has more attributes and nested keyword elements) and computes the totals that report.html shows. It uses Python's built-in XML parser.

Python Stand-in: summarize a result file

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because the result is data, you can build your own summaries, send them to chat, or chart pass rates over time.

Levels in code

This last stand-in shows how a log-level threshold decides what is kept, the same rule as --loglevel.

Python Stand-in: log level filter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Never log secrets. Use Input Password, or wrap sensitive steps with Set Log Level NONE and restore the old level afterward, so tokens do not appear in files that others will read.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which file is the raw, machine-readable result?

    output.xml holds everything; the HTML files are generated from it.
  2. 2. What does rebot do?

    rebot works on result files, so you can regenerate or combine results without re-running tests.
  3. 3. What is the default log level threshold?

    By default INFO and higher messages are kept.
  4. 4. Where does Log To Console write?

    It prints while the test runs, which is handy for progress but not recorded in the log.

Summary

  • Every run writes output.xml, log.html and report.html.
  • The log shows a tree of keywords; failures are expanded automatically.
  • --loglevel controls detail; write messages with Log and a level.
  • Capture screenshots or response bodies on failure.
  • rebot regenerates and merges results; keep secrets out of logs.

Next: Run options, tags and selecting tests