Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 36 ยท Keywords and Variables

Retrying, Expecting Errors and the Run Keyword Family

Use Wait Until Keyword Succeeds to retry, Run Keyword And Expect Error to test failures, Run Keyword And Return Status, Ignore Error and test timeouts.

Real systems are slow, flaky and sometimes supposed to fail. Robot's Run Keyword family are BuiltIn keywords that take another keyword as an argument and change how its failure is treated: retry it, ignore it, expect it, or turn it into a boolean. Together with timeouts they cover most error-handling needs without writing code.

Retrying with Wait Until Keyword Succeeds

ROBOTFRAMEWORK
*** Test Cases ***
Wait for the service
    Wait Until Keyword Succeeds    30 seconds    2 seconds    Service Should Be Ready

Retry a fixed number of times
    Wait Until Keyword Succeeds    5x    1 second    Page Should Contain    Welcome

The first argument is the total time or the number of attempts (5x), the second is the pause between attempts, then comes the keyword and its arguments. The keyword is retried until it passes. If it never does, the last error is reported with a note about the retries.

Use it for state that changes by itself (a job finishing, an element appearing). Never use Sleep with a long fixed pause as a substitute: it wastes time when the system is fast and still fails when it is slow.

Ignoring a failure

Run Keyword And Ignore Error runs a keyword and returns two values: the status (PASS or FAIL) and the return value or error message.

ROBOTFRAMEWORK
*** Test Cases ***
Optional banner
    ${status}    ${message}=    Run Keyword And Ignore Error    Click Element    id=cookie-banner
    Log    Banner click: ${status}
    IF    '${status}' == 'FAIL'    Log    No banner to dismiss: ${message}

Run Keyword And Return Status returns only True or False:

ROBOTFRAMEWORK
${visible}=    Run Keyword And Return Status    Element Should Be Visible    id=menu

Expecting a failure

Negative tests prove that bad input is rejected. Run Keyword And Expect Error passes when the keyword fails with a matching message, and fails if the keyword passes or fails differently:

ROBOTFRAMEWORK
*** Test Cases ***
Division by zero is rejected
    Run Keyword And Expect Error    *division*zero*    Evaluate    1 / 0

Exact message
    Run Keyword And Expect Error    EQUALS:Invalid age: -5    Set Age    -5

Pattern message
    Run Keyword And Expect Error    REGEXP:Invalid age: -?\\d+    Set Age    abc

The message pattern is a glob by default. Prefix it with GLOB:, EQUALS:, STARTS: or REGEXP: to choose the matching style. In Robot test data a backslash must be doubled, so \\d means the regular expression \d. For the most readable negative tests, TRY / EXCEPT (see Control Flow) is often the better choice, because it can run assertions on the error message afterwards.

Continuing after a failure

Run Keyword And Continue On Failure records the failure but keeps the test running. The test still ends as FAIL. Use it when you want to collect several independent assertion failures from one page or one response.

ROBOTFRAMEWORK
*** Keywords ***
Verify Profile Page
    [Arguments]    ${name}    ${email}
    Run Keyword And Continue On Failure    Element Text Should Be    id=name    ${name}
    Run Keyword And Continue On Failure    Element Text Should Be    id=email    ${email}

Or use the tag robot:continue-on-failure on a test or keyword so every failing step in it continues.

Running keywords chosen at run time

Keyword Use
Run Keyword Run a keyword whose name is in a variable
Run Keywords Several keywords in one place, separated by AND (handy for setups and teardowns)
Run Keyword If The older conditional form; prefer IF
Run Keyword And Return Run a keyword and return its value immediately
Run Keyword If Test Failed / Passed Conditional teardown steps
ROBOTFRAMEWORK
*** Settings ***
Test Teardown    Run Keywords    Take Screenshot    AND    Close Browser

*** Test Cases ***
Dynamic choice
    ${action}=    Set Variable    Log
    Run Keyword    ${action}    chosen at run time

Timeouts

A test or keyword that hangs would block the whole run. Timeouts stop it:

ROBOTFRAMEWORK
*** Settings ***
Test Timeout    2 minutes

*** Test Cases ***
Quick check
    [Timeout]    10 seconds
    Fetch Report

*** Keywords ***
Fetch Report
    [Timeout]    5 seconds
    Sleep    1 second

A timeout can be given in seconds or as a time string such as 1 minute 30 seconds. A keyword timeout limits one keyword; a test timeout limits the whole test. When a timeout expires the running keyword is stopped and the test fails with "Test timeout 10 seconds exceeded."

Stand-in 1: a retry loop

The blocks below are not Robot Framework. The first mimics Wait Until Keyword Succeeds: it retries a function until it passes, limited by attempts, and reports the last error if all attempts fail.

Python Stand-in: Wait Until Keyword Succeeds

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stand-in 2: expecting an error

Run Keyword And Expect Error has three outcomes. The stand-in implements them with the pattern prefixes from this lesson.

Python Stand-in: Run Keyword And Expect Error

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Choosing the right tool

Situation Use
State changes by itself soon Wait Until Keyword Succeeds
Optional UI element Run Keyword And Ignore Error or Return Status
Bad input must be rejected Run Keyword And Expect Error or TRY / EXCEPT
Several independent checks Run Keyword And Continue On Failure
Could hang Timeouts

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does Wait Until Keyword Succeeds 30 seconds 2 seconds Check Ready do?

    The first value is the total retry time, the second the pause between attempts.
  2. 2. Run Keyword And Expect Error is used on a keyword that passes. What is the result?

    The test passes only if the keyword fails with a matching error.
  3. 3. What does Run Keyword And Return Status return?

    It converts pass or fail into a boolean and never fails itself.
  4. 4. Why is Sleep 10 seconds a poor way to wait for a job?

    Retrying until a condition holds is faster when the system is fast and more robust when it is slow.

Summary

  • Wait Until Keyword Succeeds retries by time or by attempts (5x).
  • Run Keyword And Ignore Error and Return Status convert failures into values.
  • Run Keyword And Expect Error proves failures, with GLOB:, EQUALS:, STARTS: and REGEXP: patterns.
  • Run Keyword And Continue On Failure and robot:continue-on-failure collect several failures.
  • Use Run Keywords ... AND ... for combined setups and teardowns, and add timeouts to stop hangs.

Next: Standard Libraries Overview