Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 35 ยท Waiting and Synchronization

Why Tests Need Waits

Learn why a Selenium script races the page it controls, what NoSuchElement, StaleElementReference and ElementNotInteractable errors mean, and why sleep() is a poor fix.

A Selenium script and the web page it controls are two programs running at the same time. Your script sends a command as soon as it can. The page, meanwhile, is still downloading data, running JavaScript, drawing new elements and playing animations. Nothing forces the two to stay in step. When the script is faster than the page, it looks for a button that does not exist yet, clicks something that is still hidden, or holds on to an element that the page has already thrown away.

Most "my test is flaky" stories start here. In this lesson you will learn the three errors that a timing problem produces, why a fixed sleep() is a weak cure, and what the correct idea is: wait for a condition, not for a duration. The next lesson, Implicit, Explicit and Fluent Waits, shows the tools Selenium gives you for this.

The Selenium commands in this lesson are read-only, because a real browser and driver cannot run on this site. The runnable examples are stand-ins: small JavaScript programs where a fake page changes over time with setTimeout. They are not Selenium, but they reproduce the same race, so you can see what goes wrong and why.

Two programs, one page

When you call driver.get(url), Selenium waits until the browser reports that the page has finished loading (by default, the load event). That is the only automatic wait you get. Anything that happens after that moment is invisible to the driver: a request that fills a list, a menu that slides in, a spinner that disappears, a framework that re-renders a component.

Python
# Read-only: a script that races the page
driver.get("https://shop.example/products")
driver.find_element(By.CSS_SELECTOR, ".product-card").click()   # may fail: cards load via fetch()

If the product cards are inserted by JavaScript 400 ms after load, the find_element call runs when the cards are not in the page yet. Selenium does not know that cards are coming; it only knows what is there right now.

sequenceDiagram
  participant S as Test script
  participant B as Browser page
  S->>B: get(url)
  B-->>S: load event (page shell ready)
  S->>B: find .product-card
  B-->>S: NoSuchElementException
  Note over B: fetch() finishes, cards are drawn
  S->>B: find .product-card (too late)

The script asks for the element in the gap between the page load and the moment JavaScript draws it.

The errors a race produces

Different timing mistakes give different exceptions. Learning to read them tells you what to wait for.

Three timing errors

NoSuchElement The element is not in the page yet.
NotInteractable It exists, but it is hidden, covered or disabled.
StaleElement You hold a reference to an element the page removed.

All three often vanish if you wait for the right condition.

1 / 2
  • NoSuchElementException means the locator matched nothing at the moment of the search. The cause is either a wrong locator or an element that has not been created yet. Only a wait can tell the two apart: if it never shows up, the locator is probably wrong.
  • ElementNotInteractableException means the element exists but cannot receive the action. It may have display: none, zero size, be disabled, or be hidden behind an animation that has not finished. A related error, ElementClickInterceptedException, means another element (for example a cookie banner or a loading overlay) sits on top of the one you tried to click.
  • StaleElementReferenceException means the WebElement object you hold points to a node that is no longer attached to the page. This happens when the page re-renders a list, replaces a section after an Ajax call, or navigates. The object is not "broken"; it just refers to a node that is gone, and the fix is to find the element again.

A race you can watch

The first stand-in has a fake page that "loads" in two steps: the shell exists at time zero, and the product card appears after 300 ms, as if a fetch() had finished. Two scripts run side by side. The impatient one looks immediately, the patient one looks once every 100 ms until the card appears.

JS Impatient vs patient script

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The impatient script fails on its first look, with the same kind of message Selenium prints. The patient script keeps trying and succeeds after about 300 ms. It did not need a bigger magic number. It needed a loop with a goal. This is the core idea behind every Selenium wait.

Stale references

Stale elements surprise beginners because the code looks correct. You find a row, the page refreshes the table, and when you click the row you found earlier, Selenium throws. The stand-in below models the page as a list of nodes with an attached flag. Re-rendering creates new nodes and detaches the old ones.

JS A reference that goes stale

Press Run (or Ctrl + Enter) to see the console output here.

No output.

After the refresh, the rows look identical on screen, but they are different nodes with new ids. The saved reference points at the old, detached row. The cure is not "wait longer": the cure is to look the element up again when you need it, and to wait until the refresh has finished before you do.

Avoid storing WebElement objects in long-lived variables or class fields. Store the locator instead, and call find_element right before each action. Page objects, which you will meet in The Page Object Model, are built around this habit.

Why sleep() is not the answer

The first fix everybody tries is time.sleep(5) (or Thread.sleep(5000) in Java). It "works", so it spreads through the suite. It has three problems.

  1. It is too slow when the page is fast. If the card appears after 300 ms, you still wait 5,000 ms. With 200 tests and ten sleeps each, you add more than an hour of pure idling to the run.
  2. It is too short when the page is slow. A busy CI machine or a slow network can take longer than your guess. The test then fails at random, and nobody can say why.
  3. It hides the real condition. A reader sees sleep(5) and learns nothing about what the test is waiting for.

The next stand-in counts the cost. The page takes a random time to respond, but the random numbers come from a fixed seed, so you get the same numbers every run. It compares a 2-second sleep with a poll that stops as soon as the page is ready.

JS The price of sleep

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The sleep is both slower overall and wrong about one run in five, because some pages take longer than two seconds. Polling waits only as long as needed and succeeds whenever the page becomes ready inside its time limit. A longer sleep would fix the failures but make the waste even bigger.

The rule to remember

Wait for something you can observe. Before an action, ask what has to be true for it to work: the element exists, it is visible, it is enabled, the spinner is gone, the text has changed, the URL contains /cart. Then wait for exactly that, with a time limit so that a real bug still fails the test in a reasonable time. Selenium's wait classes do this for you, and the next lesson shows how to use them.

A sleep has one legitimate use: tiny, short pauses in demos or when you are debugging by hand and want to watch the browser. Never leave one in a committed test to fix a timing problem.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Selenium finished driver.get(url) and the very next find_element raised NoSuchElementException, although the element is clearly on the page a moment later. What is the most likely cause?

    get waits for the load event only. Content added later by scripts is not there yet, so the search finds nothing at that instant. A wait for the element fixes it.
  2. 2. What does a StaleElementReferenceException tell you?

    A stale reference means the node you found earlier is detached, usually because the page re-rendered it. Find the element again instead of reusing the old object.
  3. 3. Which is a real disadvantage of using a fixed sleep(5) to wait for a page?

    A fixed duration is a guess. It is too long for fast pages and too short for slow ones, so it costs time and still causes random failures.
  4. 4. Which description matches ElementClickInterceptedException?

    The click would land on a different element, for example a cookie banner or a loading mask. Wait until the overlay is gone, or close it first.

Summary

  • The script and the page run at the same time, so the script can be ahead of the page.
  • get() waits for the page load only; content added later by JavaScript needs its own wait.
  • NoSuchElement means not there yet, ElementNotInteractable means not usable yet, StaleElementReference means the page replaced the node.
  • Re-find elements instead of keeping old references, and store locators rather than elements.
  • A fixed sleep() is slow and unreliable. Wait for an observable condition, with a time limit.

Next: Implicit, Explicit and Fluent Waits.