Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 35 ยท Beyond the Basics

Visual and Accessibility Testing

Catch layout regressions and accessibility bugs automatically: screenshot baselines, toHaveScreenshot, axe-core checks and ARIA snapshots, with runnable stand-ins.

A functional test can pass while the page looks broken. The button still exists, the text is still there, and a click still works, but a stylesheet change pushed the button under the footer, or a dark-mode colour made the label unreadable. Likewise, a test can find a button by its test id while a screen reader user cannot find it at all, because it has no accessible name.

This lesson covers two kinds of checks that fill those gaps. Visual testing compares screenshots of the page against approved images. Accessibility testing runs rules (usually with the axe-core engine) over the page and compares the accessibility tree against what you expect. Both are automated, both are useful, and neither replaces a human looking at the page or using a screen reader.

The real tools cannot run on this site, so the Cypress and Playwright code is read-only. The two runnable examples are stand-ins written in plain JavaScript: a pixel comparison on tiny grids of numbers, and a rule checker over a small fake element tree. They show the idea, not the real implementation (real tools compare actual PNG images and run hundreds of axe rules).

How screenshot testing works

The workflow has three parts:

  1. Baseline. The first time the test runs, the tool takes a screenshot and saves it next to the test as the approved "golden" image.
  2. Comparison. On every later run, it takes a new screenshot and compares it with the baseline, pixel by pixel, with some tolerance.
  3. Review. If the images differ beyond the tolerance, the test fails and the tool writes the expected image, the actual image and a diff image that highlights the changed pixels. A person looks at them and decides: is this a bug, or an intended change? If it is intended, you update the baseline.
flowchart LR
  A["Run test"] --> B{"Baseline<br/>exists?"}
  B -->|"no"| C["Save screenshot<br/>as baseline"]
  B -->|"yes"| D["Compare with<br/>baseline"]
  D -->|"within tolerance"| E["Pass"]
  D -->|"too different"| F["Fail and write<br/>diff image"]
  F --> G["Review: bug or<br/>intended change?"]
  G -->|"intended"| H["Update baseline"]

A visual test is a comparison against an approved image, plus a human decision when they differ.

Screenshot assertions in Playwright

Playwright has this built in. The assertion is toHaveScreenshot:

TypeScript
// Playwright (read-only)
import { test, expect } from '@playwright/test';

test('home page looks right', async ({ page }) => {
  await page.goto('/');
  await expect(page).toHaveScreenshot('home.png');
});

test('pricing card looks right', async ({ page }) => {
  await page.goto('/pricing');
  const card = page.getByTestId('pro-plan');
  await expect(card).toHaveScreenshot('pro-plan.png');
});

On the first run there is no baseline, so the test fails once and writes the image it took as the new baseline. Run it again and it compares. Baselines are saved in a folder next to the spec file, and the file name includes the browser and operating system (for example home-chromium-linux.png), because fonts and anti-aliasing differ between platforms. When you intentionally change the page, regenerate the images:

Terminal
# read-only
npx playwright test --update-snapshots

Useful options keep screenshots stable:

TypeScript
// Playwright (read-only)
await expect(page).toHaveScreenshot('home.png', {
  fullPage: true,                    // capture the whole scrollable page
  animations: 'disabled',            // freeze CSS animations and transitions
  mask: [page.getByTestId('clock')], // paint a solid box over changing content
  maxDiffPixels: 100,                // allow up to 100 different pixels
});

There is also maxDiffPixelRatio (a fraction of all pixels) and threshold (how different a single pixel's colour may be, from 0 to 1, before it counts as changed). You can set defaults for all tests under expect: { toHaveScreenshot: { ... } } in the config file.

Cypress

Cypress can take screenshots with cy.screenshot(), but it does not compare them by itself. Teams add a plugin (several image-diff plugins exist) or a hosted service that stores baselines and shows diffs in a review UI. The principle is the same: baseline, compare, review. Check the plugin's own documentation for setup, because the details change between versions.

What makes screenshots flaky

A visual test fails for every pixel that changes, so you must remove everything that changes for the wrong reasons:

  • Dates, clocks, random data and ads. Use fixed test data, freeze time, or mask the region.
  • Animations and carets. Disable them (animations: 'disabled') and wait until the page is stable.
  • Fonts and platform. Generate and compare baselines in the same environment, normally the CI container. A baseline made on a laptop will not match a Linux CI run.
  • Loading state. Wait for images and data before the screenshot. Playwright retries a screenshot until two consecutive captures match, but it cannot know what you meant to wait for.
  • Too-large scope. A full-page screenshot fails when anything anywhere changes. Prefer screenshots of one component or region.

The first stand-in compares two tiny "images" (grids of grey levels), counts changed pixels with a per-pixel threshold, and supports a mask for regions that are allowed to change.

JS A tiny pixel-diff with threshold and mask

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Read the output from the top. The masked clock change passes, the unmasked one fails, a barely different shade is absorbed by the threshold, and the moved button fails and prints a map: # marks changed pixels, m masked ones, . matching ones. A real diff image is the same map in colour. In the stand-in, threshold plays the same role as Playwright's option of that name, and changed <= maxDiffPixels matches maxDiffPixels.

Accessibility testing

An accessible page works with a keyboard and assistive technology. Some of that can be checked by a program. The most widely used engine is axe-core. It inspects the DOM and reports violations of rules such as "images must have alternative text", "form inputs must have labels", "buttons must have discernible text", and "text must have sufficient colour contrast".

Playwright uses the @axe-core/playwright package:

TypeScript
// Playwright (read-only)
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test('checkout page has no detectable accessibility violations', async ({ page }) => {
  await page.goto('/checkout');

  const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa'])   // limit to WCAG A and AA rules
    .exclude('#third-party-chat')      // skip a widget you do not control
    .analyze();

  expect(results.violations).toEqual([]);
});

Cypress has the community plugin cypress-axe, used in two steps: cy.injectAxe() after cy.visit(), then cy.checkA11y() to run the rules and fail on violations. Both libraries need installing with npm; follow their documentation for the exact import lines in your version.

Keep the limits in mind:

  • Automated rules find only a part of the problems. They cannot tell whether alt text is meaningful, whether the focus order makes sense, or whether an error message is understandable. Treat a clean axe run as "no known mechanical problems", not as "accessible".
  • Run it on states, not only on first load. Open the dialog, trigger the error, expand the menu, then run axe.
  • Start strict on new pages; ratchet on old ones. If an existing page has 40 violations, record them and fail only on new ones, then reduce the list over time.
  • Use roles and labels in your tests anyway. Locators such as getByRole and getByLabel (see Locators in Playwright) fail when the accessible name is missing, so they double as accessibility checks.

ARIA snapshots

Playwright can also assert the accessibility tree, the structure that screen readers see, as a compact YAML outline with toMatchAriaSnapshot:

TypeScript
// Playwright (read-only)
await expect(page.getByRole('navigation')).toMatchAriaSnapshot(`
  - navigation:
    - link "Home"
    - link "Courses"
    - link "Pricing"
`);

This checks roles, names and nesting without caring about CSS classes or pixel positions, so it survives restyling but catches a missing label or a changed heading level. The method is relatively new, so make sure your Playwright version supports it, and see the docs for the matching rules (the expected outline can be partial).

The second stand-in is a miniature rule engine over a fake element tree. It runs three rules similar to real axe rules and prints each violation with a path to the element.

JS A miniature accessibility rule checker

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Real axe works the same way at heart: a list of rules, each with a selector and a check, run against the live DOM, with results grouped by rule and including the failing elements. It also understands computed accessible names, roles, ARIA attributes and colour contrast, which are far harder than this stand-in's checks.

Where these checks fit

Visual and accessibility checks are most valuable when they are few and targeted: a handful of screenshots for the pages that must look right (home, pricing, checkout), one axe scan per important page state, and ARIA snapshots for shared pieces like the navigation. Run them in the same CI environment every time, and treat baselines as code: review baseline changes in pull requests like any other change. For the React side of the same ideas (testing library queries, jest-axe, Storybook-based visual tests) see End-to-End and Accessibility Testing.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What should you do when a screenshot test fails because of an intended design change?

    A failing visual test is a prompt for a human decision. If the change is intended, you review the diff and regenerate the baseline (for Playwright, with --update-snapshots).
  2. 2. Why should baselines be generated in the same environment that runs the tests (for example the CI container)?

    Anti-aliasing, fonts and layout details vary by platform, which is why Playwright also puts the browser and OS in the baseline file name.
  3. 3. What is the purpose of the mask option in a screenshot assertion?

    Masked regions are painted over before the comparison, so changing content there is ignored.
  4. 4. A page passes an axe-core scan with zero violations. What can you conclude?

    Automated rules catch only the mechanical problems. They cannot judge whether alt text is meaningful or whether the focus order makes sense.
  5. 5. What does an ARIA snapshot assert?

    toMatchAriaSnapshot compares a YAML outline of the accessibility tree, so it ignores styling and checks what assistive technology exposes.

Summary

  • A visual test compares a new screenshot with an approved baseline; a failure needs a human decision, and an intended change means updating the baseline.
  • Playwright has toHaveScreenshot with mask, animations, maxDiffPixels and threshold; Cypress needs a plugin or service for comparison.
  • Remove sources of noise (time, random data, animations, platform differences) or screenshots become flaky.
  • axe-core (@axe-core/playwright, cypress-axe) reports violations of mechanical accessibility rules; it finds only part of the problems.
  • toMatchAriaSnapshot checks the accessibility tree rather than pixels or CSS.
  • Use few, targeted checks in a stable CI environment and review baseline changes like code.

Next: Component Testing