Part 25: Visual Testing | TechByteByByte

Visual testing compares a new screenshot with an approved baseline, like a careful spot-the-difference puzzle.

baseline + new image → differences → decision

Series: Playwright Zero to Expert | Demo app used throughout this series: saucedemo.com

Every assertion you’ve written so far checks something specific — a piece of text, a count, a URL. This part covers a fundamentally different kind of check: does the page look right, as a whole, visually? This is a genuinely different problem from everything covered so far, with its own real, honest trade-offs.

Visual Regression, and Why It’s a Different Kind of Test

A visual regression is an unintended visual change — a button that’s suddenly the wrong color, text that’s overlapping something it shouldn’t, a layout that’s broken on a particular screen size — that a purely functional test (checking text, checking counts, checking that a click leads somewhere) can completely miss, because functionally, nothing is actually “wrong.” The button still works when clicked; it’s just wrong-looking. This is exactly the gap visual testing exists to close.

import { test, expect } from "@playwright/test";

test("login page matches visual baseline", async ({ page }) => {
  await page.goto("/");
  await expect(page).toHaveScreenshot("login-page.png");
});

The first time this test runs, there’s no existing baseline to compare against — Playwright captures the current screenshot and saves it as the new baseline, the reference image every future run will be compared to. Every subsequent run captures a fresh screenshot and compares it, pixel by pixel, against that saved baseline. If they match within an acceptable, configurable tolerance, the test passes. If they differ meaningfully, the test fails, and Playwright generates a visual diff image highlighting exactly what changed.

Analogy: The Spot-the-Difference Puzzle vs. The Checklist

Functional Verification (The Checklist): Imagine auditing an office. You carry a checklist: “Does the office have a desk? Is the chair present? Are the lights operational?” If everything is present and functional, the office passes the audit. You don’t notice that the wall was accidentally painted neon pink instead of beige, because “color” wasn’t on the functional list.

Visual Regression (Spot-the-Difference): You give the auditor a master reference photograph of the beige office (the baseline). The auditor walks in and overlays a semi-transparent screen of the reference photograph onto the actual room. If the wall is neon pink, the pixels immediately mismatch. They flag the visual discrepancy instantly, even though the desk and chair function perfectly.

📊 Visual Flowchart: The Visual Regression Testing Lifecycle

Here is the step-by-step pipeline of visual verification and snapshot updates:

graph TD
    Start["Run visual test: toHaveScreenshot()"] --> CheckBaseline{"Does baseline<br>image exist?"}

CheckBaseline -->|No| Create["Capture screen & save as baseline.png"]
    CheckBaseline -->|Yes| Compare["Compare current screen pixel-by-pixel against baseline"]

Create --> Pass["Test Passes (Initial baseline established)"]
    Compare --> Match{"Do pixels match<br>within tolerance?"}

Match -->|Yes| Pass
    Match -->|No| Fail["Test Fails & generates diff.png"]

Fail --> Intentional{"Is change intentional?<br>(e.g. Redesign)"}
    Intentional -->|No / Bug| FixApp["Fix application UI styles"]
    Intentional -->|Yes| Update["Run: npx playwright test --update-snapshots"]

Update --> Pass

npx playwright test --update-snapshots

When a visual change is genuinely intentional (a real, deliberate redesign, not a bug), this command regenerates the baseline images to match the new, correct current state — a deliberate, explicit human decision, not something that happens automatically just because a test failed.

Baselines, and the Genuine Challenge of Dynamic Content

Here’s where visual testing gets honestly harder than every other kind of testing covered in this series, and it’s worth being direct about why. Imagine SauceDemo’s inventory page included a “Last updated: 2 minutes ago” timestamp, or a randomly rotating banner ad, or content genuinely different for each individual logged-in user.

A pixel-by-pixel comparison would flag this content as a “difference” on essentially every single run — a false positive: the test correctly detects that the pixels changed, but incorrectly concludes that something is wrong, when in reality this specific content was always expected, and even designed, to change.

This is worth sitting with as a real, structural tension, not a minor technical annoyance: the entire value of visual testing depends on the page being visually deterministic — the same, every time, under the same conditions — but real, modern web applications are very often deliberately not fully deterministic in exactly this way.

The practical strategies for handling this:

Masking dynamic regions

— explicitly tell Playwright to ignore specific areas of the page during comparison, rather than trying to force the entire page to be static:

await expect(page).toHaveScreenshot("inventory-page.png", {
  mask: [page.locator(".last-updated-timestamp")],
});

Disabling animations

— a moving or transitioning element captured at a slightly different frame each run will produce spurious, false differences purely from timing, unrelated to any actual visual regression:

await expect(page).toHaveScreenshot("inventory-page.png", {
  animations: "disabled",
});

Using consistent test data

— recall Part 21’s discussion of dynamic data. For visual testing specifically, this cuts the opposite way from most of this series’ advice: you generally want the page’s content to be as fixed and predictable as possible for the specific purposes of a visual comparison, even while other kinds of tests genuinely benefit from dynamic, varied data.

Testing components in isolation, rather than entire complex pages,

where practical — a full page with many independently dynamic regions is a much harder visual testing target than one small, self-contained, genuinely static component (a button, a card, a form) tested on its own.

Visual Testing Strategy — Where It Actually Belongs

It’s worth connecting this back to Part 0’s testing pyramid explicitly: visual testing is not a replacement for functional testing, and it’s not something that belongs on every single test in your suite.

A reasonable, deliberate strategy: reserve visual testing specifically for areas where the visual appearance itself is the primary thing that actually matters — a marketing landing page, a design system’s component library, a critical, highly visible piece of branding — rather than applying it broadly across an entire application’s every page and state.

Used well, on the right, deliberately chosen targets, visual testing catches a genuine, real category of bug (a CSS regression that breaks nothing functionally but looks clearly wrong) that no assertion covered earlier in this series could ever catch.

Used carelessly, applied indiscriminately across every dynamic, content-heavy page in an application, it tends to produce a steady, draining stream of false positives that erode trust in the entire suite — precisely the kind of flakiness Part 28 will address in full.

How It Works in a Real Test Run

A visual assertion renders the page, captures pixels, and compares them with an approved baseline under matching conditions. Differences can come from a genuine regression or from fonts, animation, time, data, operating system, browser version, and viewport.

Stabilize deterministic state, mask truly dynamic regions, keep baselines separated by relevant project, and review diffs rather than automatically updating every failed image.

Interview Questions

Q: What kind of bug does visual testing catch that a purely functional test (checking text, counts, or navigation) cannot?

Ans: It catches purely visual regressions — a button rendering in the wrong color, overlapping text, a broken layout — where the underlying functionality still technically works correctly (a click still navigates correctly, the right text is still present in the DOM) but the actual visual appearance is wrong. A functional test checking that a button exists and is clickable would never notice that the button is, say, rendering with completely broken CSS.

Q: What is a baseline in visual testing, and how does a test actually use it?

Ans: A baseline is a saved reference screenshot representing the expected, correct visual state of a page or component. On each subsequent test run, a fresh screenshot is captured and compared pixel by pixel against that saved baseline — the test passes if they match within an acceptable tolerance, and fails, with a generated visual diff, if they differ meaningfully.

Q: Why is dynamic content a genuine, structural challenge for visual testing, rather than just a minor inconvenience?

Ans: Visual testing fundamentally depends on the page being visually deterministic — producing the same pixels every time under the same conditions — but real content like timestamps, rotating banners, or per-user data is often deliberately expected to change between runs. A naive pixel comparison would flag this expected, intentional variation as a failure every single time, producing constant false positives that have nothing to do with an actual visual regression.

Q: What are two concrete techniques for handling dynamic content in a visual test, and what does each one actually do?

Ans: Masking explicitly excludes specific, known-dynamic regions of the page from the pixel comparison, so a timestamp or rotating element doesn’t trigger a false difference. Disabling animations prevents an in-transition element from being captured at a slightly different, inconsistent frame each run, which would otherwise produce spurious differences unrelated to any genuine visual regression.

Q: Why shouldn’t visual testing be applied broadly across every single page and state of an application, according to a healthy testing strategy?

Ans: Applying it indiscriminately, especially to dynamic, content-heavy pages, tends to produce frequent false positives, which erodes trust in the suite over time and creates real maintenance burden without a corresponding increase in genuine bug-catching value. A more deliberate strategy reserves visual testing specifically for areas where visual appearance itself is the primary concern — like a design system or a critical branding page — rather than treating it as a default check applied everywhere.

Q: A visual test fails after a deliberate, intentional redesign. What’s the correct next step, and why shouldn’t this be treated as a real test failure to “fix” by rolling back the design change?

Ans: The correct next step is to run --update-snapshots (or the equivalent) to deliberately regenerate the baseline to reflect the new, correct, intended design — this is an explicit, human decision confirming the new visual state is the desired one, not an automated action. Treating this as a failure to fix by reverting the redesign would be a misunderstanding of what the test is actually for — it’s meant to catch unintended visual changes, not to prevent legitimate, deliberate design updates from ever happening.

Exercises — Part 25

Understand: Explain, in your own words, why a page containing a live, constantly-updating “time since last login” display would be a genuinely difficult target for straightforward visual testing, and name one technique from this part that would help.

Simple Practice: Write a visual test for SauceDemo’s login page using toHaveScreenshot(), run it once to establish a baseline, then deliberately make no changes and run it again to confirm it passes against that saved baseline.

Real-World Scenario: Imagine SauceDemo’s inventory page displayed a randomly selected “featured product” banner that changed on every page load. Design a visual testing approach for this page that avoids constant false positives from that specific banner, while still meaningfully verifying the rest of the page’s visual layout — explain your reasoning.

Challenge: Deliberately introduce a small, genuine visual change to a page you control (or a local HTML file) — like changing a button’s color — after establishing a baseline. Run the visual test again, observe the failure and the generated diff image, and then use --update-snapshots to accept the change as the new correct baseline, describing what changed at each step.

Next: Part 26 — Cross-Browser and Device Testing

— running the same suite across Chromium, Firefox, and WebKit, plus mobile emulation, viewports, geolocation, and permissions.

Author: TechByteByByte Editorial Team
Reviewed by: TechByteByByte Admin
Published: August 31, 2026
Last reviewed: August 31, 2026