Eve library

OSHUN V1 Playwright Visual QA Checklist

Spec

Revision 1

On this page

Status: Mandatory for major redesigns
Owner: OSHUN Design + Product + Engineering
Last Updated: March 26, 2026
Implements: V1-UX-074

1. Purpose#

Every claimed UX improvement in a major redesign must map to:

  • a concrete Playwright spec
  • a named Playwright test or scenario
  • a tested UI state
  • a captured UI artifact from that tested state

This checklist is the automation companion to docs/releases/v1/specs/design-review-checklist.md.

The design-review checklist proves that the redesign claims are explicit and reviewable. This visual QA checklist proves that those same claims are backed by real Playwright coverage and a captured state artifact.

2. Required Inputs#

Before completing this checklist, the author must already have:

  1. claim IDs from the design-review checklist
  2. the Playwright spec path and test title that validate each claim
  3. a captured UI artifact for the tested state
  4. the command that ran the relevant Playwright coverage

Claim IDs must carry forward unchanged from the design-review checklist.

3. Acceptable Capture Artifacts#

The captured state for a claim must come from one of the following:

  • a stable Playwright snapshot assertion such as toHaveScreenshot(...)
  • a success-state screenshot captured explicitly during the Playwright run
  • an attached Playwright artifact that clearly identifies the exact validated state

Failure-only screenshots, videos, or traces are useful for debugging, but they do not count as the primary captured artifact for a successful UX claim unless the state itself is still clearly and intentionally identified.

4. Required Claim Mapping#

For every UX claim in a major redesign:

  • identify the exact Playwright spec file
  • identify the exact test title or scenario
  • identify the exact state ID being validated
  • record the viewport used for the captured state
  • record the capture artifact name or path
  • record pass/fail status and any notes

No major redesign claim should remain unmapped.

5. Mandatory Checklist#

  • Every redesign claim ID from the design-review checklist appears here.
  • Every claim maps to at least one Playwright spec path.
  • Every claim maps to an exact Playwright test title or scenario.
  • Every claim maps to at least one named UI state ID.
  • Every mapped state has a captured UI artifact.
  • Desktop and mobile are both covered when the UX claim applies to both.
  • Disclosure, overlay, empty, loading, error, degraded, and completion states are covered when the claim touches them.
  • The artifact name or snapshot name is specific enough to find again.
  • The Playwright command that produced the evidence is recorded.
  • The final checklist records whether each claim passed, failed, or is blocked.
  • Any non-automatable claim is explicitly marked blocked with rationale and a follow-up owner.

Do not sign off a major redesign until this checklist is complete.

6. Preferred Evidence Pattern#

Use this order of preference:

  1. Playwright toHaveScreenshot(...) or equivalent stable visual assertion for the exact state
  2. explicit success-state screenshot capture from the Playwright run
  3. additional trace/video/log artifacts as supporting evidence

The best visual QA package includes both:

  • a functional assertion that proves the state is correct
  • a visual artifact that lets reviewers inspect the state quickly

7. Visual QA Record Template#

Copy this block into the PR description, task comment, or review doc for every major redesign:

md
## Visual QA

- Linked design review:
- Owner:
- Run date:
- Verification command:

| Claim ID | UX claim | Playwright spec | Test title / scenario | State ID | Viewport | Capture artifact | Result | Notes |
| -------- | -------- | --------------- | --------------------- | -------- | -------- | ---------------- | ------ | ----- |
| UX-01    |          |                 |                       |          | desktop  |                  |        |       |
| UX-01    |          |                 |                       |          | mobile   |                  |        |       |
| UX-02    |          |                 |                       |          |          |                  |        |       |

### Blockers / Non-Automatable Claims

- Claim ID:
- Why automation is blocked:
- Manual fallback evidence:
- Follow-up owner:

8. Operational Rule#

For major redesigns:

  • a completed design-review checklist is required first
  • the visual QA checklist must reuse the same claim IDs
  • every claimed improvement must point to a passing Playwright test and a captured state artifact
  • missing mapping, missing artifacts, or missing automation status blocks signoff

If a redesign claim cannot be traced from claim to Playwright spec to captured state, it is not ready for final approval.