Playwright flow & journey sweep — findings report (2026-06-02)

Executable Playwright pass over the apps/oshun/web V1 PWA, run against a live prod build + BFF, to test the flows that back the WALKTHROUGH/ journeys and per-view checklists. Complements runtime-sweep-2026-05-29.md (which visited every route once with a bespoke crawler): this run executes the repo's own behavioural Playwright e2e suite — the 601 specs under apps/oshun/web/e2e/ — and records real pass/fail, then categorises every failure by root cause.

Relationship to the 2026-05-29 re-walk#

The 2026-05-29 re-walk concluded "the web app works" — every surface renders with real (mock) data, client 401s self-heal. That was a render/data walk (does each view load and populate). This run is stricter and complementary: it executes the behavioural e2e specs, which additionally assert specific interactions, exact selectors, keyboard paths, and a full WCAG 2.1 AA axe pass on every spec. Both are true at once — pages render with data and a set of behavioural/accessibility assertions fail. The findings here are spec-level (a11y violations, duplicate DOM, studio-session seeding), not a regression of the render-level conclusion.

TL;DR#

• Non-studio suite (143 spec files, 667 test cases): 320 passed, 345 failed, 2 skipped. Failures are not a blanket outage — pass/fail is mixed within files. They decompose into: real accessibility findings (38), real DOM bugs (3), data-hydration/wait gaps under the mock BFF (152), unprovisioned test dependencies (65: Storybook + visual baselines), and assorted assertion mismatches (~85).
• Studio suite (studio-a half = 229 spec files, 2,350 cases): 4 passed, 2,346 failed. This is a single systemic harness gap, not 2,346 bugs: studio specs seed a hard-coded placeholder cookie, the app correctly redirects unauthenticated studio access to /welcome?...&reauth=1, so every assertion fails. The studio auth gate works; the suite just never establishes a real session. The second studio half (studio-b, 229 files) was intentionally halted — it would reproduce the identical signal at a ~2 hr cost.
• Real, actionable product findings: WCAG color-contrast violations on several customer surfaces; 3 duplicate-DOM (strict mode) defects; the 3 new /studio/generation/{curated-cards,music,nyx-3d} surfaces render with no <h1>. (The originally-listed user-flag-to-review "no flag affordance" gap turned out to be stale — see A3; the affordance ships and works.)

2026-06-02 fix pass: the a11y findings (color-contrast, button-name) and the generation <h1> are now fixed and committed; the lz4-on-arm64 boot blocker and the Storybook-absent test failures are fixed; A3 was re-verified as already-shipped. The studio-session seed (Rec 1) was resolved by codemodding the studio specs to real-session current-UI smoke. A deep contrast pass then closed the open pwa-smoke/assistant tail: it found + fixed real defects (invisible /welcome/download install CTAs at 1.07:1, dark-on-dark preview captions, the 4.48:1 routine quick-start pill, a welcome h1→h3 skip) and proved the rest were first-load animation/double-mount test transients (fixed at the test layer via suite-wide reduced-motion + a hydration settle). pwa-smoke 23/23, assistant-accessibility 4/4, domain-readability 6/6, contrast = 0. Remaining open: the service-worker double-mount is a known sub-second first-load transient (mitigated in-suite, not a production defect), and the mock-adapter hydration cluster (A4).

• Studio surfaces verified working via a real-session render probe (below): with a properly-seeded session, every representative studio workspace renders its <h1> and does not redirect — confirming the 2,346 studio failures are exclusively the placeholder-session seed bug, not broken studio flows.

Environment#

Pre-flight blocker fixed: lz4 x86_64 on arm64#

The BFF would not boot on this Apple-Silicon host: lz4 (pulled in transitively via rbxm-parser) shipped an x86_64 native binary (xxhash.node / lz4.node), so require('lz4') threw ERR_DLOPEN_FAILED at import time — before the USE_MOCK_ADAPTERS switch could take effect. Rebuilt lz4 for arm64 (node-gyp@11 rebuild); the BFF then booted clean (/healthz 200). This blocks any local BFF run on arm64 until the dependency ships an arm64/universal prebuild — worth pinning a prebuilt or adding a postinstall rebuild.

Method & caveats#

• Verifies: the behavioural specs the repo ships — rendering, navigation, keyboard/focus, axe (WCAG 2.1 AA) accessibility (auto-afterEach on every spec), resilience/edge states, cross-domain continuity — execute green against the running build with a real customer session.
• Does NOT verify: real third-party providers (mock adapters only), DB-backed durability, webkit/firefox/native-mobile, push/telemetry delivery, real-money crypto settlement, live screen-reader audio.
• networkidle caveat: several specs waitForLoadState('networkidle'); the V1 shell runs background activity (analytics/poll), so networkidle may never settle and those waits fail at the 25s cap regardless of app health → counted as harness-artifact.
• 25s timeout: chosen so failing waits resolve fast (base config is 60s). Prod pages load < 10s, so this does not false-fail healthy pages.
• Not run to completion: the studio-b half (229 files) was halted once studio-a established the systemic studio result (below). Webkit/firefox/mobile projects were out of scope.

Headline results#

39 of the 143 non-studio spec files are fully green; 104 have ≥1 failure (most with a mix of passing and failing cases in the same file).

Non-studio failure taxonomy (345 failures)#

Reading it: ~65 failures are unprovisioned test infrastructure (Storybook + visual baselines), ~152 are data-hydration/wait under the mock adapter (the page works but mock data doesn't populate every lane), and ~41 are genuine product findings (a11y + duplicate DOM). The remainder are assertion drift that needs per-spec triage.

Real, actionable findings#

A1 — WCAG color-contrast violations (serious) — FIXED + VERIFIED#

Status: fixed and re-verified — once A2's shell-settle wait let the wcag-signoff bodies pass, axe ran clean: 0 serious/critical across home / library / search / notifications / settings + all 6 domain entries. The scrollable-region-focusable / document-title items did not recur on these surfaces (the latter was a false-positive from the now-fixed axe-on-skipped hook). Original detail below.

2026-06-02 deep pass (welcome / download / assistant + the contrast tail). A second, harder look at the pwa-smoke / assistant-accessibility contrast hits (the items left open at first pass) separated real steady-state defects from first-load test transients, using fully-settled single-mount axe scans as ground truth (a settled /welcome scan reports a single mount and zero contrast violations):

Real defects — FIXED (fix(core) 5f5097c5af):

• /welcome/download install CTAs were invisible — .welcomeDomainCta mixed var(--domain-accent) (only defined inside the domain shell, unset on the marketing page) into color-mix(), making the declaration invalid so the dark fill was dropped and #f8fafc text fell onto bare cream (1.07:1). Fixed with a var(--domain-accent, var(--l-accent)) fallback.
• /welcome/download preview-card captions were dark-on-dark (#1f1a14 title 1.23:1, #574d3f body 1.69:1) on the near-opaque ink card frame; caption text is now paper-toned (inverts ink across themes).
• Routine quick-start pills (Start <routine>) used the brand accent as text on a 10% terracotta tint = exactly 4.48:1 (just under AA). Added a theme-aware --l-accent-text / LV.accentText tint-safe token (mirrors the existing --l-accent-2-text) and switched the pills to it.
• Welcome heading skip — hero <h1> was followed by an <h3> ("Six rooms, one practice."), skipping h2 and breaking the outline (WCAG 1.3.1); promoted to <h2>.

False positives — NOT defects (fixed at the test layer, test(core) e1ff3666bc): the remaining pwa-smoke contrast hits were axe sampling the welcome page mid entrance-animation (fades composite settled-passing text to e.g. #c5beb0) and mid first-load double-mount (SSR+client briefly coexist under parallel load, duplicating the heading tree / [data-auth-entry]). Root cause: the fast-config reducedMotion never reached the browser context. Emulating reduced motion suite-wide + a waitForPageHydrated settle before the axe scan makes the gate measure steady state. Verified: pwa-smoke 23/23, assistant-accessibility 4/4, domain-readability 6/6, contrast = 0.

Two flawed keyboard checks in assistant-accessibility were also corrected (not app bugs): Send is correctly disabled (unfocusable) on an empty composer, so the test now seeds a draft before asserting reachability; the voice toggle lives in the header, so it is reached via Shift+Tab (the dock is non-modal — Tab intentionally escapes it forward).

The auto-axe afterEach fails 38 specs. Color-contrast (serious) is the dominant rule, hit across (by spec file, descending): pwa-smoke (5), nyx-full-journey (5), tara-session-player (4), tara-full-journey (3), auth-entry-flows (3), arete-habit-grounding-links (3), viewport-fit (2), assistant-accessibility (2), plus welcome-marketing, tara-teacher-pages, home-mixed-domain-recommendations, two-taps-to-value. Two dedicated wcag-aa-signoff-v1-p2-351x specs also fail. Also: 3 missing <title> and 3 non-discernible button (critical) cases. → Fix the Lilith token contrast pairs on these surfaces; add <title>/aria-label where flagged.

A2 — Duplicate-DOM (strict mode) — INVESTIGATED + FIXED#

Selectors resolved to 2 elements under the suite — [data-shell-root] (the most disruptive: it masked the WCAG-signoff bodies before axe could run), the search/explore domain chips/tabs, and [data-auth-display-name-input].

Investigation (2026-06-02), prod-vs-test: these are not a component duplicate-render — every node renders exactly once statically (SSR) and in steady state. Direct probing showed the duplicate is a sub-second transient: on a fresh browser context's first navigation the count is briefly 2 and settles to 1 (measured immediate=2 → settled=1), recurring (1→2→1) while the service worker installs/activates and the client takes over. The SW navigation strategy is network-first (it does not serve a cached shell online), and PwaBootstrap does not reload on first activation — both ruled out. So this is not a persistent, production-visible defect (real users see at most a sub-second double-render flash on first visit, then permanently one shell); it is a hydration/SW-activation race the strict-mode assertions caught mid-flight.

Fix: a shared waitForShellSettled(page) in e2e/support/shell-auth.ts (called by openAuthenticatedShellPath) waits for a stable single [data-shell-root] (≥600 ms continuously, riding out the recurring transient) before assertions run. This is a legitimate ready-wait, not SW-masking — the steady state is always one root. Verified: the wcag-signoff suite went from 14 failed → 22/22 passed, with zero [data-shell-root] strict-mode failures, which in turn let axe finally run and confirm A1 is clean (0 serious/critical on home / library / search / notifications / settings + all 6 domain entries). Specs that navigate without this helper (e.g. some studio specs) should adopt waitForShellSettled for the same guarantee.

A3 — user-flag-to-review — RESOLVED (the finding was stale)#

Re-verified 2026-06-02: the trust-&-safety flag affordance does exist and works end-to-end, contrary to the original (stale) finding. TrustSafetyFlagButton (data-testid="trust-safety-flag-button", a category + rationale dialog) is mounted on the Veritas article (confirmed in /veritas HTML), the Tara sit player, the Nisaba passage, and there is a dedicated /scene/[id]/report page. It submits via the submitTrustSafetyFlagAction server action → POST /v1/user-reports (verified live: 202, returns a review record with status: open, priority, and reviewer-assignment fields). Operators review them via apps/oshun/admin (trust-safety/page.tsx + UserReportsList.tsx). The journey is walkable. Residual (minor): generic domain-artifact / comment surfaces still don't mount the button — the high-traffic content surfaces do.

A4 — Data-hydration cluster — TRIAGED (mostly A2, not a data outage)#

The 105 toBeVisible + 47 timeouts looked like a mock-data hydration outage on profile/* and the home variants. Re-run in isolation (workers=1) tells a different story: profile/home largely pass (e.g. profile-consent-controls renders its Phase-22 flows, home-domain-narrative-order keeps Tara-anchored order — 4/7 of a focused batch passed). The pages do hydrate their data (corroborated by the real-session probe: /profile and every customer anchor render with real content, and the BFF endpoints return 200 directly).

So this bucket was inflated by A2: under suite concurrency the SW double-mount makes shell-scoped selectors ([data-domain-subtitle], list rows, etc.) resolve to 2, which surfaces as toBeVisible/toHaveCount/networkidle failures rather than a genuine empty-data state. What's left after isolating A2 is a small number of per-spec assertion mismatches (toContainText/toHaveCount/toEqual against specific copy or computed values) that read as spec staleness, not a systemic gap. Net: there is no broad mock-adapter hydration defect; fix A2 (SW isolation) first, then triage the residual handful of assertion drifts per-spec.

A5 — New generation surfaces render without an <h1>#

The real-session probe (below) found /studio/generation/curated-cards, /studio/generation/music, and /studio/generation/nyx-3d all render with no <h1> — the loading state shows "Loading…" and the loaded state shows cards with <h3> headings, but the page never exposes a top-level heading. Every sibling studio workspace (Isis/Aja/Yemaya/Bellona/Hathor/Concordia) and the fourth generation surface (/studio/generation/living-scene → "Living Scene") expose an <h1>. → Add a page <h1> to CuratedCardsStudio, MusicStudio, and Nyx3DStudio (document structure / a11y / SEO). These are the surfaces added in the 2026-06-02 walkthrough coverage sweep.

Studio suite — systemic harness gap (root-caused)#

studio-a (2,350 cases) failed 2,346. Root cause, confirmed by direct observation: studio specs seed a placeholder cookie, e.g.

// e2e/studio-isis-image-generation-smoke.spec.ts
await page.context().addCookies([{ name: 'oshun-session',
  value: 'studio-isis-image-generation-e2e-session', ... }]);

This is a non-BFF-issued literal — unlike the customer seedCustomerSession, which performs a real /v1/auth/login→/v1/auth/signup. The running app correctly treats it as unauthenticated and redirects:

navigated to "http://127.0.0.1:3010/welcome?redirect=%2Fstudio%2Fisis%2Fimage-generation&reauth=1"

So every studio spec lands on /welcome and every toBeVisible / toHaveCount / not.toHaveURL assertion fails. Failure signatures (868 toBeVisible, 844 25s-timeouts, 317 not.toHaveURL, 94 "execution context destroyed by navigation") are all downstream of that one redirect.

Two takeaways: (1) the studio auth gate is working as designed — a positive finding; (2) the studio e2e suite needs a real studio session seed (BFF login, mirroring seedCustomerSession) to test anything. studio-b was halted rather than burn ~2 hr reproducing the same redirect 1,800+ times.

UPDATE (2026-06-02): takeaway (2) is now fixed — 375 studio smoke specs were codemodded to delegate to the real seedCustomerSession, and the reauth redirect is gone (verified). The studio suite is no longer auth-blocked; the remaining studio failures are the lane-card assertion staleness above, which is the next spec-maintenance step.

Note: the four studio cases that "passed" were assertions that tolerate the /welcome landing (e.g. an index-nav link check), not evidence of a working studio workspace.

Real-session render probe (proves studio surfaces work)#

To isolate the seed bug from real app health, a one-off probe (zz-walkthrough-real-session-probe.spec.ts, since removed) seeded a real session via seedCustomerSession and visited representative studio routes + customer journey anchors (axe skipped). Result — with a real session every studio workspace renders its <h1> and none redirect:

All 12 customer anchors also rendered with a proper <h1> and no redirect (/, /tara, /arete, /veritas, /nyx, /nyx/sky-almanac, /nisaba, /metis, /sophia, /library, /explore, /profile) — corroborating the render-level health from the 2026-05-29 re-walk. Conclusion: the studio e2e-suite's 2,346 failures are entirely the placeholder-session seed; fix the seed (recommendation 1) and the studio surfaces are reachable and rendering.

Unprovisioned test dependencies (not app defects)#

• Storybook (:6006) — visual-regression-design-system.spec.ts navigates to http://localhost:6006/iframe.html?...; without it, 20 specs hit chrome-error://. Start pnpm build-storybook && serve (or storybook dev) before this spec group.
• Visual-regression baselines — *.spec.ts-snapshots/*-chromium-darwin.png are absent for this platform, so toHaveScreenshot writes "actual" and fails. Commit darwin baselines or run with --update-snapshots on a trusted build.

WALKTHROUGH journey coverage (52 journeys)#

The 52 journeys/ flows are conceptual; the e2e suite is feature/route-named (not 1:1). Coverage below is by route/feature overlap with the non-studio run (studio-dependent journeys are blocked by the studio-session gap above).

Net: customer-domain and discovery journeys are genuinely exercised (with the a11y/hydration findings above); operator/studio/tenant journeys are not meaningfully tested until the studio session seed is fixed.

Reproduction#

# 1. BFF (arm64: rebuild lz4 first — see Environment)
cd apps/oshun/web/../bff && USE_MOCK_ADAPTERS=true PORT=4010 npx tsx src/server.ts
# 2. Web (prod)
cd apps/oshun/web && pnpm build && NEXT_PUBLIC_OSHUN_BFF_URL=http://127.0.0.1:4010 pnpm start
# 3. Suite (against running servers; fast config)
cd apps/oshun/web
E2E_BASE_URL=http://127.0.0.1:3010 NEXT_PUBLIC_OSHUN_BFF_URL=http://127.0.0.1:4010 \
  node ./scripts/run-playwright.mjs test -c playwright.fast.config.ts \
  <spec-files...> --workers=5 --reporter=json
# (optional deps) Storybook on :6006 for visual-regression-design-system;
# darwin visual baselines via --update-snapshots on a trusted build.

Recommended next actions (priority order)#

1. Fix seedStudioSession to mint a real BFF session — DONE (2026-06-02). Codemodded 375 studio smoke specs to delegate to the real seedCustomerSession; the /welcome?...&reauth=1 redirect is gone (verified). But it is not sufficient on its own: the studio suite still fails on a second, pre-existing blocker — the specs assert on embedded Hathor Economy-Modeling controls that the lane-card refactor replaced with link cards (see the studio cross-cutting findings). Making the studio suite green needs that assertion-vs-lane-card spec maintenance as a follow-up; the auth seed fix is the prerequisite that unblocks it.
2. Fix the WCAG color-contrast tokens — DONE + verified (A1): tara/arete accent tokens darkened, TaraSurface button to white-on-accent, Veritas reader buttons labelled. wcag-signoff is now 22/22 with 0 serious/critical axe violations.
3. Dedupe the search/auth duplicates — resolved as a transient, not a component bug (A2): fixed with a stable shell-settle wait; no app dedupe was needed (the components render once).
4. Ship the trust-&-safety flag affordance (A3) — already shipped + verified working (2026-06-02); residual is only generic comment surfaces.
5. Provision the test dependencies (Storybook :6006; commit darwin visual baselines) so visual-regression specs are meaningful, not env-fails.
6. Triage the profile/home data-hydration cluster (A4) against a real-data BFF before signoff.
7. Publish an arm64 lz4 prebuild / postinstall rebuild so the BFF boots on Apple Silicon without manual intervention.
8. Run the suite on webkit + firefox + mobile projects (not covered here).