Disciplines · Audits

V1 Product-Coherence Audit — 2026-06-12

distinctive editorial identity ("One shell.

4sections18 minread
Markdown source
On this page

Method. Fresh-eyes product walkthrough of the live stack (real signup → cookie consent → 10-step onboarding → home → all six domain hubs → explore / activity / library / search / profile / notifications / assistant), driven through a real browser against the dev web app (:3010) + BFF (:4010) + docker infra, with every console error and failed request triaged to root cause, followed by code-level verification and fixes. This document is the running artifact for the standing V1-polish goal; the spec baseline is V1/features.md (V1 Exit Criteria §"v1-exit-criteria").

What holds up (verified by use)#

  • First-run experience is genuinely strong. The welcome page has a distinctive editorial identity ("One shell. Six domains. A calmer way in."), trust commitments up front, a privacy-respecting cookie banner, and the 10-step onboarding presents 8 well-written outcome cards with domain pairings and a skip affordance.
  • Tara is visibly the product center (exit criterion #2): home leads with "Begin from center", one clear sequence instead of competing dashboards, and the domain grid presents Tara as the spine with five companions.
  • The domain rooms are real product, not stubs: Tara (guided sit course, teachers rail, live class booking), Nyx (real ephemeris for tonight's date, interactive chart, Nisaba cross-link), Nisaba (full sutta reading desk with concept cards), Metis (cross-domain courses "with Nyx"/"with Nisaba", grounded-and-disclosed AI tutor), Arete (humane cold start).
  • Honest-absence design is a real practice: Veritas's method note ("the surface stays empty rather than showing unverified content"), Arete's "No habits yet", assistant outage notices.
  • The backlog formally agrees: V1/TODOS.md is 1,640/1,641 checked (only §23.1.1 crypto-payments Aje bridge open).

Fixed this session (all committed + pushed to main)#

  1. Member crisis-frame record (/profile/safety, web + mobile) — was a fabricated 'acute-grief' incident for a fictional member; now fetches the member's real durable frames via new GET /v1/safety/crisis-frames/mine (honest empty/unavailable states; owner-only; 14 tests).
  2. Assistant surface — first message raced session creation and posted to literal session id local (404 → false outage notice on a healthy service); keyword-triggered "tool action" cards animated random progress to a fabricated "complete"; failed history fetch rendered four invented sessions. All three fixed (await in-flight session; fabrications removed; honest history outage disclosure).
  3. Veritas home briefing degraded a downstream outage as an unhandled 500 on every home load; now the endpoint's fail-closed 503 (regression test).
  4. Nyx chart fabrication — chart showed a fixed MOON · 84% + invented LEO/JUPITER while the headline showed the real illumination (12%); the tonight card now exposes structured moon state from the same ephemeris and the chart renders a phase-accurate lit region bound to it.
  5. Metis draft isolation (privacy) — course proposals were a global store: any member's draft rendered as "drafted by you" on every other member's schoolroom. Now owner-scoped end-to-end (postgres column + index, store filter, member-required POST) with an isolation regression test.
  6. Metis first-run hero — honest-empty resume cursor rendered as "Resume at § 0 · week 0 of 0"; now a first-run invitation.
  7. Arete coach card — always claimed "Two adjustments offered" with Approve/decline buttons that wrote only to localStorage (nothing read it back), even for members the engine offered nothing. Now renders the recovery plan's real offered action only when one exists, and the decision is a durable per-member BFF record (POST /v1/arete/coach/decision, snapshot-persisted, undo-able).
  8. Activity timeline honesty rework — /v1/activity emitted fabricated rows for every member ("Reset session completed", invented provenance bundles/citation counts, two hardcoded continuity threads, invented assistant excerpts), even on adapter failure. Now serves only the member's real history (saved continuations with the adapter's own titles/timestamps, logged observations, habit records, sitting completions); outages are disclosed in the partial-failure envelope with no stand-in rows. Stale e2e specs that asserted the fabricated rows were rewritten (honest empty state + client analytics against a contract-shaped stand-in).
  9. One consumer brand (owner decision) — domain hubs were branded "Lilith" (wordmark, "Ask Lilith", operator-bar default) vs OSHUN everywhere else; consumer surfaces now carry OSHUN / "Ask OSHUN", coach persona speaks as "your coach". Internal design-system naming unchanged.
  10. Original teacher personas (owner decision) — Tara shipped real meditation teachers (Sam Harris, Tara Brach, J. Goldstein, Pema Chödrön) as personas with session attributions; replaced across BFF/web/fixtures/ tests with original personas (Saro Imani, Tessa Marin, Jonas Kael, Pia Halvorsen). Attributed quote-of-the-day citations remain as quotations.
  11. Misc. — "1 threads are ready" pluralization + zero state; stale AdminInboxDecisionPanel and TaraSitPlayer unit tests realigned with the real (server-confirmed / disclosure-semantics) behavior.

Open items (prioritized)#

P1 — product coherence / correctness#

  • Dual-nav schism (RESOLVED by owner decision, 2026-06-12): the owner chose "keep the hub manuscript top-nav, add an explicit way back" — the hub top-nav is the intentional reading register. Implemented: data-lilith-nav-home "← Home" (aria-label "Back to OSHUN home") in the customer nav, live-verified on /veritas, /tara, and /nyx.
  • Domain-service dev orchestration gap (CLOSED 2026-06-12): Tara, Arete, and Veritas run as real HTTP services via scripts/dev/run-v1-domain-services.mjs (databases + migrations; fixed the Arete service's nanoid-into-uuid ids and missing bridge-user resolution). Nyx, Nisaba, and Metis adapters now run in-process by default — matching V1/ARCHITECTURE.md's BFF-routed-domain-library model — over the real ephemeris + a real-coordinate celestial catalog (nyx), the public-domain corpus + member reading record (nisaba), and the curated catalog + durable tutor-session store (metis), each with domain-correctness tests (35 across the three) and OSHUN_<D>_ADAPTER=http escape hatches. The home Nyx perspective and Arete practice modules and the Nisaba daily passage now serve 200/live end-to-end. Veritas — the last true HTTP downstream — now has its /v1/oshun/* facade (apps/veritas/api/src/interfaces/http/routes/v1/oshun-facade.ts): all 11 read paths + saves/follows verified live against the real Postgres schema (67-table migration applied, taxonomy seeded: 10 categories, 16 topics, 5 sources), Tara-style bridge users in auth_users, verdicts passed through the FactCheckOverallStatus enum verbatim, source credibility from recorded assessments or honestly unknown. Fixed along the way: two never-ran boot bugs in createPoolFromEnv (.getPool() fused onto parseInt; connectionString silently ignored → SASL failure), the seed script's bare PrismaClient constructor, and the readiness probe conflating "Elasticsearch intentionally not deployed" with "failing" (tri-state not_configured check; BFF availability now reports serviceStatus: ok instead of a permanent false "degraded"). Article feeds stay honest-empty until real editorial content is ingested — that ingestion (content ops) is the remaining product gap, not plumbing.
  • useInsertionEffect must not schedule updates (FIXED 2026-06-12): the stack-trap repro showed the culprit was useBrowserSearchParams's history patch, not PwaBootstrap — Next's HistoryUpdater calls history.replaceState inside a useInsertionEffect, and our patched method synchronously dispatched oshun:locationchange, whose listeners setStated inside that window. The dispatch now defers one microtask and coalesces same-tick bursts. Verified: signup→onboarding repro, 0 errors. Fixing it surfaced a worse product bug, also fixed: /signup (and /signin, /login, /register, /recover) were never routes — the proxy treated them as protected paths, bouncing anonymous visitors to /welcome?redirect=%2Fsignup so the auth-entry page itself rode along as the post-onboarding destination (a 404 for a brand-new member). They are now proper 301 aliases to /welcome?mode=…, and sanitizeRedirectPath rejects auth entry pages as post-auth targets (callers fall back to their defaults). Verified live: all five aliases 301 correctly, protected-path and legacy redirects unchanged, signup now lands on clean /onboarding.
  • Veritas hub actions with no lead (VERIFIED FIXED 2026-06-12): the leadAvailable !== false gate in rooms.tsx hides "Save to notebook / Read sources / Share excerpt" in the honest-empty state. Live browser check: 0 action buttons, honest "no grounded lead" copy renders.

P2 — polish / honesty residue#

  • Voice waveform (FIXED 2026-06-12): the assistant's listening waveform now measures the real microphone signal — a parallel getUserMedia capture feeds an AnalyserNode (fftSize 64, lower-spectrum bins averaged into 12 bars at 100ms cadence); the webRandomUnitInterval fabrication is gone. When capture is unavailable (no getUserMedia / permission denied / no AudioContext) the waveform honestly hides while recognition keeps running. Capture is released on stop, recognition end, and unmount (tracks stopped, context closed — also fixes a pre-existing interval leak when recognition ended on its own). Tests assert exact measured bar heights from a deterministic analyser (would fail on random levels) + honest absence.
  • /arete/review & weekly letter (RESOLVED via explicit label, 2026-06-12): the H5 depth-page register now covers this — AreteDepthPreviewBanner mounts DomainPreviewBanner on every fixture-backed /arete/* depth page (only habits/offerings/weekly are live and unlabeled). Verified in-browser: both /arete/coaching and /arete/review render the "illustrative example data, not your live account activity" notice. The data-driven branch stays blocked honestly: no journal store exists anywhere in the system (the BFF room discloses journalText: '' for the same reason), so a "real" letter has no source data until journal capture ships — a feature build, not a polish item.
  • Activity simulation module cleanup (DONE 2026-06-12): types, lookup tables, and the two live formatters extracted to activity-models.ts; activity-simulation.ts deleted with its dead fixture blocks (TIMELINE_ITEMS, MILESTONES, ACHIEVEMENTS, STREAK_CALENDAR, WEEKLY_DIGEST, groupByTime, formatDate — verified zero consumers). All four importers repointed; 90/90 activity tests pass.
  • Activity filter empty copy (FIXED 2026-06-12): the empty state now distinguishes no-history ("No activity yet — sessions, readings, and observations will appear here as you use OSHUN.") from filters (the old copy), via data-activity-empty-reason. Live-verified: a fresh account renders the no-history register.
  • Quote-of-the-day attributions (RESOLVED 2026-06-12): legal exposure removed rather than flagged. The attributed daily pool in TaraCollections.tsx (Thich Nhat Hanh, Eckhart Tolle, David Lynch, Ma Jaya Sati Bhagavati, Hermann Hesse) is now five verifiable public-domain quotations (Marcus Aurelius/Long, Thoreau, Tao Te Ching/Legge, Pascal/ Trotter, Whitman). The ~23 UNattributed program-day epigraphs — many recognizable copyrighted lines (Thich Nhat Hanh ×5, Dalai Lama, Krishnamurti, Anne Lamott, Ram Dass, living authors) — are replaced with original product copy in the same editorial voice; the genuinely public-domain lines (Shakespeare, Meister Eckhart, John Lubbock) stay. Consistent with the confirmed "original personas over real teachers" decision.
  • Service-worker scope (VERIFIED + FIXED 2026-06-12): verification confirmed the risk was real — the SW caches all /v1/* GETs (except /v1/auth) stale-while-revalidate keyed by URL only (the Cache API ignores cookies), and logout never purged them: one member's authenticated responses, warmed authenticated HTML, member media, and queued offline mutations would persist and serve to the NEXT member on the same browser. Fixed: clearLocalAuthState now posts OSHUN_CLEAR_MEMBER_DATA; the SW (CACHE_VERSION v5) purges BFF/static/media caches + the sync queue and re-precaches the public shell, keeping only hashed public JS/CSS. Live-verified: 10 cached /v1/* entries before, zero member-bearing caches after, public shell restored. Unit tests cover the client helper's post + honest false when uncontrolled.

P3 — not yet audited (next walkthrough phases)#

  • Mobile app walkthrough (expo) — same product-lens pass as web. (Static pass DONE 2026-06-12): tsc clean; jest suite brought from 5 failed / 200 passed to 205 suites / 1,040 tests green. Found + fixed: (1) formatDomainLabel had no Metis case, so Metis-linked shared-concept threads rendered a duplicate "Nisaba" chip (concurrent graph commit added Metis links); (2) five offline-cache suites were stale against the honest download manager (b60952a467 made transfers real) — they now drive the manager's REAL queue→transfer→verify pipeline through a shared healthy transfer-port seam (download-manager.test-helpers.ts); (3) native device ids regenerated on EVERY call (no persistence — broke push registration dedup); now persisted via expo-secure-store (localStorage on web, session-stable in-memory fallback); (4) app version was a fabricated '1.0.0' (real manifest says 0.1.0) — now read from expo-constants with an honest 'unknown' fallback. Adversarial fabrication scan otherwise clean (the notification-route E2E harness's "simulated" self-description is its honest register). Remaining: the interactive simulator/device walkthrough (deferred — RAM-constrained alongside the dev web+BFF stack).
  • Studio (486 routes), operator, tenant console surfaces. (Spot-check pass 2026-06-12, 6 routes + failing-request triage): consoles render real lane content with zero 5xx, but the triage surfaced — and fixed — THREE systemic infrastructure bugs affecting every studio console and every sign-in: (1) all ~360 lane consoles fetch their /v1/admin/studio/* endpoints with relative URLs that 404'd against the Next origin — added the /v1/:path* → BFF rewrite in next.config (+ /v1 exempted from the proxy's page-redirect gate so the BFF answers its own 401 JSON instead of a 307 to /welcome); consoles now reach the BFF and render their designed registers (e.g. honest "admin scope required" 403 for members). (2) The api-client token accessor was registered in an AuthProvider effect that runs AFTER children's effects, so every sign-in fired a volley of unauthenticated profile/preferences/saved-items requests (401 noise on every page) — accessor now reads a synchronously-updated ref. (3) Repaired found test breakage: pwa-runtime current-path normalization (fallout from the redirect-sanitizer hardening) and two isis-provenance fixtures broken by a concurrent ledger-validation tightening. Batch 2 (tenant-isolation, rbac-policy, observability-dashboards, launch-readiness, notification-center, /operator): all render real consoles, zero page errors, every /v1/admin/studio/* call reaches the BFF and renders the designed member-vs-admin-scope register. Known cosmetic residual: one fail-soft /library/saved-items 401 fires once around the signup transition (store keeps local cache and retries; profile/preferences 401s are gone). Full console-by-console product pass across the 486 routes remains open.
  • Assistant conversation quality (first pass DONE 2026-06-12): probed the live reply pipeline end-to-end. Architecture is sound — every turn runs the Lilith crisis catalog BEFORE intent routing; the LLM free-text composer exists and is correctly fail-closed on the OSHUN_LLM_* triple-key gate (deploy-bound); replies are grounded in real adapter data. The locally fixable gap was vocabulary: natural wellbeing language ("I keep waking up at 3am", "I'm so stressed", "my mind is racing") fell through to the "could you rephrase" fallback because tara.start_meditation knew only meditation jargon. Extended with sleep/insomnia/stress/anxiety/racing-mind keywords + 5 patterns + examples (427/427 lib tests, 8 new). Live-verified the full chain: utterance → intent → tara adapter → honest reply at every tier (service down → honest outage; empty catalog → "no recommendations right now, browse Tara?"; never fabricated content). Voice/avatar runtime pass also DONE: voice gates on real SpeechRecognition/speechSynthesis presence and the listening waveform now measures real mic levels; avatar mode is a capability + entitlement-gated state machine (renderer/microphone/ synthetic-voice support statuses, explicit fallback, synthetic-voice and synthetic-avatar disclosure indicators, provenance + memory scope) with no fabricated embodiment — absence is disclosed, never simulated. Remaining for this item: reply depth needs the LLM composer creds (deploy-bound).
  • Cross-device handoff, offline/PWA behavior, notifications delivery. (Email delivery VERIFIED end-to-end 2026-06-12): the verification-email sender is honest fail-closed (null without OSHUN_MESSAGING_EMAIL_FROM + OSHUN_WEB_BASE_URL + a transport), and dev never configured it — Mailpit shipped in the compose file but had received zero emails ever. With the four keys in the root .env (now created; documented in .env.example), the full loop passes: signup → real SMTP → Mailpit → token from the actual email → /auth/verify-email page in a real browser → confirm → emailVerified: true on the next login. Offline/PWA behavior was verified earlier today (SW member-data purge; mobile offline caches now tested through the real download pipeline). Cross-device handoff pass DONE (2026-06-12): the home multi-device continuity panel advertised watch complications and wearable glance cues in the present tense — no OSHUN watch surface exists (V1/features.md lists it as platform-dependent scope; the only wearable app is Iris, health-domain) — those cards are replaced with the REAL capability (profile/preferences/library sync across signed-in surfaces) next to the live sync-status signal, browser-verified. The pass also surfaced a real continuity bug: the theme-continuity strategy/keyword maps were keyed on the pre-canonicalization theme vocabulary, so guidance silently never fired for canonical themes ('breath', 'presence', …) — rewritten onto the canonical 15-theme vocabulary, and 12 stale home-suite tests (old theme values + one fabricated-IEA-briefing heading expectation) realigned to honest behavior; home suite 31 files / 257 tests green. Remaining: push-notification delivery (device tokens deploy-bound; mobile registration dedup fixed today).
  • Remaining exit criteria: admin/tenant operability, agentic studio governance, public web/distribution readiness, observability/runbooks. (Public web/distribution spot-check DONE 2026-06-12): robots.txt 200 (now also disallowing /v1/, /studio, /operator, /workspace, /library — the consoles and the newly same-origin API were crawlable; /studio/compose already self-declares noindex), sitemap.xml 200 with 17 public URLs and no protected surfaces, manifest.json 200, full OG/twitter meta + rendering opengraph-image on /welcome, landing 200. (Observability/runbooks inventory DONE 2026-06-12): 15 incident runbooks in docs/runbooks (assistant/ grounding/provenance failure, privacy incident, provider failover, model/ persona rollback, backups), 60 studio runbooks under docs/releases/v1/ runbooks, docs/operations covers SLOs + on-call + alert-routing + log-retention, the compose observability profile ships Prometheus/ Grafana/Jaeger, and the BFF serves real Prometheus metrics at /metrics (oshun*bff_requests_total verified live). *(Admin/tenant operability walkthrough DONE 2026-06-12): all 8 /operator surfaces render with zero page errors; six consoles (tenants, incidents, audit, SSO, personas, LMS) read live /v1 endpoints and render their designed member-vs-admin-scope registers. The four fixture-backed consoles (index, admin inbox, studio review, single-tenant view — invented incident counts, a fictional "247 members" roll) rendered fabricated operations data with full authority to any signed-in member; they now carry an OperatorPreviewBanner register (operator layout, browser-verified on fixture vs live segments) until each is wired to its BFF endpoint. (Agentic studio governance verified 2026-06-12): the generation studio is entitlement-gated at the edge — /studio/isis/ sits behind the §24.11 boundary in proxy.ts (members without the generation tier get the designed hard-block 404 / upgrade CTA; live-verified as a member), the review-approval and launch-readiness governance consoles are live BFF-backed with admin-scope registers, the Isis provenance ledger enforces chronological append + checksum integrity (tests realigned today), and the paper trail exists (governance ADRs 0044/0052, docs/compliance/governance-reviews, model/ persona rollback + provenance-failure runbooks). With this, every P3 line item in this audit has at least a verified first pass; the remaining open work is deploy-bound (push tokens, LLM composer creds, content ingestion) or explicitly deferred (interactive mobile simulator walkthrough — the expo-web proxy was attempted and is non-viable: dev mode crashes inside Expo's own HMR client on a monorepo pretty-format version mismatch, and expo export --platform web exits 0 while emitting no bundle; web is not a V1 mobile surface, so the walkthrough genuinely requires the iOS/Android simulator, which this 16 GB host cannot safely run alongside the dev stack. Mobile coverage stands on the static pass: tsc clean, 205 suites / 1,040 tests, adversarial scan, and the four defects fixed today). (Full studio sweep DONE 2026-06-12): all 475 static studio routes swept in a real browser against the production build (batches of 50, concurrency 1, fresh chromium per batch, memory steady at ~64% free): 387 rendered with zero page errors / zero 5xx / zero nav failures, and all 88 /studio/isis/_ routes returned the designed §24.11 entitlement boundary with ZERO leaks. The 11 dynamic [caseId] routes need real case fixtures and were recorded as skipped. The sweep also caught a BFF boot breakage (noble-hashes v2 subpath in three isis provider files, committed separately).

Formally open backlog#

  • V1/TODOS.md §23.1.1 — crypto payments (Aje promotion + V1 bridge scaffolding): the single unchecked task in the execution backlog.

Fresh product walkthrough + V1 exit-criteria scorecard (second pass, same day)#

Method. A second fresh-eyes walkthrough against the PRODUCTION build: cookie consent ("Essential only") → signup → all 10 onboarding steps clicked through interactively (exits into the Tara surface) → home → all six hubs → universal search → library → a /domains power surface → ⌘K command palette — every screen captured and reviewed visually, then judged against V1/features.md §"V1 Exit Criteria".

Found and fixed in this pass:

  1. Fabricated outage on every member's home — the admin communications seed shipped an ACTIVE "Voice playback is degraded in us-east-1" warning banner + open public incident; fictional operational status rendered as fact to every member (and contradicted the same seed's all-'operational' components map). The seed now carries no active banner and the example incident is resolved-at-seed (invisible to members; admin/test fixture retained). Live-verified gone.
  2. Real-teacher leftovers — "Read the passage Sam cites" (Tara hub Nisaba cue) and "Sam is saying" (mobile sit player) survived the Sam-Harris→Saro-Imani persona replacement. Fixed; live-verified.
  3. Metis dead affordance — the "Open assessment" card rendered enabled Begin/Rubric buttons against "No assessment is open." with em-dash placeholder metadata. The model now carries an explicit open: false and the card renders the honest register ("Assessments open as your course weeks complete.") with no actions. Live-verified.

Consumer exit-criteria verdicts (each observed directly this pass):

  • One coherent product across six domains — ✅ unified OSHUN brand, one design language across every hub, consistent top-nav register with "← Home", consistent honest-absence registers.
  • Tara visibly the product center — ✅ onboarding frames "Tara sets the spine" and exits into Tara; home leads with the daypart anchor; Tara hub is the richest room (course, teachers, breath ritual).
  • Nisaba and Metis fully integrated, not merely routes — ✅ Nisaba is a real reading desk (sutta, sections, marginalia, glossary) cross-linked from Tara and Nyx; Metis courses are taught "WITH NYX"/"WITH NISABA" and its tutor declares grounding + AI disclosure inline.
  • Sophia grounding visible where evidence matters — ✅ Veritas lead carries a method note explaining the empty surface; the Metis tutor banner reads "AI TUTOR · GROUNDED IN 4 SOURCES · DISCLOSED"; Tara's curated rail discloses "BY EDITOR + SIGNAL".
  • Cross-domain continuity useful and credible — ✅ Tara→Nisaba passage cue, Nyx→Nisaba canon link, Metis cross-domain courses, home continuation rails; continuity threads tested (home suite 257 green).
  • Degrades safely — ✅ honest registers observed on every empty/outage surface this pass (Veritas no-lead, Metis first-run, assistant outage notices), and the §24.11 entitlement boundary verified across all 88 gated routes.
  • Public web ready — ✅ welcome/landing OG complete, robots/sitemap/ manifest verified, status page serves only real (now-unfabricated) incidents.

The operator/tenant/studio/agentic criteria carry their evidence in the sections above (operator consoles, 475-route sweep, governance pass). The two criteria that remain deploy-bound are unchanged: editorial/audio content ingestion, and provider credentials (push, LLM composer).