# V1 Residual Audit — Slice 13: Foundations, Design System, Analytics/Observability, Testing/QA, Localization, Launch + Exit Criteria Date: 2026-06-11 · Spec: `V1/features.md` lines 5983–6242 · Auditor scope: RESIDUAL after the 2026-06-10 ground-truth audit (all 57 backlog tasks landed by 2026-06-11). Method: static source reading + grep only (no builds/tests run). Every claim below was verified against current code in this session; backlog task text was used only as a map, never as proof. --- ## Part 1 — Findings ### F1. The C11 analytics loop is closed on paper only: the ONLY module wired to the real transport is itself orphaned; every LIVE customer telemetry module still discards **Severity:** P0-STRUCT **Evidence:** - `apps/oshun/web/src/analytics/transport.ts:17` — `createSharedAnalyticsSinks()` (real `createBufferedSink` → `POST /v1/analytics/events`). Grep for consumers: `createSharedAnalyticsSinks` appears in exactly ONE non-test module — `taraAnalytics.ts:394`. - `taraAnalytics.ts` itself has **zero** importers: grep for `analytics/taraAnalytics`, `useTaraAnalytics`, `trackMeditationStarted`, `trackOnboardingCompleted` across `apps/oshun/web/src/{components,app,lib}` → 0 hits outside the analytics dir. - The telemetry modules that ARE imported by live components all still console-discard. Census of the 58 non-test modules in `apps/oshun/web/src/analytics/`: - **11 modules have component importers.** 10 of them use a local `console.info('[oshun-analytics]', …)` sink with the literal comment _"Temporary local sink until production telemetry transport is connected"_: `activityReentryTelemetry.ts:94-96`, `domainLaunchTelemetry.ts:16-19`, `homeContinuationTelemetry.ts:159-160`, `librarySaveTelemetry.ts:26-28` (7 importers), `nisabaStudyTelemetry.ts:139-140`, `publicAuthFunnelTelemetry.ts:122-123`, `recommendationTelemetry.ts:123-124`, `searchResultTelemetry.ts:61-62`, `shellNavigationTelemetry.ts:6-9`, `veritasWorkspaceTelemetry.ts:13`. - The 11th, `studioWorkspaceMountTelemetry.ts:67-159`, transmits via its own pre-C11 `BffTelemetrySink` to `/v1/studio/telemetry/workspace-events` (route real: `apps/oshun/bff/src/routes/studio-workspace-telemetry.ts:37`). **Spec promise:** features.md 6113-6118 — customer event taxonomy for activation, retention, domain navigation, home, search, library, onboarding; assistant taxonomy. **What the code actually does:** The BFF ingest (`apps/oshun/bff/src/analytics/events-ingest.ts`) and the shared transport are real, well-built code — but in the running product, **zero events from the customer activation/retention/search/library/auth-funnel taxonomy ever reach them.** The only web telemetry that transmits anywhere is the studio workspace-mount module (3 event kinds, to a different endpoint). The 06-10 headline "68/69 web telemetry modules never transmit" is now "57/58 never transmit, and the 1 flipped module has no callers." **Fix sketch:** Swap the 10 live modules' local console sinks for `createSharedAnalyticsSinks()` (one-line change each — the shapes already match), and either wire `useTaraAnalytics()` into the Tara session player/onboarding components or delete taraAnalytics. --- ### F2. 28 telemetry modules route through a singleton that is never initialized → silent no-op discard in every environment **Severity:** P1 **Evidence:** - `libs/oshun/analytics/src/singleton.ts:30-36,52-56` — `getOshunAnalytics()` returns `lazyNoOpClient()` whose sink body is `// Silently discard when analytics is not yet initialized.` unless `initOshunAnalytics()` was called. - Grep for `initOshunAnalytics` across `apps/` and `libs/` (excluding tests): the ONLY hit is the definition in `singleton.ts` itself. **No app ever initializes it.** - 28 web modules depend on it (`studioPerformanceBudgetsTelemetry.ts:62-74`, `studioExperimentationFeatureFlagsTelemetry.ts`, `studioObservabilityOperationalDashboardsTelemetry.ts`, `studioAccessibilityGovernanceTelemetry.ts`, all `studioBellona*` bridge telemetry, etc.). Their fallback (`studioPerformanceBudgetsTelemetry.ts:49-55`) only console-warns in development _if the dynamic import fails_ — in production every event is swallowed without a trace. **Spec promise:** features.md 6113-6120 (admin + studio event taxonomies). **What the code actually does:** A worse variant of the console-sink modules: not even console output. These are silent-discard façades in dev AND prod. **Fix sketch:** Call `initOshunAnalytics({ context, sinks: createSharedAnalyticsSinks() })` once in `lib/providers.tsx` (next to `ExperimentationBootstrap`). One call un-façades all 28 modules. (Most are also orphaned — see F11 — so triage which to keep first.) --- ### F3. The entire `apps/oshun/web/src/observability/` directory is dead: never initialized, and posts to five BFF routes that do not exist **Severity:** P1 **Evidence:** - `performance-telemetry.ts:138,177,217` posts to `/v1/web/telemetry/startup`, `/v1/web/telemetry/route-transition`, `/v1/web/telemetry/diagnostics`; `error-monitoring.ts:291,309` posts to `/v1/web/errors`, `/v1/web/vitals`; `crash-reporting.ts:203,219` posts to `/v1/web/crashes`, `/v1/web/diagnostics`. - Grep `v1/web` across `apps/oshun/bff/src`: **zero route registrations.** All seven endpoints 404. - Grep for `initPerformanceTelemetry`, `initWebErrorMonitoring`, `initializeCrashReporting` in web `src/`: zero callers (web has no `instrumentation.ts`; `app/error.tsx:26` merely mentions the module in a comment). **Spec promise:** features.md 6121-6126 (service-health dashboards, structured tracing) and 6240-6242 (post-deploy monitors as a launch gate). **What the code actually does:** Web crash reporting, error monitoring, web-vitals reporting, and startup/route-transition performance telemetry are all non-functional twice over: nothing starts them, and their fetch targets don't exist. The `catch {}` in each `postTelemetry` would swallow the 404s even if they were started. **Fix sketch:** Add BFF receiver routes (or point them at the C11 ingest envelope), then initialize from a client-root effect. Or delete the directory and route errors through the C11 ingest. --- ### F4. Mobile crash reporting IS initialized — and posts every crash into a 404 **Severity:** P1 **Evidence:** - `apps/oshun/mobile/app/_layout.tsx:37,130` calls `initializeCrashReporting(...)`. - `apps/oshun/mobile/src/observability/crash-reporting.ts:290,309` posts to `/v1/mobile/crashes` and `/v1/mobile/diagnostics`. - Grep `mobile/crashes|mobile/diagnostics` in `apps/oshun/bff/src`: zero routes. **Spec promise:** features.md 6148-6150 (mobile coverage incl. retry/diagnostics), 6240-6242. **What the code actually does:** Real crashes on real devices are serialized, POSTed, 404'd, and silently dropped. This is façade telemetry on the live mobile path — the one place a crash report matters most. **Fix sketch:** Register `POST /v1/mobile/crashes` + `/v1/mobile/diagnostics` in the BFF backed by a durable snapshot store (the C11 ingest pattern is directly reusable), with an operator read. --- ### F5. Localization is a façade: 8 launch languages and a polished switcher over a hardcoded-`en`, zero-consumer message system **Severity:** P1 (with a UX-dishonesty component) **Evidence:** - `libs/oshun/i18n/src/index.ts:40-50,99` — `OSHUN_LAUNCH_LANGUAGE_PREFERENCES` = en, es, fr, de, ar, he, ja, pt; `RTL_LOCALES` = ar, he. - `apps/oshun/web/src/i18n/messages/` contains ONLY `en/` — 5 namespace files, **208 lines total** for a 684-route app. - `apps/oshun/web/src/i18n/request.ts:5` — `const locale = defaultLocale;` — the next-intl request config **hardcodes `en`**; the user's stored preference is never consulted for messages. - Grep `useTranslations|getTranslations|NextIntlClientProvider` across web `src/` excluding `src/i18n/`: **zero hits.** Not one component renders a translated string. - `components/LanguageSwitcher.tsx:26-35,128-139` — a full dropdown with flags for all 8 locales that persists the selection (`savePreferences({ language: loc })`) and flips document direction (`locale-store.ts:8 applyLocaleToDocument`) — while every visible string stays English. **Spec promise:** features.md 6181-6184 — "Locale architecture, externalized strings, …, RTL and text-expansion QA, and launch locale coverage for customer and admin journeys." **What the code actually does:** A member who selects Arabic gets an RTL document direction over 100% English hardcoded strings. The locale machinery (config, store, formatting, preference sync) is real; the localization itself does not exist, and the switcher implies it does. No deferral for i18n is recorded in the 06-10 audit backlog. **Fix sketch:** Either (a) honest-scope it: ship the switcher as "English (more languages coming)" with non-en options disabled, or (b) thread the stored locale into `request.ts`, externalize shell-level strings, and produce real catalogs for the declared launch locales. --- ### F6. The public /status page renders hardcoded specimen data while the live E8 status API has zero consumers **Severity:** P1 **Evidence:** - `apps/oshun/web/src/app/status/page.tsx:3,13` renders `StatusPage` from `components/lilith/system-pages.tsx`, whose data is the hardcoded `STATUS_COMPONENTS` array (`system-pages.tsx:545`), fixture incidents (`:564-566` — "Fixture · Tara · CDN edge…"), and a pill that literally reads `Specimen · fixture data` (`:601`). - The real route `GET /v1/status` (`apps/oshun/bff/src/routes/customer-communications.ts:149`, E8: live-derived component states, operator overrides win) has **zero consumers**: grep `/v1/status` across all apps excluding the BFF → only an unrelated `lilith/svc-analytics` doc route. **Spec promise:** features.md 6230-6231 (exit criterion: "the public status page [is] ready for launch") and 6031-6032 (status plumbing "bound to component health checks"). **What the code actually does:** The page is honest (it labels itself specimen), so this is not result-faking — but the launch-facing status page shows fake components/incidents and the genuinely live status derivation built for E8 is a dead route. **Fix sketch:** Make `StatusPage` fetch `GET /v1/status` (server component fetch is fine — the route is public) and render the real components/incidents/maintenance; keep the specimen behind a dev query param like the banner catalog. --- ### F7. Outbound webhook plumbing: authored webhooks are never dispatched by any real event **Severity:** P1 **Evidence:** - The only "delivery" code is `apps/oshun/bff/src/studio/webhook-delivery-store.ts:1-12` — a pure, deterministic **classifier** of already-known response statuses ("0 Date.now/random"), feeding the admin console route. It performs no HTTP. - Grep `dispatchWebhook|deliverWebhook|webhook.*fetch(` across `apps/oshun/bff/src`: no outbound dispatcher exists. Webhook/API-key records live in the integrations registry stores (`admin/admin-integrations-registry-store.ts`) and tenant console, with no event→delivery wire. **Spec promise:** features.md 6016-6018 — "Webhook and event-bus plumbing for outbound notifications, with topic registry, schema versioning, signing, retry/backoff, dead-letter routing, replay, and delivery audit." **What the code actually does:** Tenant/operator-authored webhooks are stored configuration that no runtime event ever fires. This is the surviving half of the 06-10 "management without enforcement" headline (#5) that the backlog never tasked. **Fix sketch:** A small dispatcher worker: subscribe to the existing in-BFF event seams (feedback intake, generation release, incident publish), sign + POST to registered endpoints, and feed real outcomes through the existing classifier/backoff store. --- ### F8. Stored API keys are never validated on any inbound request **Severity:** P1 **Evidence:** - API keys exist only as managed records: `apps/oshun/bff/src/admin/admin-integrations-registry-store.ts` / `routes/admin-integrations-registry.ts` (all handlers are admin-console CRUD, preHandlers `[abuseProtection, authProtection]`). - Grep `x-api-key|validateApiKey|apiKeyAuth` across `apps/oshun/bff/src` (and middleware/): zero. No request path on the BFF accepts or checks an API key. **Spec promise:** features.md 6012-6015 — "scoped tokens, …, rate limits, per-key quotas, audit" for the public API platform; exit criterion 6220-6223 (Tenant Console "API keys"). **What the code actually does:** The tenant-console "API keys" surface mints credentials that grant nothing and guard nothing — stored-config-never-enforced. (Unchanged from the 06-10 headline; no backlog task covered it.) **Fix sketch:** Either remove/disable the key-issuing surface for V1 (honest), or add an `x-api-key` preHandler that resolves keys to tenant+scopes for a defined public-API route subset. --- ### F9. Contracts still sit beside the routes, not under them **Severity:** P1 **Evidence (counts from this session):** - `apps/oshun/bff/src/routes/` has **487** route files; **31** import `@oshun/contracts`; **44** call `.parse(`/`.safeParse(`; **12** import zod directly. The overwhelming majority hand-roll `typeof`/shape checks (e.g. `routes/feature-flags.ts:107-224` — 120 lines of manual parsing) or validate nothing. - Even the new C11 ingest hand-rolls (`analytics/events-ingest.ts:99-119` — `typeof` checks per field) although the analytics library ships `validateEventEnvelope` (`libs/oshun/analytics/src/validation.ts:108`) and a full event-name taxonomy it never consults. - `libs/contracts/src/` is a large, real schema tree (common/, events/, per-domain) — with `contracts.spec.ts` style self-tests but thin route-side enforcement. **Spec promise:** features.md 6058-6059 — "Object-property authorization, excessive-data-exposure, and mass-assignment protections **enforced through canonical Zod request/response shapes**"; 6002-6005 (response normalization, contract tests). **What the code actually does:** The contract library exists and drifts checked against itself, but ~94% of route files don't bind it to their payloads. Hand-rolled parsing is mostly careful, but response-shape (excessive-data-exposure) protection is uniformly absent as a systematic guarantee. **Fix sketch:** Adopt `fastify-type-provider-zod` (or a thin `parseWith(schema)` helper) and migrate routes in tranches starting with mutation routes; add a lint rule banning untyped `request.body` access in new routes. --- ### F10. No OpenTelemetry / distributed tracing anywhere on the Oshun runtime path; tracing manifest has zero consumers **Severity:** P2 **Evidence:** - Grep `opentelemetry` in `apps/oshun/bff` (src + package.json): zero. OTel deps exist only in `apps/lilith/svc-media/package.json:31-37` and `libs/iris/core` (its own tracing module) — neither is imported by the Oshun BFF or web. - The only trace awareness in the BFF is `middleware/residency-guard.ts:200-202` reading an inbound `traceparent` header into an audit field. Logging is the Fastify default (`app.ts:557 logger: true`). - `libs/oshun/analytics/src/tracing-manifest.ts` (plus `alerts-manifest.ts`, `incident-ownership-manifest.ts`): grep across the repo — zero consumers outside the library. **Spec promise:** features.md 6125-6126 — "Structured tracing across shell, BFF, Sophia, Iris, Psyche, Lilith, Isis, Metis, review, support, and admin workflows." **What the code actually does:** Request-scoped pino logs + a Prometheus request counter (C11, real — `app.ts:706-720`). No spans, no propagation, no exporter. The tracing manifest is documentation-as-code with no enforcement seam (unlike the journey-coverage manifests, which DO verify spec files exist — `__tests__/web-playwright-journey-coverage.test.ts:32`). **Fix sketch:** `@opentelemetry/sdk-trace-node` + fastify instrumentation in the BFF behind an env-gated exporter; propagate `traceparent` to domain-service adapters. Or record an explicit V1.x deferral — today the spec sentence is simply unmet. --- ### F11. 47 of 58 web telemetry modules are orphaned dead code; dashboard/evaluation/experimentation manifests have zero consumers **Severity:** P2 **Evidence:** - Importer census (this session): of 58 non-test modules in `apps/oshun/web/src/analytics/`, **47 have zero importers** outside the analytics dir (all `studio*Telemetry` flow-span modules, `taraAnalytics`, `taraExperiments`). 11 are imported (F1 lists them). - `libs/oshun/analytics/src/dashboards-{customer-kpi,nisaba,assistant,service-health,queue-health, operational-readiness,satisfaction}.ts`, `evaluation-manifest.ts`, `experimentation-manifest.ts`: zero consumers anywhere outside the library (grep across `apps/` + `libs/`). The only symbols any app imports from `@oshun/analytics` are the client/sink/singleton ones. - The operator read `GET /v1/admin/analytics/events` (`events-ingest.ts:127`) also has **no UI consumer** — no admin console fetches it. **Spec promise:** features.md 6121-6124 — "Dashboards for activation, retention, cross-domain use, … service health, queue health, … launch readiness." **What the code actually does:** The dashboard layer of the spec exists as typed manifest objects that nothing renders, queries, or enforces. There is no activation/retention dashboard a human can open. **Fix sketch:** One admin "Analytics" console reading `/v1/admin/analytics/events` facets + `/metrics`-derived route health would make the C11 ingest observable; delete or enforcement-test the manifests like the journey-coverage ones. --- ### F12. Feature-flag/experimentation machinery runs end-to-end but gates nothing; all experiment definitions are at 0% rollout **Severity:** P2 **Evidence:** - Server: `apps/oshun/bff/src/feature-flags/tenant-feature-flags.ts:129` defines real flags (`shellDomainConfiguration`, `domainSwitcher`, `quickActionsTray`, `homeLayoutV2`, …) with a real deterministic bucketing evaluator; the experiment-bearing definitions carry `rolloutPercentage: 0` at lines 197, 213, 229, 242, 258. Env overrides exist (`OSHUN_TENANT_FEATURE_FLAG_OVERRIDES`, line 319). Route `routes/feature-flags.ts:63` is live. - Client: `experimentation/bootstrap.tsx:73-93` initializes + refreshes flags on every pathname/ profile change from `lib/providers.tsx:12`. Hooks exist (`experimentation/hooks.ts:20,37`). - Consumers: grep `useFeatureFlag|useFeatureFlagValue` outside `src/experimentation/`: **zero.** Not one component branches on a flag. - `analytics/taraExperiments.tsx` (FNV-1a deterministic assignment, exposure/conversion tracking, `useTaraExperiment`): zero importers. **Spec promise:** features.md 6127-6128 — "Experiment support for recommendation ranking, assistant presentation, disclosure presentation, and home/dashboard composition"; 6026-6027 tenant-aware flags/experiment scoping; 6051 "validated configuration for feature flags … experiment guardrails." **What the code actually does:** A flag evaluation round-trip fires on every page navigation (a real network cost) and the result is read by nobody. Experimentation capability: present; experimentation: nonexistent. Not result-faking (nothing claims an experiment ran) — orphaned machinery + wasted hot path. **Fix sketch:** Wire `homeLayoutV2`/`domainSwitcher` into their components as the first real gates, or stop bootstrapping flags on every navigation until a consumer exists. --- ### F13. `GET /metrics` is unauthenticated and enumerates the full route inventory **Severity:** P2 **Evidence:** `apps/oshun/bff/src/app.ts:717-720` — `app.get('/metrics', …)` with no preHandler; `prometheusMetrics()` emits per-route counters (route template names incl. `/v1/admin/*` paths, status codes, latencies) for anything that has handled a request. **Spec promise:** features.md 6071-6072 — "API inventory and shadow-endpoint discovery checks for deprecated, preview, internal, and admin-only routes" (the spec treats route inventory as sensitive); 6068-6070 resource-consumption controls. **What the code actually does:** Any anonymous caller can scrape admin route names, traffic volumes, and error rates. Acceptable on a private network; this BFF also serves the public web. **Fix sketch:** Gate `/metrics` behind the observability admin scope or a static bearer from env (`OSHUN_METRICS_TOKEN`), defaulting closed in production. --- ### F14. Anonymous flooding can evict the entire 5000-event analytics retention ring **Severity:** P2 **Evidence:** `apps/oshun/bff/src/analytics/events-ingest.ts:87-123` — `POST /v1/analytics/events` is sessionless by design (documented), accepts 100 events/request, `unshift`s and prunes to `RETENTION_MAX = 5000`. Only the generic abuse-protection limiter stands between an attacker and 50 requests that erase all real telemetry (and the durable snapshot persists the junk). **Spec promise:** features.md 6068-6070 — resource-consumption and abuse controls for analytics APIs. **What the code actually does:** Honest short-retention design, but eviction-by-write means the operator read can be blinded cheaply. Low stakes today (nothing consumes the data — F11), higher once dashboards exist. **Fix sketch:** Partition retention per platform/origin, or require a session for non-anonymous event names, or rate-limit by event volume not request count. --- ### F15. OpenAPI documents ~91 paths of a 487-route-file surface **Severity:** P2 **Evidence:** `apps/oshun/bff/openapi/oshun-bff.openapi.yaml` (25,612 lines, ~91 `/v1/` paths). The contract tests check the documented subset exists at runtime (`__tests__/contract/openapi-runtime-drift.test.ts`) and that a hand-picked "critical" list is documented (`openapi-coverage.test.ts:113-126`) — there is no inverse check, so hundreds of live routes (incl. the new C11/C8/E8 routes) are undocumented and undetectable as drift. **Spec promise:** features.md 6002-6005 — "OpenAPI coverage … schema drift checks." **What the code actually does:** One-directional drift checking over a fraction of the surface. **Fix sketch:** Add the inverse assertion (enumerate `app` routes, allowlist intentionally undocumented ones) so new routes must either be documented or explicitly waived. --- ### F16. BFF `telemetry/` modules (search + recommendation) — verify their feed **Severity:** P2 (downgraded observation) **Evidence:** `apps/oshun/bff/src/telemetry/{search-telemetry.ts,cross-domain-recommendation-telemetry.ts}` exist server-side, but their client-side feeders (`searchResultTelemetry`, `recommendationTelemetry`) are console-only (F1) — the spec's "recommendation **signal ingestion**" (6232-6233) therefore has no live client signal. The recommendation scoring itself is real (`recommendations/scoring.ts` consumed by `routes/recommendations.ts`), and search offline evals exist (`search/offline-eval-route.ts`, `libs/oshun/search-discovery/src/evals/offline-evals.ts`). **Fix sketch:** Falls out of F1 — flipping `searchResultTelemetry`/`recommendationTelemetry` to the shared transport gives the ingestion leg its first real signal. --- ## Part 2 — What is genuinely solid in this slice (verified, not assumed) - **C11 server side is real:** durable capped ingest with prune-on-write (`events-ingest.ts:64-67`), scope-gated operator read with facets, and a genuinely registered `BffMetricsCollector` — `app.ts:707-716` records every response via an `onResponse` hook and `/metrics` serves computed Prometheus text. The library `BufferedAnalyticsSink` (`libs/oshun/analytics/src/buffered-sink.ts`) is a real batching/retry/backoff HTTP sink. - **Accessibility posture is strong:** `e2e/support/accessibility.ts` runs `AxeBuilder` with WCAG 2.0/2.1/2.2 A+AA tags; 10 spec files consume it (`shell-route-accessibility.spec.ts` hard-fails on any violation at lines 142-143; `wcag-aa-signoff-*`, assistant/nisaba/search/tara a11y specs); `prefers-reduced-motion` honored in `globals.css` and player/assistant components; `a11y-pa11y-nightly.yml` + `accessibility.yml` in CI. - **Performance budgets are enforced:** `apps/oshun/web/lighthouserc.cjs` asserts numeric budgets (perf ≥0.88, FCP ≤1800ms, LCP ≤2500ms, TBT ≤250ms, CLS ≤0.1) over authenticated critical routes, with `lighthouse-budget.json` route budgets, wired to PR/push in `oshun-web-lighthouse.yml` (+ admin/metis variants, `oshun-web-bundle-monitor.yml`). - **E7 design-system decision is real and enforced:** `src/design-system/__tests__/lilith-palette-regression.test.ts` literally diffs the TS palette hexes against `lilith.css` custom properties. - **e2e/CI discipline holds:** 254 customer + 76 admin Playwright specs + mobile Maestro flows; journey-coverage manifests are enforcement-tested against real spec files (`web-playwright-journey-coverage.test.ts:32` `existsSync` per entry); CodeQL, scheduled authenticated DAST (`security-dast.yml`, daily vs staging), SBOM/signing, secret scanning, `stub-indicator-scan.yml`, `test-coherence-check.yml` all present as real workflows. - **Docs/runbooks/launch material exist as deliverables:** `docs/runbooks/` covers the spec's named runbooks (shell-outage, grounding-failure, assistant-failure, provider-failover, moderation-surge, privacy-incident, model-workflow-rollback, persona-rollback, provenance-failure); `docs/launch/` has go-no-go, canary-analysis, post-deploy-monitoring, critical-journeys, locale-coverage, exit-criteria-signoff, private-beta, dogfood-drill; `docs/training/` exists. (Documents verified present; their operational truth is inherently a deploy-time question.) - **Feature-flag evaluator and kill-switch enforcement are real server code** (F12 is about consumers, not fakery): `agentic/runs-route.ts` maps operator-armed kill switches into the execution guard (A5), and the flag evaluator does deterministic bucketing. --- ## Part 3 — V1 Exit Criteria scorecard (features.md 6199-6242) Verdicts: **MET / PARTIAL / UNMET**, each from code read in this session plus the 06-10 audit's completed-task evidence where cross-slice. | # | Criterion (line) | Verdict | Evidence | | --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | One coherent product across Tara/Arete/Veritas/Nyx/Nisaba/Metis (6203-04) | **PARTIAL** | Shell + hub rooms real/honest; but the `/domains/*` depth layer remains simulation-backed, now explicitly labeled via `DomainPreviewBanner` + `DOMAIN_PREVIEW_SURFACES` (D1, commit 9a14916848). Honest, not coherent-real. | | 2 | Tara visibly the product center (6205) | **MET** | `libs/oshun/domain-registry/src/registry.ts:493` `OSHUN_SHELL_PRIMARY_DOMAIN = 'tara'`; Tara per-member data real (C3: today/sittings/ritual; B9 analytics from real completions; D2 ambient audio). | | 3 | Nisaba + Metis fully integrated, not routes (6206) | **PARTIAL** | Real ingest core now exists (`metis/ingest-pipeline.ts` — deterministic outline derivation, honest `awaiting_source_content`); D6 seeded an honest Nisaba corpus; BYOM operator decisions persisted (`metis-byom-decision-store.ts`). But the deep engines (IRT 3PL, collation/philology, gradebook) remain library-side with the BFF not consuming them, and metis depth surfaces are preview-labeled. | | 4 | Sophia grounding visible wherever evidence matters (6207) | **PARTIAL** | Fabricated grounding eradicated (B5/B6/B15 — live Telegram webhook grounds in real sources; teacher lineage honestly "not yet verified"); explainer journey carries evidence pins. Answer generation itself is LLM+corpus deploy-bound (recorded in V1_DEPLOYMENT_REQUIREMENTS). Visible-where-it-exists: yes; everywhere evidence matters: no. | | 5 | Cross-domain continuity useful, credible, tested (6208) | **MET (partial credibility)** | C15 rewired activity/explore off fixtures to live reads; C16 mobile home/handoff real; `e2e/activity-cross-domain-continuity.spec.ts` exists against the real BFF. | | 6 | Admin web + mobile operate ALL listed workflows (6209-11) | **PARTIAL** | 76 admin e2e specs; operator legs verified real in the 06-10 audit; C13 mounted the admin-mobile review action sheet; C9 agentic ops console reads real run lifecycle. "All … workflows" is still over-claimed: several governance cascades lack native triggers (per the production-readiness memory) and admin-mobile covers urgent queues, not the full matrix. | | 7 | Oshun Studio: authoring/calendar/asset/taxonomy/versioning/localization/collab + release-gate enforcement (6212-15) | **UNMET** | The 06-10 verdict "no persisted authored artifact; lanes are stateless analyzers" was only fixed for communications artifacts (C8: banners/help/templates/status authored→consumed). `@oshun/studio-authoring` (~30 modules) remains ~1-consumed; no editorial-calendar/versioning/localization authoring persistence wired. No backlog task existed. | | 8 | Agentic AI Studio operational w/ governance gates + tenant budgets (6216-19) | **PARTIAL** | Server-side kill-switches + budgets now enforced at `/v1/agentic/runs/execute` (A5; `agentic/runs-route.ts:38-59` maps operator-armed switches); runs persisted and bridged into the ops console (C9); customer-facing "Run Now" wired on /arete/review. Replay/gold-sets/capability-audit remain library-side; multi-agent pipelines not runtime-wired. | | 9 | Tenant Console: SSO/SCIM/OneRoster, roles, audit, API keys, webhooks, help, status/banner (6220-23) | **PARTIAL** | C12 real BFF reads (members/invites/SSO/audit, tenant-scoped); B3 honest provisioning; D3 OIDC runtime (SAML deferred per default); OneRoster apply wired durable. **API keys grant nothing (F8) and webhooks never fire (F7)**; help/banner/status authoring is consumed (C8). | | 10 | Iris/Psyche/Lilith/Sophia/Isis/Metis ownership true in product (6224-25) | **PARTIAL** | Iris: real consent-gated durable memory in the assistant path (C4). Lilith: 13-rule crisis policy now runs on every member turn (C5). Isis: gates real at generation release (C6). Psyche: text-WS per D5 default, voice/avatar deferred. Sophia/Metis: partial per #4/#3. | | 11 | External model sources cannot bypass review/rights/safety/provenance/Isis gates (6226-27) | **MET (at the wired paths)** | C6: outputs catalog/lineage/provenance written ONLY on Isis gate `complete`; blocked outputs never cataloged (tested); Civitai intake is review-gated; BYOM decisions audited. No in-BFF path was found that releases an external model around the gate. | | 12 | Metis high-stakes outputs emit claim-level evidence + pass grounding/pedagogy/safety/rights/integrity/standards/drift gates (6228-29) | **UNMET** | `apps/oshun/bff/src/metis/` contains ingest, BYOM decisions, tutor memory, integrity appeals — no claim-level evidence emission and no runtime gate chain; the pedagogy/safety harness lives un-consumed in `libs/metis/*`. No deferral recorded. | | 13 | Public website, app store listings, deep links, public status page ready (6230-31) | **PARTIAL** | Landing/public entry + canonical deep-link e2e specs real; `apps/oshun/mobile/store/metadata.json` exists with submission on the DEPLOY register (recorded); **the public status page is specimen fixture data while the live API goes unconsumed (F6)**. | | 14 | Recommendation candidate gen, signal ingestion, offline eval, observability operational (6232-33) | **PARTIAL** | Candidate generation + scoring real (`recommendations/scoring.ts` ← `routes/recommendations.ts`); offline evals exist for search (`search/offline-eval-route.ts`); but client signal ingestion is console-only (F1/F16) and recs observability has no dashboard (F11). `@oshun/search-discovery` retirement-to-V1.x is a recorded decision (E5) — not double-counted here. | | 15 | Aja embodied-instruction services ready for Metis or explicitly deferred with approved scope (6234-35) | **UNMET (as stated)** | `apps/oshun/bff/src/aja/` is a large real store surface (75+ modules) but has zero linkage from `apps/oshun/bff/src/metis/` (grep: no aja import), and no explicit deferral with approved scope boundaries exists in the 06-10 audit or deployment requirements. The criterion's escape hatch ("explicitly deferred") was never exercised. | | 16 | All critical workflows have risk-tiered automated verification (6236-39) | **PARTIAL→MET** | 254 customer + 76 admin specs over the real BFF; journey/coverage manifests enforcement-tested against real spec files; `OSHUN_V1_COVERAGE_GAP_MATRIX` maps journeys→required layers (`qa-governance-and-enforcement.ts:140`); CI blocks via oshun-ci/e2e/lighthouse workflows. "Risk-tiered" exists as matrix metadata rather than an enforced tiering gate — close, with that caveat. | | 17 | Accessibility, performance, security, resilience, **observability**, docs, training, runbooks, canaries, rollback triggers, post-deploy monitors satisfy launch gates (6240-42) | **PARTIAL — observability is the failed leg** | A11y: MET (axe e2e hard-fail + nightly pa11y). Performance: MET (LHCI budgets on PR). Security: MET in CI (CodeQL/DAST/SBOM/secrets) with pen-test/DAST target deploy-bound. Docs/training/runbooks: present. Canary/rollback/post-deploy: documented procedures, runtime monitors deploy-bound. **Observability: UNMET — no live customer telemetry transmits (F1/F2), web+mobile crash/error/vitals reporting dead (F3/F4), no tracing (F10), no dashboards (F11).** | **Scorecard totals: 3 MET · 10 PARTIAL · 4 UNMET (criteria 7, 12, 15, 17-observability-leg; #17 counted PARTIAL overall).** Strictly: MET 3, PARTIAL 11, UNMET 3 if #17 is taken as one PARTIAL. --- ## Severity count table | Severity | Count | Findings | | ---------- | ----- | ---------------------------------------------------------------------------------------------------------------------------------------- | | P0-SEC | 0 | — | | P0-HONESTY | 0 | (no result-faking found in this slice's current code; honest-discard ≠ fabricated success) | | P0-STRUCT | 1 | F1 | | P1 | 8 | F2, F3, F4, F5, F6, F7, F8, F9 | | P2 | 7 | F10, F11, F12, F13, F14, F15, F16 | | UX | 1 | (component of F5 — language switcher implies localization that doesn't exist) | | DEPLOY | 0 new | (app-store submission, DAST staging target, pen-test, post-deploy monitors — all already on the recorded DEPLOY register; not re-raised) | **Total: 16 findings (1 P0-STRUCT, 8 P1, 7 P2, 1 UX-component).**