Date: 2026-05-13. Mandate: V1-DOC-009 (launch-readiness manifest entry
grounding_failure_runbook).
Scope#
Customer Library saved-item rail exposes three categories of trust signals sourced from the Sophia substrate:
- Provenance badge — content has a verifiable lineage record bound through the Sophia evidence model.
- Confidence badge — claim or generated artifact carries a confidence band computed by the grounded-answer pipeline.
- Grounding status — saved items derived from grounded answers expose
groundedorpartially_groundedso customers can tell whether every claim was traceable to a bound source.
This audit doubles as the runbook reference for grounding failures (the on-call
runbook lives at docs/runbooks/grounding-failure.md) by capturing what the
Library surface promises and what failure modes that promise depends on.
Surface contract#
Per-item signals are exposed through stable data-* attributes that the signoff
suite asserts against:
| Surface element | Selector contract |
|---|---|
| Confidence band | [data-confidence-badge] |
| Provenance lineage | [data-provenance-badge] |
| Grounding status badge | [data-library-grounding-badge][data-library-grounding-status="grounded|partially_grounded"] |
| Item-level lineage note | [data-library-item-signal-note] (e.g. "citation lineage") |
| No-signal items (e.g. ritual artifacts) | [data-library-item-signals] collection is empty for non-evidentiary kinds |
The fixture set covers four representative Library items and is locked by the test:
| Fixture id | Domain | Signals |
|---|---|---|
veritas-ai-governance-claim |
Veritas | confidence + provenance + partially_grounded |
veritas-iea-source-packet |
Veritas | citation-lineage note |
nisaba-compassion-passage |
Nisaba | grounded |
tara-evening-reset |
Tara | no signals (ritual artifact) |
Automated coverage#
Locked by apps/oshun/web/e2e/library-provenance-grounding.spec.ts. The spec
seeds a deterministic customer session, navigates to /library, and asserts
each contract row above. The launch-readiness manifest binds this audit to the
spec under the grounding_failure_runbook surface.
Companion specs that exercise the upstream Sophia path:
apps/oshun/web/e2e/assistant-invocation-source-inspection.spec.ts— grounded-answer source-inspection panel.apps/oshun/web/e2e/nyx-event-pages.spec.ts— concept-graph link assertions that cross from Nyx into Nisaba and Tara.
Sophia-side integration tests live under libs/oshun/evidence-sophia/src/
(adapter + canonical adapter + source-set + source-lifecycle suites) and the
libs/sophia/* runtime libraries.
Failure-mode mapping (Library surface → runbook section)#
| Customer-visible failure | Runbook section in docs/runbooks/grounding-failure.md |
|---|---|
Library item is missing the partially_grounded / grounded badge it earned. |
Detection: evidence.confidence.below_threshold; Diagnosis steps 2–3 (retriever trace, manifest comparison). |
Confidence badge shows below min_confidence for >5% of items in a session. |
Severity sev2 row; Mitigation step 2 (kill-switch retriever). |
| Provenance badge resolves to a retracted source. | Severity sev1 row; Mitigation step 1 (disable grounded publication). |
| Citation-lineage note references the wrong source bundle. | Detection: evidence.citation.mismatch_reported; Diagnosis step 1 (cluster mismatches by source ID). |
Customer-reported citation mismatch via evidence.citation.mismatch action. |
Customer signal in detection section; Post-incident: file rubric-failure ticket against bundle owner. |
Gaps observed#
- Saved-item grounding status currently surfaces two values
(
grounded/partially_grounded). A futureungroundedvalue is reserved for Library items whose source binding was revoked post-save; tracked in the Sophia source-lifecycle suite. - The visual confidence band uses the canonical bands from
libs/oshun/analytics/src/customer-taxonomy.ts; localized band labels live inlibs/oshun/i18n/(evidence.confidence.lowbaseline; band copy for medium/high is generated from the same key family).
No gap in this audit blocks launch.
Companion artifacts#
docs/runbooks/grounding-failure.md— on-call runbook.libs/oshun/analytics/src/alerts-manifest.ts—citation-flag-rate,grounding-low-confidence-spike,grounding-fallback.libs/oshun/analytics/src/dashboards-service-health.ts— Sophia panel.libs/oshun/analytics/src/dashboards-assistant.ts— assistant evidence quality panel.libs/oshun/analytics/src/evaluation-manifest.ts—grounded_answersbenchmark scope and thresholds.