Eve library

OSHUN V1 Library Provenance and Grounding Audit

Guide

Revision 1

On this page

Date: 2026-05-13. Mandate: V1-DOC-009 (launch-readiness manifest entry grounding_failure_runbook).

Scope#

Customer Library saved-item rail exposes three categories of trust signals sourced from the Sophia substrate:

  • Provenance badge — content has a verifiable lineage record bound through the Sophia evidence model.
  • Confidence badge — claim or generated artifact carries a confidence band computed by the grounded-answer pipeline.
  • Grounding status — saved items derived from grounded answers expose grounded or partially_grounded so customers can tell whether every claim was traceable to a bound source.

This audit doubles as the runbook reference for grounding failures (the on-call runbook lives at docs/runbooks/grounding-failure.md) by capturing what the Library surface promises and what failure modes that promise depends on.

Surface contract#

Per-item signals are exposed through stable data-* attributes that the signoff suite asserts against:

Surface element Selector contract
Confidence band [data-confidence-badge]
Provenance lineage [data-provenance-badge]
Grounding status badge [data-library-grounding-badge][data-library-grounding-status="grounded|partially_grounded"]
Item-level lineage note [data-library-item-signal-note] (e.g. "citation lineage")
No-signal items (e.g. ritual artifacts) [data-library-item-signals] collection is empty for non-evidentiary kinds

The fixture set covers four representative Library items and is locked by the test:

Fixture id Domain Signals
veritas-ai-governance-claim Veritas confidence + provenance + partially_grounded
veritas-iea-source-packet Veritas citation-lineage note
nisaba-compassion-passage Nisaba grounded
tara-evening-reset Tara no signals (ritual artifact)

Automated coverage#

Locked by apps/oshun/web/e2e/library-provenance-grounding.spec.ts. The spec seeds a deterministic customer session, navigates to /library, and asserts each contract row above. The launch-readiness manifest binds this audit to the spec under the grounding_failure_runbook surface.

Companion specs that exercise the upstream Sophia path:

  • apps/oshun/web/e2e/assistant-invocation-source-inspection.spec.ts — grounded-answer source-inspection panel.
  • apps/oshun/web/e2e/nyx-event-pages.spec.ts — concept-graph link assertions that cross from Nyx into Nisaba and Tara.

Sophia-side integration tests live under libs/oshun/evidence-sophia/src/ (adapter + canonical adapter + source-set + source-lifecycle suites) and the libs/sophia/* runtime libraries.

Failure-mode mapping (Library surface → runbook section)#

Customer-visible failure Runbook section in docs/runbooks/grounding-failure.md
Library item is missing the partially_grounded / grounded badge it earned. Detection: evidence.confidence.below_threshold; Diagnosis steps 2–3 (retriever trace, manifest comparison).
Confidence badge shows below min_confidence for >5% of items in a session. Severity sev2 row; Mitigation step 2 (kill-switch retriever).
Provenance badge resolves to a retracted source. Severity sev1 row; Mitigation step 1 (disable grounded publication).
Citation-lineage note references the wrong source bundle. Detection: evidence.citation.mismatch_reported; Diagnosis step 1 (cluster mismatches by source ID).
Customer-reported citation mismatch via evidence.citation.mismatch action. Customer signal in detection section; Post-incident: file rubric-failure ticket against bundle owner.

Gaps observed#

  • Saved-item grounding status currently surfaces two values (grounded/partially_grounded). A future ungrounded value is reserved for Library items whose source binding was revoked post-save; tracked in the Sophia source-lifecycle suite.
  • The visual confidence band uses the canonical bands from libs/oshun/analytics/src/customer-taxonomy.ts; localized band labels live in libs/oshun/i18n/ (evidence.confidence.low baseline; band copy for medium/high is generated from the same key family).

No gap in this audit blocks launch.

Companion artifacts#

  • docs/runbooks/grounding-failure.md — on-call runbook.
  • libs/oshun/analytics/src/alerts-manifest.ts — citation-flag-rate, grounding-low-confidence-spike, grounding-fallback.
  • libs/oshun/analytics/src/dashboards-service-health.ts — Sophia panel.
  • libs/oshun/analytics/src/dashboards-assistant.ts — assistant evidence quality panel.
  • libs/oshun/analytics/src/evaluation-manifest.ts — grounded_answers benchmark scope and thresholds.