# OSHUN V1 Shared Object Persistence, Migration, and Backfill Plan

Status: Active

Date: March 23, 2026

Owners: OSHUN Engineering, OSHUN Platform, OSHUN Product

Related:

- `docs/releases/v1/specs/todos.md`
- `docs/releases/v1/reports/subsystem-ownership-matrix.md`
- `docs/releases/v1/specs/program-charter.md`
- `docs/releases/v1/specs/web-app-admin-v1-prd.md`
- `docs/releases/v1/audits/cross-cutting/contract/to-prisma-alignment.md`
- `libs/contracts/src/common/contract-prisma-alignment.ts`
- `libs/contracts/src/common/contract-prisma-migrations.test.ts`
- `apps/oshun/bff/src/nisaba/state.ts`
- `apps/isis/output-registry/src/services/output.service.ts`
- `apps/lilith/svc-review/src/app.ts`
- `apps/lilith/bff/src/utils/translation-review-workflow.ts`
- `apps/lilith/svc-spiritual-guidance/src/personas.ts`
- `apps/lilith/svc-ai/src/personality-archetypes.ts`

## 1. Purpose

This document is the authoritative plan for `V1-ARCH-078`: define persistence,
migration, and backfill plans for the new OSHUN V1 Nisaba, notebook, evidence,
review, persona, and provenance objects that were added as canonical contracts
and owner-bound Prisma models.

This plan is intentionally owner-bound.

When later tasks refer to "shared object storage", they do not authorize a new
shell-owned source of truth. Shared objects remain canonically owned by their
declared subsystem or domain. The shell and BFF may compose, index, and cache
them, but they must not become the durable business-logic owner.

## 2. Existing Durable Patterns To Reuse

The repo already contains the migration and backfill patterns this plan should
reuse instead of reinventing.

| Existing implementation                                                                                     | Why it matters                                                                                                                                                     |
| ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `libs/contracts/src/common/contract-prisma-alignment.ts`                                                    | It is the executable manifest of canonical owner boundaries and Prisma targets.                                                                                    |
| `libs/contracts/src/common/contract-prisma-migrations.test.ts`                                              | It already proves forward and rollback SQL can be generated for the new owner models, even where checked-in migration directories do not yet exist.                |
| `apps/isis/output-registry/src/services/output.service.ts`                                                  | It demonstrates idempotent provenance backfill, batch scanning, enrichment, source counting, and marker-based replay safety.                                       |
| `apps/oshun/bff/src/nisaba/state.ts`                                                                        | It is the concrete current source for Nisaba passage, notebook, annotation, compare, and grounded-evidence demo state that must be retired from in-memory storage. |
| `apps/lilith/svc-review/src/app.ts`                                                                         | It shows the current review queue/item/assignment/escalation model and its file-backed snapshot format.                                                            |
| `apps/lilith/bff/src/utils/translation-review-workflow.ts`                                                  | It shows an existing restart-safe but local workflow snapshot model that must be absorbed into the canonical review package owner.                                 |
| `apps/lilith/svc-spiritual-guidance/src/personas.ts` and `apps/lilith/svc-ai/src/personality-archetypes.ts` | They are the clearest current persona registries to seed the new canonical persona store.                                                                          |

## 3. Authoritative Owner Map

The canonical owner map is already settled by the ownership matrix, PRD, and
contract-to-Prisma alignment audit.

| Object family       | Canonical owner | Canonical model                                              | Current legacy or precursor sources                                                                      |
| ------------------- | --------------- | ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------- |
| Nisaba passage data | Nisaba          | `libs/nisaba/database/prisma/schema.prisma` `Passage`        | Nisaba corpus/reference/edition data and the temporary BFF Nisaba reader state                           |
| Notebook            | Sophia          | `libs/sophia/database/prisma/schema.prisma` `Notebook`       | Nisaba in-memory notebooks, Nisaba `PersonalLibraryItem`, Nisaba `ResearchProject` collaboration context |
| Evidence pack       | Sophia          | `libs/sophia/database/prisma/schema.prisma` `EvidencePack`   | Sophia citations/documents, Veritas claim evidence, Nisaba grounded evidence views                       |
| Persona             | Lilith          | `libs/lilith/database/prisma/schema.prisma` `Persona`        | Lilith persona registries, archetype catalogs, policy-adjacent persona definitions                       |
| Provenance bundle   | Isis            | `libs/isis/database/prisma/schema.prisma` `ProvenanceBundle` | Isis `GeneratedOutput`, `Provenance`, `GenerationJob`, `LineageEdge`, output-registry metadata           |
| Review package      | Isis            | `libs/isis/database/prisma/schema.prisma` `ReviewPackage`    | Lilith `svc-review`, translation review workflow snapshots, admin-review seed queues                     |

Normative rule:

- `Passage`, `Notebook`, `EvidencePack`, `Persona`, `ProvenanceBundle`, and
  `ReviewPackage` are the system-of-record rows after cutover.
- BFF route state, shell caches, compare views, grounded evidence views, and
  queue summaries are read models or temporary scaffolding, not durable truth.

## 4. Cross-Cutting Migration Rules

### 4.1 ID strategy

- If the legacy source already uses the target contract ID shape and that ID is
  stable, preserve it.
- If the legacy source uses `cuid`, slug-like strings, or compound natural keys
  and the canonical row requires a UUID, generate a deterministic UUIDv5 from:
  `owner + object_type + legacy_source + legacy_id`.
- Every backfill must also persist the original legacy identifier in metadata or
  a migration ledger so reverse lookup and replay remain possible.

### 4.2 Replay and idempotency

- Every backfill runner must be upsert-based, not insert-only.
- Every runner must maintain a durable migration ledger containing at least:
  source name, source cursor or ID, canonical owner, canonical ID, content hash
  or version hash, status, started at, finished at, and last error.
- A second run over the same source data must be a no-op unless the source hash
  changed.
- Partial-failure recovery must restart from the migration ledger, not from an
  in-memory cursor.

### 4.3 Write cutover pattern

Each object family follows the same cutover stages:

1. Deploy canonical owner schema and repository path.
2. Run dry-run extraction and row-shape validation.
3. Run idempotent backfill into the canonical owner store.
4. Enable dual-write on new mutations where a legacy write path still exists.
5. Run parity checks on counts, checksums, and representative object samples.
6. Switch read paths to the canonical owner.
7. Remove or quarantine the legacy source-of-truth path.

### 4.4 What not to backfill

- Route caches
- compare views as standalone durable objects
- grounded evidence views as standalone durable objects
- ephemeral pagination/filter state
- presentation-only queue summaries
- shell-composed aggregates that can be rebuilt from canonical rows

## 5. Object-Specific Plans

### 5.1 Nisaba Passage And Workspace Data

Canonical persistence:

- `Passage` is the canonical shared passage object and lives in Nisaba.
- Nisaba-owned adjuncts such as `Annotation`, `AnnotationLayer`,
  `ResearchProject`, `PersonalLibraryItem`, and `Bookmark` remain Nisaba-owned
  local records unless and until a future shared contract explicitly replaces
  them.

Migration sources:

- Nisaba corpus/reference/edition inputs already represented in the Nisaba
  schema.
- Temporary shell-facing records in `apps/oshun/bff/src/nisaba/state.ts`.

Backfill rules:

- Backfill `Passage` first. Nothing else in this task block should create a
  notebook or evidence-pack link to a passage until the passage crosswalk
  exists.
- Use normalized reference plus language plus source collection as the primary
  dedupe key for corpus imports.
- Preserve passage provenance as first-class data. Corpus or edition lineage
  must not be flattened away during passage creation.
- Nisaba `Annotation` rows remain Nisaba-owned and should be retained locally;
  they are projected into notebooks rather than migrated out as a new canonical
  shared object.

Derived-state rule:

- Nisaba compare views are derived from passages and evidence, not persisted as
  their own canonical object.
- Nisaba grounded evidence views are derived from `EvidencePack` plus passage
  context, not their own durable store.

### 5.2 Notebook Backfill Into Sophia

Canonical persistence:

- `Notebook` lives only in Sophia after cutover.

Primary legacy sources:

- Nisaba BFF notebooks in `apps/oshun/bff/src/nisaba/state.ts`
- Nisaba `PersonalLibraryItem`
- Nisaba `ResearchProject` collaborator and workspace context

Migration rules:

- Create notebook shells first with owner, domain, visibility, and status.
- Materialize sections and items only after the passage crosswalk is available.
- Use a deterministic UUIDv5 for notebook IDs derived from the legacy notebook
  or collection identity.
- Populate notebook collaborators from explicit project membership when the
  source is a shared research workspace. Do not invent collaborators for
  personal-library imports.

Important contract accommodation:

- The current Nisaba BFF notebook item kinds include `annotation`,
  `grounded-evidence-view`, and `compare`.
- The canonical `NotebookItemKindSchema` does not support those item kinds
  directly.

Required transforms:

- legacy `passage` item -> canonical notebook item with `kind: "passage"` and a
  `passage` link.
- legacy `annotation` item -> canonical notebook item with `kind: "note"`,
  excerpt and note body copied into the item body or summary, and a `passage`
  link back to the annotated passage.
- legacy `grounded-evidence-view` item -> canonical `EvidencePack` backfill in
  Sophia plus an optional notebook `note` item carrying the user-facing summary
  and route URL.
- legacy `compare` item -> do not create a new durable compare object; store a
  notebook `note` summary plus linked passages and regenerate the compare view
  on demand.

This transform is required because lossless one-to-one item migration is not
possible under the current notebook contract.

### 5.3 Evidence Pack Backfill Into Sophia

Canonical persistence:

- `EvidencePack` lives only in Sophia after cutover.

Primary sources:

- Sophia `Document`, `DocumentChunk`, `Citation`, `Source`
- Veritas `ClaimVerification` and `ClaimEvidence`
- Nisaba grounded evidence view state currently held in
  `apps/oshun/bff/src/nisaba/state.ts`

Migration rules:

- Backfill or resolve supporting `Source` and `Citation` records before creating
  the evidence pack row.
- Create the pack shell, then hydrate claims, excerpts, citations, retrieval
  trace, rationale, and review summary.
- Set `notebookId` only after the notebook crosswalk exists.
- Set `provenanceBundleId` only when the evidence pack directly supports a
  generated artifact, generated report, or governed synthetic output.

Derived-state rule:

- Reader-facing "grounded evidence views" in the BFF are projections over the
  evidence pack plus passage context. They should be regenerated after cutover,
  not migrated as a second source of truth.

### 5.4 Persona Backfill Into Lilith

Canonical persistence:

- `Persona` lives only in Lilith after cutover.

Primary sources:

- `apps/lilith/svc-spiritual-guidance/src/personas.ts`
- `apps/lilith/svc-ai/src/personality-archetypes.ts`
- related persona-selection or persona-detail catalogs that currently encode
  runtime persona metadata in code

Migration rules:

- Seed the canonical persona table from the highest-fidelity persona profile,
  then enrich it from archetype catalogs where they provide capabilities, style,
  or teaching metadata absent from the base profile.
- Use persona slug or source persona key as the deterministic migration key.
- Leave `voiceProfileId` and `avatarPackId` nullable unless a verified Psyche
  asset registry mapping already exists.
- Do not mark a persona as approved or published unless there is existing human
  approval evidence. Default migrated records to review-safe statuses such as
  `draft`, `review`, `unsubmitted`, or `pending_review`.
- Keep code-defined registries as seed fixtures only during migration; after
  cutover they must become import sources, not runtime truth.

### 5.5 Provenance Bundle Backfill Into Isis

Canonical persistence:

- `ProvenanceBundle` lives only in Isis after cutover.

Primary sources:

- Isis `GeneratedOutput`
- Isis `Provenance`
- Isis `GenerationJob`
- Isis `LineageEdge`
- output-registry metadata and existing provenance backfill semantics in
  `apps/isis/output-registry/src/services/output.service.ts`

Migration rules:

- Backfill provenance bundles after output, workflow, model, and lineage source
  data can be resolved.
- Reuse the output-registry marker pattern so provenance-bundle backfills can be
  replayed safely and enriched later without duplicating work.
- If a legacy generated-output identifier is not already a UUID, use the
  deterministic UUIDv5 crosswalk for `primaryArtifactId` and keep the original
  output ID in bundle metadata.
- Build artifacts, workflow, models, inputs, lineage nodes, lineage edges,
  watermark signals, disclosure, rights, and integrity from legacy generation
  data in one canonical bundle rather than splitting them across multiple new
  tables.

Cutover rule:

- New generation and output registration flows should dual-write the legacy
  `Provenance` record and the canonical `ProvenanceBundle` until read parity is
  proven.

### 5.6 Review Package Backfill Into Isis

Canonical persistence:

- `ReviewPackage` lives only in Isis after cutover.

Primary sources:

- `apps/lilith/svc-review/src/app.ts`
- `apps/lilith/bff/src/utils/translation-review-workflow.ts`
- any shell or admin seed queues that currently represent review state outside
  the canonical owner

Migration rules:

- Review backfill is last. It depends on canonical artifact IDs from passage,
  notebook, evidence, persona, and provenance migrations.
- Preserve stage history, decisions, delegation, SLA timing, and escalation
  lineage. Review migration is not complete if it only copies current status.
- Import active and closed reviews separately so reopened or escalated reviews
  can be replayed safely.
- For file-backed Lilith review snapshots, import the full snapshot as of the
  cutover point and then dual-write to the new canonical review package store
  until the old services are retired.

Artifact-linking rule:

- `artifactRef` may preserve the original legacy artifact locator.
- `artifactId` must be rewritten to the canonical UUID after the relevant object
  family crosswalk exists.
- Dependencies on evidence or provenance must point to canonical `EvidencePack`
  and `ProvenanceBundle` IDs, not legacy local references.

## 6. Required Sequencing Order

This is the required migration order for `V1-ARCH-079`.

### Wave 0: Migration Framework

- create the durable migration ledger and crosswalk mechanism
- add dry-run extraction commands and row-shape validation
- add per-object feature flags for dual-write and read cutover

### Wave 1: Foundational Objects

- backfill Nisaba `Passage`
- backfill Lilith `Persona`

Rationale:

- notebooks need passage IDs
- review packages may need persona artifact IDs

### Wave 2: Research Workspace Objects

- backfill Sophia `Notebook`
- backfill Sophia `EvidencePack`

Rationale:

- notebook item transforms depend on passage crosswalks
- evidence packs may attach to notebooks and passages

### Wave 3: Generation Governance Objects

- backfill Isis `ProvenanceBundle`

Rationale:

- review packages frequently depend on provenance and rights lineage

### Wave 4: Approval And Governance Objects

- backfill Isis `ReviewPackage`
- switch admin review reads to canonical review packages

Rationale:

- review migration must bind to already-migrated canonical artifact IDs

## 7. Known Gaps And Required Accommodations

### 7.1 Notebook Item Shape Gap

Current Nisaba notebook items are richer than the canonical notebook item
enumeration. `annotation`, `grounded-evidence-view`, and `compare` require
transform-based migration rather than one-to-one persistence.

### 7.2 No Checked-In Prisma Migration Directories Yet

The repo currently validates migration and rollback SQL by `prisma migrate diff`
and committed schema SQL artifacts rather than by checked-in per-domain Prisma
migration folders.

That means `V1-ARCH-079` must treat rollout sequencing, SQL generation,
backfill, and cutover as one coordinated deployment program. It cannot assume
"run the old migration directory" is already available.

### 7.3 Legacy String IDs Versus Canonical UUID Owners

Several legacy sources use `cuid` or ad hoc string IDs while the new canonical
objects use UUIDs. Deterministic UUIDv5 crosswalks are mandatory for replay-safe
migration and for review/provenance foreign-key integrity.

## 8. Verification Gates For Implementation

`V1-ARCH-079` and follow-on execution work are not complete until all of the
following exist and pass:

- dry-run extraction for every object family
- idempotent rerun tests proving no duplicate canonical rows are created
- parity checks for counts and representative object samples
- rollback validation for owner-schema changes
- BFF or adapter integration tests proving reads come from canonical owner
  stores after cutover
- partial-failure recovery tests proving migration resumes from the durable
  ledger
- audit or provenance verification where review and provenance migrations depend
  on prior artifact backfills

## 9. Exit Criteria For V1-ARCH-078

`V1-ARCH-078` is complete only when:

1. the owner-bound persistence targets are explicit
2. the legacy migration sources are explicit
3. the backfill rules are explicit
4. the sequencing order is explicit
5. the known contract and rollout gaps are explicit
6. `V1-ARCH-079` can be implemented directly from this document without
   re-deciding ownership or cutover order
