Status: Active Owner: OSHUN Engineering Last Updated: March 23, 2026 Related:
docs/releases/v1/audits/cross-cutting/in/memory-runtime-state.md,
docs/releases/v1/specs/in-memory-runtime-state-classification.md,
docs/releases/v1/reports/subsystem-ownership-matrix.md,
docs/releases/v1/specs/web-app-admin-v1-prd.md
1. Purpose#
This document is the authoritative plan for V1-ARCH-049: define a concrete
replacement path for every release blocker identified in
docs/releases/v1/specs/in-memory-runtime-state-classification.md.
The goal is not to invent new subsystem ownership. The goal is to move each release-blocking in-memory state store onto the already-declared canonical boundary with a durable system of record, restart-safe recovery behavior, and a clear cutover path.
2. Existing Durable Patterns To Reuse#
The repo already contains durable implementations that should be treated as positive controls rather than reinvented:
| Existing Implementation | Why It Matters To This Plan |
|---|---|
apps/isis/generation-api/src/services/job.service.ts |
Demonstrates PostgreSQL-backed job state, Redis/BullMQ queueing, outbox replay, dead-letter handling, and idempotent lifecycle management. |
apps/isis/workflow-registry/src/services/workflow.service.ts |
Demonstrates service-local PostgreSQL persistence, migrations, audit-log semantics, and removal path for quality-critical in-memory control-plane state. |
services/psyche/orchestrator/src/orchestrator/session/state.py RedisStateStore |
Shows the intended durable live-session store already exists inside Psyche; the remaining gap is defaulting and enforcement. |
apps/lilith/svc-conversation/src/session/redis-store.ts |
Shows a horizontally scalable Redis-backed session/presence store already exists for Lilith conversation flows. |
apps/lilith/svc-conversation/src/db/migrations/*.ts |
Shows Lilith conversation already has durable relational tables for threads, messages, summaries, participants, citations, feedback, and shares. |
apps/lilith/svc-user-preferences/src/app.ts |
Shows a mixed durable pattern where MongoDB is the source of truth and Redis is an acceleration/sync layer. |
apps/lilith/svc-knowledge/src/storage/index.ts |
Shows the platform already models durable-vs-memory storage backends explicitly for documents and graphs, including PostgresDocumentStore and Neo4jGraphStore. |
3. Coverage Map#
All 12 release-blocking findings are covered by the six replacement programs
below.
| Replacement Program | Covered Blockers |
|---|---|
| Iris canonical assistant memory plane | apps/iris/api/src/services/memory.service.ts, apps/iris/api/src/services/conversation.service.ts, apps/iris/api/src/services/agent.service.ts, apps/iris/api/src/services/knowledge.service.ts |
| Psyche session and replay plane | services/psyche/orchestrator/src/orchestrator/session/state.py memory-backed session state and checkpoints |
| Sophia durable grounding graph plane | apps/sophia/knowledge-graph/src/main.ts CombinedMemoryStore / MemoryGraphStore runtime |
| Lilith review and approval plane | apps/lilith/svc-review/src/app.ts, apps/lilith/bff/src/utils/translation-review-workflow.ts |
| Lilith conversation and contemplative assistant state plane | apps/lilith/svc-conversation memory session store and memory fallback, apps/lilith/svc-spiritual-guidance/src/conversation-memory.ts, apps/lilith/bff/src/utils/long-term-memory.ts |
| Lilith research workflow plane | apps/lilith/bff/src/utils/translation-open-research.ts |
4. Replacement Programs#
4.1 Iris Canonical Assistant Memory Plane#
Covered blockers:
apps/iris/api/src/services/memory.service.tsapps/iris/api/src/services/conversation.service.tsapps/iris/api/src/services/agent.service.tsapps/iris/api/src/services/knowledge.service.ts
Current problem:
- Iris keeps its canonical assistant memory, conversations, agents, tasks, and
knowledge-base records in in-process
MapandSetstate. - Restarting the service drops assistant continuity and makes replay/audit impossible.
- The service boundary already claims canonical ownership in the ownership matrix, but the implementation does not yet honor that contract durably.
Target durable architecture:
- Make Iris the only system of record for assistant memory, conversation history, agent/task lifecycle, and knowledge documents/chunks.
- Use PostgreSQL through
@oshun/databaseas the canonical persisted store for relational truth and metadata. - Use Redis only for optional cache, hot-session acceleration, short-lived locks, and idempotency keys; never as the sole durable memory record.
- Add append-only lifecycle events or outbox tables for memory mutation, conversation mutation, and agent/task transitions so downstream systems can replay or consume them safely.
Implementation plan:
- Add migrations for assistant memory entries, core-memory blocks, conversations, messages, agents, tasks, knowledge bases, knowledge documents, and knowledge chunks.
- Extract repository interfaces from the current service classes so the in-memory implementations can be replaced cleanly with Postgres-backed repositories.
- Introduce dual-write instrumentation behind a feature flag so new writes can hit both the legacy in-memory structures and the durable repositories during migration.
- Switch read paths to the durable repositories once parity checks pass, then remove the in-memory system-of-record path entirely.
- Add service readiness checks for PostgreSQL and Redis connectivity so the health surface reflects the new persistence contract.
Cutover strategy:
- Dev/test may keep explicit non-production memory repositories, but production startup must fail if the durable Iris store is unavailable.
- Backfill any seed or fixture data into the new tables before read cutover.
- Expose migration metrics for dual-write mismatches, message-count parity, and search/index parity before removing the old path.
Verification required:
- migration tests
- repository unit tests
- service integration tests covering restart recovery
- idempotent create/update tests for messages and tasks
- outbox replay tests for downstream consumers
Primary downstream tasks:
V1-ARCH-052V1-ARCH-057V1-ARCH-058V1-ARCH-059
4.2 Psyche Session And Replay Plane#
Covered blocker:
services/psyche/orchestrator/src/orchestrator/session/state.py
Current problem:
- Psyche already has
RedisStateStore, but bothCheckpointManagerandSessionStateManagerstill default toMemoryStateStore. - Replay-sensitive live assistant state can therefore remain process-local even though a durable-capable path exists.
Target durable architecture:
- Require Redis-backed live session state for all production Psyche runtimes.
- Persist checkpoints and replay artifacts into a durable checkpoint/event-log plane rather than only process-local dictionaries.
- Treat in-memory state only as test/dev scaffolding and reject it in production startup configuration.
Implementation plan:
- Introduce an explicit environment-backed store resolver with production validation that forbids memory mode.
- Replace implicit
MemoryStateStore()defaults with injected dependencies so no critical path silently falls back to memory. - Persist checkpoints into a durable journal keyed by session and version, retaining enough metadata for replay, debugging, and incident review.
- Add reconciliation logic so Redis live state and durable checkpoints stay version-aligned.
Cutover strategy:
- Ship store-resolution validation first so the production path cannot drift back to memory.
- Run checkpoint dual-write to the durable journal before using it for replay.
- Remove memory fallback only after restart-recovery tests prove session resume and replay.
Verification required:
- unit tests for store-resolution rules
- integration tests for restart recovery from Redis
- replay tests from persisted checkpoints
- fault-injection tests for Redis reconnect and checkpoint write failures
Primary downstream tasks:
V1-ARCH-052V1-ARCH-056V1-ARCH-058
4.3 Sophia Durable Grounding Graph Plane#
Covered blocker:
apps/sophia/knowledge-graph/src/main.ts
Current problem:
- The standalone knowledge-graph service boots on
CombinedMemoryStore, which extendsMemoryGraphStore. - That makes the primary grounding graph lose entities and relationships on restart.
Target durable architecture:
- Replace the production
MemoryGraphStorepath with a durable graph backend behind the existing graph-store abstraction. - Reuse the durable-storage pattern already present in
apps/lilith/svc-knowledge/src/storage/index.ts, where graph backends are chosen explicitly and memory is only a local/testing mode. - Emit durable mutation events or change-log records for entity and relationship changes that matter to research and evidence workflows.
Implementation plan:
- Extract the standalone service boot path to accept a graph-store config
rather than constructing
CombinedMemoryStoredirectly. - Implement or reuse a durable production graph store, likely Neo4j-backed or an equivalent durable graph service already aligned to Sophia ownership.
- Restrict
MemoryGraphStoreto tests, demos, and local-only tooling. - Add startup validation so production cannot run with the memory graph store.
- Add import/backfill tooling so any existing seed graph data can be loaded durably.
Cutover strategy:
- Keep the memory store only for fixtures and fast local tests.
- Build side-by-side read/write validation on a non-production environment before switching the service entrypoint.
- Require migration or graph bootstrap completion before the new service is considered healthy.
Verification required:
- service integration tests against the durable graph backend
- restart-persistence tests for entities and relationships
- query-parity tests for traversal, neighbors, and statistics
- startup-validation tests proving production rejects memory mode
Primary downstream tasks:
V1-ARCH-054V1-ARCH-056V1-ARCH-060
4.4 Lilith Review And Approval Plane#
Covered blockers:
apps/lilith/svc-review/src/app.tsapps/lilith/bff/src/utils/translation-review-workflow.ts
Current problem:
svc-reviewcurrently holds queues, review items, decisions, assignments, escalations, SLAs, and metrics in memory.- The BFF also keeps review/approval workflow state in memory, which duplicates ownership and makes restart loss likely.
- PRD section
16.4explicitly calls review and approval state durability a production requirement.
Target durable architecture:
- Make
apps/lilith/svc-reviewthe only system of record for review and approval workflow state. - Persist review items, stage transitions, assignments, decisions, escalations, SLA timers, and immutable audit events in a durable relational store.
- Reduce the BFF to a client/orchestration layer that calls
svc-reviewinstead of storing review lifecycle truth locally. - Align record shapes and stage semantics with
docs/releases/v1/specs/studio-review-and-approval-workflows-dependency-and-data-contract-map.md.
Implementation plan:
- Add
svc-reviewstorage abstractions and relational schema for review queues, items, decisions, assignments, escalations, SLAs, notifications, and audit events. - Refactor route handlers in
svc-reviewto use repositories rather than process-localMapinstances. - Replace BFF-local translation review state with service calls to
svc-reviewand persisted workflow references. - Add idempotency keys for decision submission and assignment transitions.
- Record append-only audit events for every review/approval transition.
Cutover strategy:
- Introduce service-backed write-through from the BFF before removing the local workflow store.
- Migrate existing test fixtures to repository-seeded fixtures.
- Remove the BFF memory store only after all review routes use
svc-reviewas the source of truth.
Verification required:
- migration tests
svc-reviewroute integration tests- BFF integration tests proving it no longer owns review truth
- restart-recovery tests for pending, assigned, escalated, approved, and rejected reviews
- idempotency and immutable-audit-event tests
Primary downstream tasks:
V1-ARCH-050V1-ARCH-051V1-ARCH-057V1-ARCH-058V1-ARCH-059
4.5 Lilith Conversation And Contemplative Assistant State Plane#
Covered blockers:
apps/lilith/svc-conversationapps/lilith/svc-spiritual-guidance/src/conversation-memory.tsapps/lilith/bff/src/utils/long-term-memory.ts
Current problem:
- Lilith conversation still exposes memory session storage and silent memory fallback, even though a Redis session store already exists.
svc-conversationalso keeps conversation-critical runtime state in process memory despite already having database migrations for durable conversation records.svc-spiritual-guidanceand the BFF long-term memory utility keep assistant continuity state outside the canonical Iris boundary.
Target durable architecture:
- Use Redis-backed session/presence storage for live connection/session state in
svc-conversation, with production startup rejecting memory fallback. - Use the existing relational thread/message/summary schema in
apps/lilith/svc-conversation/src/db/migrationsas the durable source of truth for conversations. - Move long-term assistant memory ownership fully behind the canonical Iris boundary; Lilith services may cache or reference memory, but may not own its durable truth.
- Restrict
svc-spiritual-guidanceto durable references, user journey metadata, and calls through canonical memory/conversation services.
Implementation plan:
- Remove silent fallback from
createSessionStoreFromEnv()so production must provide Redis. - Wire
svc-conversationruntime paths to the existing database schema for threads, messages, summaries, citations, feedback, and participants. - Replace local long-term memory writes in the BFF with Iris adapter calls and durable identifiers.
- Refactor spiritual-guidance memory helpers to persist only owned workflow state and retrieve canonical assistant memory through Iris.
- Add restart rehydration for live sessions from durable conversation records plus Redis presence/session state.
Cutover strategy:
- Enable dual-write from existing runtime paths into the durable conversation repositories first.
- Switch read paths per feature area: conversation history, summaries, participant state, then live-session rehydration.
- Delete BFF-owned long-term memory stores only after all consuming flows use Iris identifiers and durable records.
Verification required:
- migration validation
- conversation repository integration tests
- websocket/session integration tests with Redis
- restart-recovery tests for conversation resume and summary continuity
- cross-service tests proving Lilith long-term memory routes through Iris
Primary downstream tasks:
V1-ARCH-052V1-ARCH-056V1-ARCH-058V1-ARCH-059
4.6 Lilith Research Workflow Plane#
Covered blocker:
apps/lilith/bff/src/utils/translation-open-research.ts
Current problem:
- Research submission, review, approval, and publication state is currently stored in memory in a BFF utility.
- That violates the PRD durability requirement and bypasses the subsystem boundaries for research and evidence handling.
Target durable architecture:
- Replace BFF-owned research lifecycle state with a durable service-backed workflow model.
- Persist submissions, source artifacts, review decisions, approval state, publication readiness, provenance references, and immutable lifecycle events in a durable relational store.
- Connect research outputs to Sophia-owned evidence/grounding references rather than storing opaque process-local workflow state in the shell.
Implementation plan:
- Move research workflow state into a dedicated durable service module or an existing subsystem boundary that already owns research/review operations.
- Add relational tables for submissions, artifacts, reviewer decisions, approval stages, publication records, and event history.
- Replace BFF-local workflow state with API calls, durable IDs, and derived view models.
- Emit immutable provenance/event records suitable for restart recovery and later replay.
Cutover strategy:
- Introduce a service-backed adapter in the BFF first while preserving current UI contracts.
- Backfill any fixture or seeded research records into the durable tables.
- Remove in-memory workflow truth only after research status, approval state, and publication state all read from the durable source.
Verification required:
- migration and repository tests
- BFF integration tests against the research service boundary
- restart-recovery tests for submissions in each lifecycle stage
- provenance and event-history validation
Primary downstream tasks:
V1-ARCH-054V1-ARCH-056V1-ARCH-057V1-ARCH-058
5. Recommended Sequencing#
Recommended execution order for the implementation tasks that follow:
- Lilith review and approval plane
- Lilith conversation and contemplative assistant state plane
- Iris canonical assistant memory plane
- Psyche session and replay plane
- Lilith research workflow plane
- Sophia durable grounding graph plane
Reasoning:
- review and approval are explicit PRD release gates and also unblock audit and replay work
- Lilith conversation and BFF long-term-memory work removes the most direct shell-to-memory continuity violations
- Iris and Psyche must then formalize the canonical durable assistant boundary end to end
- research and Sophia graph durability depend on the earlier ownership cleanup
6. Exit Criteria For V1-ARCH-049#
V1-ARCH-049 is complete only if all of the following are true:
- Every
release blockerfrom the classification matrix is covered. - Each blocker has an explicit durable system of record.
- Each blocker is assigned to the canonical owning subsystem boundary.
- Each blocker has a migration/cutover path, not only an end-state diagram.
- Each blocker has required verification called out before implementation.