Persistence & Data

Persistence is the bottom slab of the platform stack: the shared Postgres/Redis substrate and the residency, migration, tombstone, and deletion machinery that sits on top of it. It is composed of five real libraries — the two substrate clients @oshun/database (libs/shared/database) and @oshun/cache (libs/shared/cache), the contract-envelope persistence foundation @oshun/persistence (libs/oshun/persistence), and the two privacy-of-data cores @oshun/data-residency (libs/shared/data-residency) and @oshun/deletion-fanout (libs/shared/deletion-fanout). Between them they answer four questions every domain would otherwise re-answer badly: how does a service get a pooled, instrumented database client; how does a validated domain contract become a durable, tenant-partitioned row; how is a cross-zone data transfer allowed or refused; and how is a person's data actually erased — verifiably — across every store that holds it. This is the foundation the layered model in the platform overview labels "Persistence."

The point of writing this once is that the cost of getting it wrong is paid in correctness and compliance, not just convenience. @oshun/database is imported by 107 non-test files and @oshun/cache by ~12 across apps/ and libs/, so a single connection-pool default or cache TTL is enforced everywhere rather than copy-pasted. @oshun/persistence stores 83 object contracts and 24 enum contracts across twelve domains as one uniform envelope shape (libs/oshun/persistence/src/contract-persistence-registry.ts:213,410), which is why a Veritas Story and a Tara RitualSession persist, tombstone, and erase through identical code. And because deletion and residency are written as domain-agnostic shared cores, a person's right-to-erasure produces a signed, cryptographically verified receipt instead of a soft flag that each team implemented differently. The sections below trace each of these from the registry through to the live BFF routes that call them.

What ships, honestly#

The substrate is implemented and load-bearing. @oshun/database opens a real pg connection pool (libs/shared/database/src/postgres-client.ts:32,40) and exports real transaction, health, query-builder, migration, and metrics surfaces; @oshun/cache is a real reliability toolkit (circuit breaker, distributed lock, pub/sub, invalidation, in-memory tier) over ioredis. Both are tested and widely imported.

The contract-envelope store is implemented and wired into the live BFF. The registry, the Prisma schema renderer, the generic runtime repository, the tombstone/erasure paths, and the createV1PrismaClient entry point all exist and are exercised by real routes — createContractPersistenceService is called from apps/oshun/bff/src/privacy/dsar-erasure-route.ts:38, from the Veritas retraction-cascade runtime, and from a dozen BFF domain stores (metis, messaging, generation, nyx, living-scenes, device-tokens, and more). Two honest caveats: the V1 contract store is a single canonical database addressed by OSHUN_V1_DATABASE_URL, not a per-domain shard farm; and the per-domain migration plans are generated, dry-runnable runbooks whose execution is a deploy-time operation, not something that runs in-process.

The two privacy cores are implemented but intentionally seam-bound. @oshun/data-residency is, by design, I/O-light: it decides whether a transfer is legal and emits an audit event through an injected publisher — "storage decisions (which DB shard / which processing pool) belong upstream of the enforcer" (libs/shared/data-residency/src/index.ts:11-15). @oshun/deletion-fanout is a complete fail-closed orchestration core with real Ed25519 attestations; its live Redis event bus and per-domain Prisma deleters are injected at the BFF / deploy boundary, and its signer resolves to null (fail-closed) in production when no real key is configured (deletion-signer-env.ts:54-56). None of these are stubs — they are honest seams that refuse to fabricate a transfer approval or a deletion that did not happen.

The substrate clients: how a domain gets a database#

@oshun/database is the single data-access surface every service composes instead of importing pg or ioredis directly. Its barrel (libs/shared/database/src/index.ts) exports far more than a connection factory:

• Pooled clients. PostgresClient wraps a real pg Pool with the full config surface — max/min pool size, idle and connection timeouts, a statement_timeout/query_timeout, application_name, and SSL (postgres-client.ts:37-59). Factories cover the three ways a service acquires one: createPostgresClient(config), createPostgresClientFromEnv(), and createPostgresClientFromUrl(url). RedisClient / RedisClusterClient mirror this for Redis.
• Transactions as helpers, not boilerplate. withTransaction, withReadOnlyTransaction, withSerializableTransaction, and withRollbackTransaction bracket a unit of work at a chosen TransactionIsolationLevel; createSavepoint / withSavepoint give nested rollback points, and withAdvisoryLock exposes Postgres advisory locks for cross-process mutual exclusion — all from one import.
• Health and metrics. checkPostgresHealth, checkRedisHealth, checkAllDatabasesHealth, and the HealthMonitor feed the platform's dependency health checks; createPostgresMetrics, InstrumentedPostgresClient, and PoolStatsMonitor emit pool-saturation and query-latency metrics so a starved pool is observable rather than a mystery hang.
• A safe query builder. sql (a tagged-template builder), buildWhereClause, buildInsertStatement, pagination helpers, and sanitizeIdentifier / quoteIdentifier exist so hand-written SQL is parameterized and identifier-escaped by default.
• A connection-string toolkit and a generic migration runner. parse/build/ mask functions for both Postgres and Redis URLs (the mask variant keeps credentials out of logs), plus a MigrationRunner with createSqlMigration for services that manage their own non-Prisma migrations.

@oshun/cache is deliberately more than a Redis wrapper — it is a small reliability kit. Alongside OshunRedisClient/OshunRedisClusterClient it ships an in-process OshunMemoryCache tier, an OshunCircuitBreaker (with CircuitOpenError so a flapping Redis fails fast instead of piling up timeouts), an OshunLockManager distributed lock, an OshunPubSubClient, an OshunInvalidationManager, and the with-cache.ts wrappers — withRedisCache, withMemoryCache, and the cachedMethod method-memoizer — that turn any async function into a read-through cache. The TTL vocabulary is shared and named, not magic numbers: TTL runs from VERY_SHORT (30 s) to MONTH (30 d) and DOMAIN_TTL pins domain-specific lifetimes such as JWT_VALID (5 min) and SESSION (1 h) (libs/shared/cache/src/types.ts:498,520). key-builder.ts provides the typed key namespace (userKey, sessionKey, jwtKey, pattern builders) so two services never collide on a Redis key by accident. The summary framing of these two packages lives in Shared Libraries; this page owns their internals.

The contract-envelope persistence foundation#

@oshun/persistence is where a domain's validated contract — a Zod schema from Contracts — becomes durable state without each domain hand-rolling a table, a repository, and a migration. The insight is that almost every V1 domain object has the same persistence needs (tenant partition, a natural id, optimistic drift detection, soft-delete with audit, true erasure), so they are stored as one uniform contract envelope and served by one generic runtime layer.

The registry is the spec. V1_OBJECT_PERSISTENCE_CONTRACTS and V1_ENUM_PERSISTENCE_CONTRACTS (contract-persistence-registry.ts:213,410) enumerate every persisted contract as a small descriptor binding a domain, a contractName, the Zod schema, and two derived names: a Prisma modelName of ${PascalCase(domain)}${contractName} (e.g. VeritasStory) and a tableName of v1_${snake(domain)}_${snake(contractName)} (e.g. v1_veritas_story) (:191-194). Twelve domains are covered — tara, arete, veritas, nyx, nisaba, metis, library, atelier, studio, messaging, identity, and a cross-cutting bucket for shared ledgers (tenants, consent, personas, model cards, agent runs). Every envelope carries the same ten control fields (V1_PERSISTENCE_CONTROL_FIELDS, :167): persistenceId, tenantId, contractSchema, contractVersion, sourceRecordId, payloadHash, and the four tombstone columns.

From that single registry the whole storage pipeline is generated, so it cannot drift:

renderContractPersistencePrismaSchema() (prisma-renderer.ts:31) walks the registry and emits the Prisma datamodel: three support models (OshunV1ContractBackfillSource, OshunV1MigrationLedger, OshunV1TombstoneAuditEvent), three hand-authored bespoke models that do not fit the envelope (IrisMemoryEntryRevision, AdminAuditEvent, AdminStoreSnapshot), the 24 enums, and one model per object contract — 89 models and 24 enums in the current prisma/schema.prisma. (The in-code comment at contract-record-repository.ts:5 still says "the 85-model schema"; the registry has grown since, which is exactly why the schema is generated rather than maintained by hand.) Each generated object model gets six indexes by construction (prisma-renderer.ts:138-144): a unique (tenantId, contractSchema, sourceRecordId) identity, a tenant index, and the live-scan and payload-hash indexes the runtime queries depend on. When a contract owns a string id it becomes the row @id; otherwise a persistenceId cuid is synthesized.

The runtime repository#

ContractRecordRepository (contract-record-repository.ts:133) is the generic runtime layer — one instance serves every domain because the envelope is uniform. It is driven by the same field-projection logic the renderer uses (getObjectFieldProjections), so the columns it writes can never diverge from the columns the schema declares. Its operations are small but carefully guarded:

• put Zod-parses the record, derives sourceRecordId from the contract's id, and upserts keyed on the (tenantId, contractSchema, sourceRecordId) identity, stamping the envelope and a payloadHash. Two fail-loud guards matter: if a contract field shares a column with the envelope (most commonly its own tenantId) and disagrees with the value being stamped, the write throws rather than silently storing one and returning the other (:163-171); and an upsert that targets an already-tombstoned identity throws — "Restore must be an explicit audited workflow," never a silent recreate (:176-182).
• get / findLive re-hydrate a row back through the Zod schema and return null for anything tombstoned, so reads are validated on the way out as well as the way in.
• tombstone / tombstoneScope mark rows deleted without removing them, writing an idempotent audit event for each, and fail closed if no audit delegate is configured ("Refusing to write an unaudited tombstone," :362-372).
• purge is the only path that issues a real deleteMany — TRUE erasure of the row and its payload — and is reserved for DSAR / right-to-erasure (:309).

The payloadHash is a stable, order-independent SHA-256 over a recursively key-sorted canonicalization of the payload (computeContractPayloadHash, :107), so two structurally-equal records hash identically regardless of property order — which is what makes drift detection and backfill idempotency reliable.

The ergonomic facade#

Domain code does not touch the repository or the registry descriptor directly. The ContractPersistenceService (contract-persistence-service.ts:45) is a registry-aware facade keyed by (domain, contractName): service.put('veritas', 'Story', record, ctx), get, list, tombstone, tombstoneDomainTenant, purge, and a listContracts introspection. An unknown or mistyped pair fails fast with a message telling the caller to register it in V1_OBJECT_PERSISTENCE_CONTRACTS first (:150-162). createContractPersistenceService(client) (:170) wires it to a generated Prisma client via prismaDelegateResolver, and a real client is constructed once through createV1PrismaClient(connectionString) over the @prisma/adapter-pg driver (prisma-client.ts:17). That is the entire path a BFF route needs: one client, one service, and a (domain, contractName) call.

Migration: consolidating per-domain stores into one canonical DB#

The V1 contract store is not greenfield — it is the destination of a migration from the older per-domain databases, and migration-plan.ts is the machine that plans that consolidation. V1_DOMAIN_MIGRATION_PLANS (:229) builds one plan per domain from a DOMAIN_CONTEXTS table that names each legacy source — tara from TARA_DATABASE_URL, veritas from VERITAS_DATABASE_URL, and so on — and a backfill ordering that respects intra-domain dependencies (Veritas backfills Source and Claim before the Story that references them, :93-104). Every plan is dry-runnable: the schema step is prisma migrate diff --from-url … --to-schema-datamodel schema.prisma --script, and the apply step pipes that diff through psql with ON_ERROR_STOP=1 (:330-331). Backfills read adapter-exported payloads from the shared oshun_v1_contract_backfill_sources staging table and upsert with ON CONFLICT (tenant_id, contract_schema, source_record_id) — skipping rows that are already tombstoned or whose payload_hash is unchanged, so a re-run is a no-op (:475-478). Each plan carries a dryRunHash (a SHA-256 over its step SQL) and a four-step rollbackPlan, and the whole runbook renders to Markdown via renderMigrationPlanMarkdown() for operators.

Performance is gated, not assumed. index-requirements.ts (:39) declares three required indexes per table — identity_lookup (unique), live_contract_scan, and payload_hash_drift (:162-181) — and ships both a static checker (validateStaticIndexCoverage parses the Prisma schema) and EXPLAIN (FORMAT JSON) SQL. CI runs @oshun/persistence:explain:check against OSHUN_V1_DATABASE_URL with sequential scans and explicit sorts disabled for the session, and fails unless each query plan actually names the Prisma-mapped index — so an index regression is caught by a real query planner, not a code-review hope. Note the two distinct migration systems on this page: @oshun/database's generic MigrationRunner (for services running their own SQL migrations) and Prisma Migrate (for this generated V1 schema, configured in prisma.config.ts against OSHUN_V1_DATABASE_URL).

Tombstones, erasure, and the DSAR cascade#

Deletion here is two different operations with two different legal meanings, and the foundation keeps them strictly separate. A tombstone is a soft, audited, reversible-only-by-workflow marker that retains the payload for the immutable audit trail; an erasure (purge) physically removes the row. tombstone-semantics.ts (:23) generates a TombstonePolicy for every contract — the identity columns (tenant_id, contract_schema, source_record_id), the single audit table oshun_v1_tombstone_audit_events, the two propagation scopes (source-record then domain-tenant), and the SQL itself, which uses COALESCE(... ) and WHERE tombstoned_at IS NULL so a retry never rewrites the original deletion metadata, plus an ON CONFLICT (event_id) DO NOTHING audit insert so the audit event is idempotent (:83-127).

Erasure is the path that satisfies a Data Subject Access Request's right-to-be-forgotten, and it was built specifically because the audit found the old privacy path "soft-flag only" — marked but never actually erased. executeDsarDeletionCascade (dsar-deletion-cascade.ts:82) purges the subject's contract records and forgets their Iris memory revisions, returning a DsarDeletionReceipt. It is fail-closed: not-found is acceptable (the data was already gone), but a single failed target means complete is false (:157). executeDsarErasureForSubject (dsar-erasure-runtime.ts:60) is the caller that ties this to the privacy state machine: it refuses to even attempt erasure unless the request is in soft-deleted state and past its hardDeleteAtUnixSeconds grace window (:72-77), runs the cascade, and only calls advanceToHardDelete from @oshun/privacy when the receipt is complete — so a DSAR is never reported satisfied while the subject's data might still persist. The Iris side of erasure lives in DurableMemoryStore.forget (durable-memory-store.ts:251), which hard-deletes every revision of an entry, in deliberate contrast to its tombstone, which retains the body for the §10.8 immutable-revision audit chain.

A worked example makes the fail-closed posture concrete. The live route POST /v1/admin/privacy/dsar/erase (apps/oshun/bff/src/privacy/dsar-erasure-route.ts) authenticates the operator (401 without context, 403 without the admin:workspace:privacy scope), builds a ContractPersistenceService over the real Prisma client, and calls executeDsarErasureForSubject. It returns 200 only when result.satisfied is true; an erasure still inside its grace window or with any failed target returns 409 with the stable reason — "never a 2xx that masks an incomplete delete" (dsar-erasure-route.ts:74-82).

Data residency enforcement#

@oshun/data-residency (the V1-PRIV-018 service) decides whether a proposed cross-zone data transfer is allowed and produces the audit trail when it matters. It is intentionally a decision and routing layer, not a storage layer: it reads the canonical residency rule tables from Contracts and emits canonical audit events through an injected publisher, holding no persistence or in-memory state of its own.

ResidencyEnforcementService.evaluate(request) (enforcer.ts:161) looks up the rule for the artifact type, applies the deployment policy via evaluateDeployedTransfer, and — when the outcome demands it — builds a canonical IngestCanonicalAuditEventRequest and publishes it, returning a typed outcome with a block flag and a stable reason. enforce(request) (:194) is the throwing wrapper a Fastify route uses to abort with 403 and that stable reason. Around the legality decision sits traffic shaping: createResidencyRoutingContext (traffic-shaping.ts:85) keeps routine customer traffic in the authenticated subject's home data plane until a transfer is explicitly warranted, classifying each request as home_zone_default, same_zone_target, explicit_cross_region_consent, or legacy_default_home_zone. The context flows through an AsyncLocalStorage and is projected onto a fixed set of request headers — X-Oshun-Residency-Home-Zone, -Route-Zone, -Target-Zone, -Transfer-Mechanism, and X-Oshun-Cross-Region-Consent (:18-22); the injector strips any inbound copies of those headers before re-adding the trusted ones, so a client cannot spoof its own residency by setting them. resolveHomeZoneFromClaim (home-zone.ts:22) is the rollout-safety fallback: a bearer token without a homeZone claim resolves to the deployment's defaultHomeZone instead of surfacing an "unknown zone" failure.

DSR routing is zone-pinned. createDsrResidencyRoutingDecision (dsr-routing.ts:46) maps the four request kinds — access, erasure, portability, rectification — to a queue named themis.privacy.dsr.{routeZone} so a subject's request is processed in their own data plane, and it hard-codes standaloneDsrOpsConsoleAllowed: false (:79), encoding the policy that DSR operations never escape residency routing through a side console. The shipping consumers are the BFF residency middleware and guard, the domain-service adapters, and the admin residency-audit state — this is a live enforced path, with the honest boundary being that the enforcer decides; the actual shard or processing-pool selection lives upstream of it.

Deletion fan-out across derived-artifact services#

The DSAR contract cascade erases a subject's contract rows and memory; @oshun/deletion-fanout is the complementary half that erases the six derived-artifact services a full account deletion must also reach — memory_scope, voice_profile, avatar_pack, generated_artifact, personalization_vector, and conversation_history (deletion-fanout.ts:23). It is scope:shared precisely so any domain — not only the BFF — can run its own deletion consumer against the shared event bus.

Its trust property is that completion can never be claimed, only proven. Each per-service erase outcome is canonicalized into unambiguous bytes prefixed oshun.deletion.attestation.v1 and signed with Ed25519 (canonicaliseDeletionAttestation, :81), and verifyDeletionAttestation re-derives those exact bytes — pinned to this subject and deletion id — to check the signature (:121-141). There are two ways to run the fan-out. The synchronous executeDeletionFanout (:181) invokes each injected eraser in-process and signs the result; an eraser that throws becomes a failed attestation rather than a silent drop, and complete is true only when no service failed (:215). The event-driven orchestrateSubjectDeletion (deletion-event-orchestrator.ts:116) publishes a single subject.deletion.requested event, collects the signed attestations domain consumers reply with, and cryptographically verifies each one against the audit public key before it can count — a forged, tampered, wrong-subject, or replayed receipt fails verification and leaves its service unsatisfied. Its UnsatisfiedReason distinguishes failed (a verified failure), missing (no reply before the window closed), and invalid (a reply that did not verify) (:78-81), and the whole deletion is complete only when every requested service is satisfied.

Two design choices keep this from ever fabricating a deletion. First, the erasers are honest about structure: createSubjectRowEraser (service-erasers.ts:46) reports erased only when rows were actually removed and not_found for zero, while the two shared catalogs that hold no per-user rows use createNoSubjectDataEraser (:76) to report not_found with a documenting detail — honest absence, not a faked deletion. buildBffDeletionErasers (:116) deliberately omits any category whose real deleter has not been injected, so the orchestrator reports it missing and the deletion stays incomplete rather than silently passing. Second, the signer fails closed: resolveDeletionAttestationSigner (deletion-signer-env.ts:38) returns null in production when OSHUN_DELETION_ATTESTATION_ED25519_PRIVATE_KEY is absent — no signer, no runner, deletions stay scheduled — and throws on a malformed key rather than dropping to the deterministic dev key. The composition root buildDeletionRunner (a 30 s default collection window) wires the consumers, transport, and orchestrator into the single SubjectDeletionRunner the BFF deletion route and grace-expiry worker call, and the transport subscribes to the reply channel before publishing so a fast consumer's receipt is never lost to a race (deletion-event-transport.ts:81-102).

Why persistence stays a shared foundation#

The recurring discipline across all five libraries is that the data layer is the only place real connections, real erasure, and real residency decisions live, and everything above it borrows that machinery rather than re-implementing it. @oshun/database and @oshun/cache are the only packages holding live Postgres/Redis handles. The contract-envelope store means a new domain object is a registry entry plus a generated migration, not a bespoke table-and-repository — and the generation step guarantees the runtime, the schema, the tombstone policy, and the index requirements all read from one source and cannot drift apart. @oshun/data-residency and @oshun/deletion-fanout stay domain-agnostic (a residency zone and a deletion service are not domains), which is why they can be scope:shared and composed by the BFF, by apps/iris/api, and by the themis privacy library alike. The payoff is the one the platform overview promises: persistence, residency, and erasure are written once, with a version, a test suite, and an owner — so a request that reads a Tara ritual or honors a right-to-erasure borrows a verified foundation instead of trusting a per-product reinvention.

Related#

• The Shared Platform — the layered model this page fills in at the persistence slab.
• Shared Libraries — the @oshun/* infrastructure, including the summary framing of @oshun/database and @oshun/cache.
• Contracts — the Zod schemas the persistence registry stores and re-validates on every read and write.
• Domain Orchestration — the libs/oshun service layer that calls the persistence facade and the deletion cascade.
• BFF & Gateway — where the DSAR erasure route, residency middleware, and deletion runner are wired to real clients and the event bus.
• Auth & Identity — the identity, consent, and authorization model that gates every persistence and deletion path.