Operations, Rollout & Launch

Launch is a matrix, not one boolean. Each client tier, platform, region, tenant, provider, and capability enters only after its gates and can be independently held, degraded, or rolled back.

V3 — the Lilith Metaverse — is not one product that ships once. It is three tenants (Tara, Saraswati, Lilith Commons) across four client tiers and nine certification platforms, opening region by region under a different rating board, residency rule, and music-rights clearance in each one, staffed by a 24×7 trust-operations org, and held back from "go" by a gate that refuses to go green while any of those disagrees. The operational question is therefore not "does the feature work?" but "is this tenant cleared in this region for this device, with operators on shift to answer a report in the user's language, and is the launch gate honestly green?" This page is the answer: how one identity follows a user across web, mobile, and VR (cross-platform & cross-progression); how GA opens in wave-1 regions and routes everyone else to a waitlist (region rollout); how the operator staffing model scales from a lean day-one pool to steady state; and how launch readiness aggregates all of it into a single, unforgeable go/no-go that today reads — correctly — red. Each claim below is pinned to the code that enforces it, or labelled honestly where the code is a policy artifact rather than a runtime. The enforcement spine it rides on — observability, eval gates, performance budgets, security — is its architecture companion ../architecture/observability-performance-security-and-launch.md; the device ladder it routes across is ./client-tiers-fidelity-stadium.md; the accessibility, localization, and QA gates a region must clear are ./accessibility-localization-quality.md. The feature hub this page belongs to is ../V3_features.md.

What ships, honestly#

The identity bridge is real and account-authoritative. bindV3AvatarToV1Account (libs/v3/lilith-identity-bridge/src/index.ts) binds a V3 avatar to the V1 Oshun account id (v1UserId), writes it to the V1 profile field v3.activeAvatarId with persistedToV1Profile: true, and emits a v3.account_avatar.linked audit event — there is no V3-private account store. The canonical single-account authority is @oshun/identity itself (V2OshunIdentityAccountState.sourceOfTruthPackageName === '@oshun/identity', with validateV2OshunIdentityAccountState asserting platformAccountsBindToSingleOshunAccount).

The region router is a real two-layer decision, typed and tested. The shell BFF availability gate resolveV3LilithLaunchDecision (apps/oshun/bff/src/routes/v3-lilith-launch.ts) answers "is this region open and this tenant cleared here?" over a typed region/tenant matrix; the per-region rules engine evaluateV3PerRegionRules (libs/v3/lilith-identity-bridge/src/per-region-rules.ts) applies five jurisdiction profiles' fan-token, age, content-rating, and residency rules and returns an allow | adapt | block decision. Both are exercised by report builders that assert the decisions match expected fixtures, and both back green standalone gates (verify:v3 region-rollout-compliance, verify:v3 per-region-rules).

The operator staffing model and GA-week rota are authored machine-readable artifacts (V3/operations/operator-staffing-model.json, V3/operations/on-call-rota-ga-week.json) — pools, SLAs, language coverage, incident-commander rota, and a green pager test — not prose promises.

Two honest qualifications carry the page. First, the staffing/rota artifacts read green standalone, but the aggregate launch-readiness run currently records operator-console-staffed: fail and operator-pools-day-one: fail; the page reports the aggregate, not the more flattering standalone file. Second, the launch-readiness gate as a whole is honestly red — 30 of 41 constituent verifiers passing, gaReady: false — and that red is the point: the gate is wired so its green cannot be faked.

Cross-platform and cross-progression#

V3 has no platform-specific account and no platform-specific progression. A user is one identity everywhere, and the code makes "everywhere" concrete.

One account, bound to the V1 Oshun identity#

Every V3 binding is keyed by v1UserId. bindV3AvatarToV1Account refuses to proceed without one (errorCode: 'missing-v1-user'), and on success writes the active avatar into the V1 profile (profileField: 'v3.activeAvatarId', persistedToV1Profile: true) so the V1 account — not a V3 shadow record — is the source of truth. v3AvatarIdVisibleInV1Profile then confirms an avatar is visible from the V1 side only when both the active-avatar pointer and a persisted binding agree. Upstream, @oshun/identity enforces the invariant that matters for cross-progression: a platform account binds to exactly one Oshun account (validatePlatformBinding pushes an error if binding.linkedOshunAccountId !== oshunAccountId), so two headsets cannot silently fork one player into two progressions.

Headset account linking — Quest, Vision Pro, PSVR 2#

The VR platforms link through a typed OAuth bridge. V3_SUPPORTED_OAUTH_BRIDGE_PLATFORMS is exactly ['quest', 'vision-pro', 'psvr2'], and v3OAuthBridgeIssuerForPlatform pins each to its real issuer — auth.meta.com/quest, appleid.apple.com/vision-pro, ca.account.sony.com/psvr2. The bind path fails closed on two distinct dishonest inputs: an unrecognised platform (unsupported-oauth-platform) and an account whose issuer does not match the platform's canonical issuer or that lacks a verified access-token proof (oauth-account-not-verified). The headset issuer's subject is mapped to the V1 account and the link is persisted to the V1 profile and audit-logged — exactly the "audit-logged" link the feature canon promises.

Server-authoritative progression#

State authority lives on the Rust world server, not the client, which is why switching devices cannot diverge: a ticket consumed on desktop is consumed on the Quest because both read and write the same authoritative shard (the interest and capacity model is in ./world-rooms-presence-multiplayer.md, and the cross-tier authority rule — authority sees only gateway RTT, never the video path — is in the architecture companion). The identity bridge is the persistence seam for the parts that are profile state (active avatar, linked headsets, operator display-name overrides); the broader "wardrobe / follow lists / subscriptions / ticket inventory / Arete continuity all follow the user" claim is a server-authority property of the world server, honestly labelled here rather than re-implemented in the bridge. One operator-facing seam closes the loop: applyV3RealmDisplayNameOverride lets a moderator rename a user's realm display name with the change writtenToV1AuditLog and visibleToOperatorConsole — cross-progression and operations meet at the V1 audit log.

Region rollout strategy#

GA does not mean global. V3 opens in wave-1 regions; everyone else is routed to a waitlist, and the wave structure is encoded, not aspirational.

The wave structure, in code#

resolveV3LilithLaunchDecision carries a region-profile table whose status field is the rollout gate:

Any region whose status !== 'open' — or any wave-1 region whose requested tenant is not availableAtGa — returns tier: 'regional-waitlist' with a landing URL of the form /v3/landing?mode=regional-waitlist&region=…&tenant=…. This is currently a routing state, not a persisted signup flow. Because no consent, delivery, DSAR, or durable lead authority is configured, POST /api/v3/lilith/launch/waitlist fails closed with 503 regional_waitlist_not_configured and emailStored: false; the legacy record lookup always returns 404. The machine-readable mirror of this matrix is V3/regions/tenant-region-availability.json, and the whole thing is gated by a green verify:v3 region-rollout-compliance.

Per-tenant rollout differs from the wave#

Saraswati Stage opens in fewer regions than Tara and Commons, because music rights clear jurisdiction by jurisdiction. V3_LILITH_TENANT_REGION_AVAILABILITY encodes it precisely: all three tenants are ga in US-CA, Germany, and UK, but in India and Brazil Saraswati is availableAtGa: false, mode: 'waitlist' ("Music-rights and lyric-tone review remain gated…"), while Tara and Commons open normally. A user in Mumbai gets Tara at GA and a Saraswati waitlist sign-up on the same screen — the router resolves tenant availability independently of region openness. (Saraswati's rights model is detailed in ./saraswati-economy-voice-genre-rights.md.)

Per-region prerequisites, made executable#

The "must be cleared before a region opens" checklist is not a doc — it is the five-profile rules engine evaluateV3PerRegionRules. Each V3PerRegionJurisdictionProfile binds a region to its rating authority, fan-token policy, age floor, content-rating ceiling, and residency zone:

The engine returns block if any of age, content-rating, residency, or fan-token fails; adapt when a rule is satisfied via fallback (Germany's native-token request becomes a v1-account-benefit delivery — adaptationApplied: true); and allow otherwise. Residency is genuinely pinned: passed requires both storageResidencyZone === requiredResidencyZone and the compute zone to be in the profile's allowedComputeResidencyZones, so a German user's storage and compute stay in eu-de. This composes the shared @oshun/data-residency machinery — resolveHomeZoneFromClaim (libs/shared/data-residency/src/home-zone.ts) and the enforcer — whose GDPR Art. 9 special-category branch (blocking a voice_profile or avatar_pack from leaving its zone) is documented in the architecture companion. buildV3PerRegionRulesJurisdictionReport asserts all five profiles are covered and every fixture's decision matches — the gate fails if a jurisdiction is dropped or a rule silently stops applying.

Operator staffing model#

Lilith Operator Console actions are staffed by a mix of shared and per-tenant pools, lean on day one and ramping against measured concurrency. The model is a checked-in artifact (V3/operations/operator-staffing-model.json) with a GA-week rota beside it (V3/operations/on-call-rota-ga-week.json), both gated by verify:v3 operator-runbooks-on-call (§73).

Minimum viable launch (day one)#

Sized for ~5 k DAU growing to ~25 k across wave-1 over 90 days, ~30 staff total:

Lilith-Safety's three shifts carry in-house language coverage — APAC English / Japanese / Hindi, EU and NA English / Spanish / Portuguese — and other wave-1 locales (fr, de, ar, he) route to a localization partner under a 4-hour SLA (localizationPartner.contractId: 'loc-wave1-2026-ga'). The GA-week rota is drilled: a green pager test (ackP95Seconds: 41, missedPages: 0 across PagerDuty, the Slack incident bridge, and the Operator Console) and a follow-the-sun incident-commander rotation (ic-na/ic-eu/ic-apac) covering each launch-week day with a backup.

Scale-up ramp and steady state#

A hiringPlaybook maps measured triggers to staffing actions with named owners and a 60-day lead-time cap: Safety queue p95 SLA breach → +2 operators per affected shift; sustained DAU 50 k → +safety per shift across wave-1; DAU 100 k → full per-region editorial; a new wave-2 region → +1 operator per active pool before the region opens; a large festival → double the safety and capacity rota for the event window. The scaleOutTarget (GA + 12 months, status: 'on-plan') is 8–12 Safety operators per shift, per-region editorial across all pools, and a weekly leadership rebalancing review. Tone policy differs per tenant — Tara contemplative vs. Saraswati pop-cultural — and is maintained per tenant by the owning editorial pool even though in-the-moment moderation runs on the shared Safety pool. Every operator action audit-logs into V1 @oshun/audit-platform; the realm display-name override above is one such audited action.

The honest gate state#

Standalone, both operator-staffing-model.json and on-call-rota-ga-week.json declare releaseGate.status: green. But a passing standalone artifact is structural, not operational, and the aggregate launch-readiness run is the source of truth — and it currently records operator-console-staffed: fail, operator-pools-day-one: fail, and verify:v3 operator-runbooks-on-call: fail among its red gates. The gate fails closed (a missing dependency, an unchecked local task, or evidence drift all flip it red), so the page reports the aggregate's fail, not the standalone green.

Launch readiness and exit criteria#

§75 aggregates everything above into one go/no-go. It is not a checklist someone ticks — it is the output of scripts/v3/verify-v3-launch-readiness.mjs, which executes every verifier command the 39 named gates declare (41 distinct constituent commands), ANDs the results, and writes V3/launch/launch-readiness-release-gate.json. The verifier refuses to pass while the committed artifact disagrees with the run it just performed, so a hand-edited green fails on drift. The artifact's own note records why this machinery exists: a 2026-05-31 audit (V3/VERIFICATION_AUDIT_2026-05-31.md) found a circular gate reporting green over red sub-gates, and the de-circularized verifier exists to make that impossible.

Today's measured state — honestly red#

status: red   ·   passingGateCount: 30 / 41   ·   passingNamedGateCount: 24 / 39   ·   gaReady: false

The red is load-bearing, not a defect in the doc. Gates that pass on real evidence include c2pa-every-export, dsar-v3-session-data, ue-anti-mod-anti-cheat-posture, recording-consent-drills, wave1-region-prerequisites, pxstream-abuse-posture, pxstream-pop-capacity, stadium-banded-seating, saraswati-stadium-concert, minor-protection, royalty-quarter-settlement, and public-web-store-listings. Gates that honestly fail include cross-platform-certification, pixel-streaming-fleet-5x, tara-ga-inventory / saraswati-ga-inventory / commons-ga-inventory, client-build-size-budgets, mobile-deep-links, steam-deck-compatibility, metahuman-mobile-streaming, tier2-fallback-parity, concert-authoring-cadence, operator-console-staffed, operator-pools-day-one, release-thresholds, and sections-1-74-complete. Many fail because they need real fleet load, store certification, or committed UE content this repo's engineering scaffold does not yet carry — the truthful state of a pre-GA build. The aggregate flips to green only when §1–74 carry no unchecked or waived local tasks and all 41 constituent commands exit green — not a moment before.

The high-level checklist behind the gates#

The §75 checklist the gates encode: all three tenants ship the GA inventory (≥ 12 Tara instructors, ≥ 6 Saraswati artists with debut catalogs, ≥ 8 Commons venues); cross-platform clients pass cert / browser-compat / perf budgets across all nine platforms; the Pixel Streaming fleet sustains 5× expected concurrency for 7 consecutive days at p99 ≤ 60 ms and ≥ 99.5 % session-start success; the operator console is fully staffed with documented runbooks; generation-provider failover is drilled and the provenance inspector signed off by Lilith-Rights; and all eval-set release gates plus the adversarial stub scan are green. Accessibility is itself a gate, not an aspiration — the Tier-2 fallback's WCAG 2.2 AA conformance is a launch-readiness item, detailed in ./accessibility-localization-quality.md.

V3 exit criteria#

V3 is feature-complete only when every section above is fully shipped, and the adversarial scan turns up zero actionable hits, and launch-readiness drills have run on all six certification surfaces — web, iOS, Android, Quest 3, Vision Pro, and PSVR 2. The exit bar is deliberately conjunctive: a green eval set with a red fleet gate is not GA, and a green launch gate that drifted from its evidence is — by construction — not possible.

Where this connects#

• ./client-tiers-fidelity-stadium.md — the four-tier client ladder the region router hands a cleared user to, and the stadium banding that launch-readiness gates against.
• ./accessibility-localization-quality.md — the WCAG, localization, and QA gates a region must clear and that the launch gate aggregates.
• ./world-rooms-presence-multiplayer.md — the world server that holds the server-authoritative progression cross-device consistency rides on.
• ./saraswati-economy-voice-genre-rights.md — why Saraswati opens in fewer regions than Tara and Commons at GA.
• ../architecture/observability-performance-security-and-launch.md — the architecture companion: the release-gate machinery, residency enforcement, performance budgets, and the de-circularized launch verifier in engineering depth.
• The feature hub: ../V3_features.md.