Lilith Metaverse · Runbooks

Lilith Operator Console Action Runbooks

attention state.

1section5 minread

On this page

runbook-set: v3-operator-console-actions.v1 owner: Lilith Operations

Every action below has an expected SLA and a verification command. Operators must record the action id in the audit event before closing the case.

Actions#

action: live-room-dashboard-triage#

  • Owner: Lilith-Safety
  • Trigger: room health, report queue, or launch-decision dashboard enters attention state.
  • SLA: acknowledge within 2 minutes; first mitigation within 5 minutes.
  • Steps: open the room dashboard, confirm tenant and region, inspect active reports, check Pixel Streaming POP health, assign commander when multi-team work is needed.
  • Rollback: return room routing to the previous capacity policy after green telemetry for 10 minutes.
  • Escalation: incident commander, Pixel Streaming on-call, then Lilith-Safety lead.
  • Audit event: operator.live_room_dashboard.triaged
  • Verification: pnpm verify:v3 lilith-live-room-dashboard

action: room-takeover#

  • Owner: Lilith-Safety
  • Trigger: credible abuse, room overload, or host absence.
  • SLA: acknowledge within 90 seconds; complete takeover within 4 minutes.
  • Steps: lock room mutations, assign operator host, publish user-facing banner, preserve evidence bundle, and hand back only after the incident commander approves.
  • Rollback: unlock room mutations and remove the banner after safety review.
  • Escalation: Lilith-Safety lead, incident commander, then legal/privacy owner if evidence includes protected data.
  • Audit event: operator.room_takeover.completed
  • Verification: pnpm verify:v3 lilith-room-takeover

action: concert-calendar-approval#

  • Owner: Saraswati Editorial
  • Trigger: concert schedule moves from draft to publish review.
  • SLA: review within 1 business day; hotfix schedule within 30 minutes.
  • Steps: validate region availability, ticket policy, recording consent model, pre-warm reservation, C2PA recording plan, and operator coverage.
  • Rollback: unpublish calendar entry and notify ticket holders through the incident communication template.
  • Escalation: Saraswati Editorial lead, Commerce lead, then release captain.
  • Audit event: operator.concert_calendar.approved
  • Verification: pnpm verify:v3 lilith-concert-calendar-approval

action: instructor-verification-queue#

  • Owner: Tara Editorial
  • Trigger: instructor submits or updates credentials.
  • SLA: review within 2 business days; minor-facing corrections within 4 hours.
  • Steps: verify credentials, background-check state, liability insurance, lineage display, voice consent, and minor-safety eligibility.
  • Rollback: suspend instructor publishing and remove minor-facing availability.
  • Escalation: Tara Editorial lead, minor-safety reviewer, then legal owner.
  • Audit event: operator.instructor_verification.reviewed
  • Verification: pnpm verify:v3 lilith-instructor-verification-queue

action: generation-queue-inspector#

  • Owner: Lilith-Editorial Ops
  • Trigger: generation provider outage, policy hold, or queue age breach.
  • SLA: acknowledge within 5 minutes; route failover within 15 minutes.
  • Steps: inspect queued jobs, compare policy verdicts, move blocked jobs to editorial review, and route eligible jobs to the approved failover provider.
  • Rollback: restore primary provider after the provider health gate is green for 30 minutes.
  • Escalation: generation provider owner, Lilith-Editorial Ops lead, then release captain for launch-blocking queues.
  • Audit event: operator.generation_queue.inspected
  • Verification: pnpm verify:v3 lilith-generation-queue-inspector

action: takedown-executor#

  • Owner: Lilith-Rights
  • Trigger: rights request, contributor withdrawal, or policy adjudication.
  • SLA: acknowledge within 4 business hours; urgent takedown within 1 hour.
  • Steps: validate claimant, freeze affected artifact, execute takedown cascade, update C2PA/provenance references, and notify impacted customers.
  • Rollback: restore artifact only after adjudication reversal and provenance repair are complete.
  • Escalation: Lilith-Rights lead, Isis Provenance owner, then legal owner.
  • Audit event: operator.takedown.executed
  • Verification: pnpm verify:v3 lilith-takedown-executor

action: provenance-inspector-signoff#

  • Owner: Isis Provenance
  • Trigger: artifact moves from review to publish.
  • SLA: review within 1 business day; launch-blocking failures within 2 hours.
  • Steps: inspect source chain, policy verdicts, C2PA manifest, AJE mint state, reviewer identity, and export hash.
  • Rollback: return artifact to review and invalidate the pending publish token.
  • Escalation: Isis Provenance owner, editorial lead for the tenant, then release captain.
  • Audit event: operator.provenance.signoff
  • Verification: pnpm verify:v3 lilith-provenance-inspector

action: refund-tax-reversal#

  • Owner: Lilith-Commerce
  • Trigger: user cancellation, provider outage cancellation, chargeback, or tax correction.
  • SLA: acknowledge within 1 business day; outage refunds within 2 hours.
  • Steps: verify entitlement, ticket or subscription state, tax jurisdiction, Stripe reversal, royalty waterfall adjustment, and customer notice.
  • Rollback: reopen billing case and hold royalty payout when reversal fails.
  • Escalation: Lilith-Commerce lead, finance owner, then Lilith-Rights when royalty settlement is affected.
  • Audit event: operator.commerce.refund_tax_reversal
  • Verification: pnpm verify:v3 lilith-refund-tax-reversal

action: waitlist-promotion#

  • Owner: Lilith-Commerce
  • Trigger: event capacity opens or verified instructor class capacity changes.
  • SLA: process within 15 minutes for live events; 1 business day otherwise.
  • Steps: rank waitlist, check regional eligibility, reserve capacity, send join offer, and expire stale offers.
  • Rollback: release reserved slot and notify next eligible user.
  • Escalation: Lilith-Commerce lead, capacity on-call, then event commander.
  • Audit event: operator.waitlist.promoted
  • Verification: pnpm verify:v3 lilith-waitlist-promotion-flow

action: incident-communications#

  • Owner: Incident Commander
  • Trigger: service degradation, POP outage, provider outage, or recording worker failure.
  • SLA: first internal update within 5 minutes; public update within 15 minutes for customer-impacting incidents.
  • Steps: classify severity, select template, publish status update, schedule the next update, and attach postmortem owner.
  • Rollback: correct status copy through an amended update rather than deleting the incident timeline.
  • Escalation: incident commander, communications lead, then executive on-call for severity-1 incidents.
  • Audit event: operator.incident_communications.published
  • Verification: pnpm verify:v3 grafana-dashboards

action: release-readiness-decision#

  • Owner: V3 Release Captain
  • Trigger: GA, hotfix, or regional wave approval.
  • SLA: decision captured before any production rollout.
  • Steps: check §70 QA, §71 docs, staffing, region, security, capacity, inventory, and launch-readiness evidence before approving.
  • Rollback: revoke rollout token and move the release back to blocked.
  • Escalation: release captain, engineering lead, product lead, then executive approver.
  • Audit event: operator.release_readiness.decision
  • Verification: pnpm verify:v3 documentation-runbooks

action: privacy-dsar-review#

  • Owner: Lilith-Privacy
  • Trigger: DSAR request, consent withdrawal, or recording redaction request.
  • SLA: acknowledge within 1 business day; statutory deadline tracked per region.
  • Steps: collect V3 session data, consent ledger entries, recordings, exports, and V1-linked records; approve redaction or export bundle.
  • Rollback: reopen request with corrected evidence and attach amended export.
  • Escalation: Lilith-Privacy lead, legal owner, then incident commander if a statutory deadline is at risk.
  • Audit event: operator.privacy_dsar.reviewed
  • Verification: pnpm verify:v3 dsar-coverage