Disciplines · Launch

Post-Deploy Monitoring — V1 Launch Gate

The dashboards below are the canonical machine-readable specifications in

7sections5 minread
Markdown source
On this page

Gate owner: SRE Lead + Release Captain. Required for §33.2. Verifies every launch-critical dashboard is wired, populated, alerting, and pulled into the launch war-room before GA.

1. Dashboard inventory#

The dashboards below are the canonical machine-readable specifications in libs/oshun/analytics/src/dashboards-*.ts. Each definition compiles into the production Grafana dashboard via the dashboard-emitter pipeline.

1.1 Service health#

  • Source — libs/oshun/analytics/src/dashboards-service-health.ts.
  • Required panels — availability, latency p50/p90/p99, error rate, CPU/memory/IO saturation, SLO compliance, request volume.
  • Targets — every entry in OSHUN_V1_SERVICE_HEALTH_TARGETS (every Iris subsystem plus the five shared dependencies: Postgres, Redis, object storage, search index, queue).
  • Live verification — unit-level test at libs/oshun/analytics/src/__tests__/dashboards-service-health.test.ts proves the definition is well-formed (run via npx vitest run libs/oshun/analytics/src/__tests__/dashboards-service-health.test.ts per CLAUDE.md §Worktrees). The SRE lead emits the compiled definitions to production Grafana via the dashboard provisioners under docker/observability/grafana/provisioning/dashboards/ (mirror provisioners exist for domain-specific stacks under deploy/yemaya/grafana/, infra/monitoring/iris/grafana/, and infrastructure/psyche/docker/grafana/).

1.2 Customer KPI#

  • Source — libs/oshun/analytics/src/dashboards-customer-kpi.ts.
  • Required panels — D1/D7 retention, onboarding completion, crash- free sessions, ANR rate, web JS-error rate, NPS, conversion.
  • Live verification — same emitter.

1.3 Assistant#

  • Source — libs/oshun/analytics/src/dashboards-assistant.ts.
  • Required panels — grounded-answer rate, citation rate, persona- tone drift, memory writes, privacy-pause toggle rate, persona switch latency.

1.4 Queue health#

  • Source — libs/oshun/analytics/src/dashboards-queue-health.ts.
  • Required panels — moderation queue depth, p95 dwell, appeals queue, persona-approval queue, model-promotion queue, incident queue.

1.5 Satisfaction#

  • Source — libs/oshun/analytics/src/dashboards-satisfaction.ts.
  • Required panels — CSAT, NPS by cohort, support contact rate, shake-to-report submissions.

1.6 Nisaba#

  • Source — libs/oshun/analytics/src/dashboards-nisaba.ts.
  • Required panels — grounding-source bind rate, passage open p95, annotation save success rate, mobile-resume continuity rate.

2. Alerts inventory#

Source: libs/oshun/analytics/src/alerts-manifest.ts.

Required alert categories live and routed before GA:

Category Severity Routed to Verified by
audit_lag sev1 Pager + #incidents paging test in last 7 d
grounding_degradation sev1 Pager + #incidents paging test in last 7 d
citation_degradation sev2 #incidents paging test in last 7 d
memory_privacy sev1 Pager + #incidents paging test in last 7 d
provider_outage sev1 Pager + #incidents paging test in last 7 d
provider_failover sev2 #incidents paging test in last 7 d
crisis_flow_regression sev1 Pager + #incidents paging test in last 7 d
persona_drift sev2 #persona-ops paging test in last 7 d
tenant_isolation sev1 Pager + #security paging test in last 7 d
moderation_backlog sev1 Pager + #moderation paging test in last 7 d
dsar_failure sev1 Pager + #privacy paging test in last 7 d
watermark_verification sev2 #provenance-ops paging test in last 7 d
canary_rollback sev1 Pager + #launch-war-room rehearsed in rollout dry run
slo_breach sev2 #incidents paging test in last 7 d

Routes are defined in libs/oshun/analytics/src/incident-ownership-manifest.ts.

3. Synthetic monitors#

Synthetic monitors run continuously, executing critical journeys against the production cluster. The runtime is built on the shared monitor framework at libs/iris/testing/synthetic/src/synthetic-monitor.ts; each Playwright spec in apps/oshun/web/e2e/ listed below ships a "synthetic" mode that hits the production cluster on the cadence given.

Journey Source spec (Playwright, in synthetic mode) Frequency Region
Tara ritual end-to-end apps/oshun/web/e2e/tara-full-journey.spec.ts every 5 min us-west-2 + eu-west-1
Arete daily check-in apps/oshun/web/e2e/arete-full-journey.spec.ts every 5 min us-west-2
Veritas briefing apps/oshun/web/e2e/veritas-full-journey.spec.ts every 5 min us-west-2
Nyx tonight view apps/oshun/web/e2e/nyx-full-journey.spec.ts every 10 min us-west-2
Nisaba passage + annotation apps/oshun/web/e2e/nisaba-reading-compare-annotate.spec.ts every 10 min us-west-2
Metis grounded course apps/metis/web/e2e/critical-paths.spec.ts every 15 min us-west-2
Assistant grounded answer apps/oshun/web/e2e/assistant-grounding-state.spec.ts every 5 min us-west-2
Sign-up + sign-in + sign-out apps/oshun/web/e2e/auth-entry-flows.spec.ts every 5 min us-west-2 + eu-west-1 + ap-southeast-2
DSAR export round-trip apps/oshun/admin/e2e/privacy-dsar-end-to-end.spec.ts every 1 h us-west-2
Watermark verify libs/aphrodite/stream-watermarking/src/ + libs/aja/content-watermarking/src/ verifier suites driven by a synthetic worker job every 1 h us-west-2

Failures auto-page on three consecutive misses.

4. RUM (Real-User Monitoring)#

RUM ingested for web + mobile + admin. Required dimensions:

  • Substrate (web / iOS / Android / admin).
  • Cohort (canary / pilot / general).
  • Region.
  • Device class.
  • Locale + RTL.
  • Network class.
  • Persona (where applicable).

RUM panels live on dashboards-customer-kpi and feed the rollout-promotion auto-decision in docs/launch/canary-analysis.md.

5. War-room consoles#

The launch war-room aggregates the dashboards into a single Grafana playlist:

  1. Service health.
  2. Customer KPI.
  3. Assistant.
  4. Queue health.
  5. Satisfaction.
  6. Nisaba.
  7. Rollout progress (per-substrate gauge from the canary-analysis service).
  8. Open alerts (filtered to severity in (sev1, sev2)).

The playlist is provisioned via the Grafana provisioner config under docker/observability/grafana/provisioning/dashboards/ and verified in the 14 days preceding GA.

6. Live verification checklist#

Required for §33.2 signoff:

  • Every dashboard in §1 is emitted to production Grafana and visually populated (not empty / not "no data").
  • Every alert in §2 has a paging test recorded in the last 7 days.
  • Every synthetic monitor in §3 has a successful run within the last hour and a failing run that paged the on-call within the last 30 d (proves alert wiring works).
  • RUM data is present for every substrate-cohort combination.
  • War-room playlist (§5) loads with all panels populated.
  • Customer-experience monitor for tara-full-journey completed in every required region within the last hour.

Evidence: screenshots + emitter run logs committed at docs/releases/v1/verification/post-deploy-monitoring/YYYY-MM-DD/.

7. Cross-references#

  • Dashboards: libs/oshun/analytics/src/dashboards-*.ts.
  • Alerts: libs/oshun/analytics/src/alerts-manifest.ts.
  • Tracing manifest: libs/oshun/analytics/src/tracing-manifest.ts.
  • Incident ownership: libs/oshun/analytics/src/incident-ownership-manifest.ts.
  • Rollout plan: docs/launch/rollout-plan.md.
  • Canary analysis: docs/launch/canary-analysis.md.
  • Go/no-go: docs/launch/go-no-go.md.