Status: Active
Owner: OSHUN Studio Architecture + OSHUN Web + OSHUN BFF + Data Platform +
Platform SRE + OSHUN Security + Program Operations
Last Updated: 2026-02-23
Related:
docs/adr/ADR-0040-oshun-studio-metrics-and-analytics-instrumentation.mddocs/releases/v1/specs/studio-metrics-and-analytics-instrumentation-dependency-and-data-contract-map.mddocs/releases/v1/specs/studio-metrics-and-analytics-instrumentation-user-journeys-and-edge-cases.mdapps/oshun/web/src/components/studio/StudioMetricsAnalyticsInstrumentationWorkspace.tsxapps/oshun/web/src/analytics/studioMetricsAnalyticsInstrumentationTelemetry.ts
Purpose#
Define operational ownership, SLOs, incident playbooks, and escalation paths for Studio metrics and analytics instrumentation workflows.
This document is the execution artifact for OST-00224.
1. Ownership Model#
| Surface | Primary Owner | Backup Owner | Dependencies |
|---|---|---|---|
| Metrics instrumentation workspace route | OSHUN Web | Platform Web | route contracts + instrumentation UX orchestration |
| Metric definition publishing and lineage | OSHUN BFF + Data Platform | Platform SRE | schema registry + entity/ownership contracts |
| Flow-span and KPI evaluation orchestration | Data Platform | BFF | analytics sinks + threshold evaluation services |
| Permission-tier and CAB release policy checks | Security + Compliance | Program Operations | RBAC/policy bundles + governance approval systems |
| Audit evidence and telemetry quality gating | Platform SRE + Compliance | Data Platform | telemetry ingestion + quality verifiers + audit export |
2. Core SLOs#
| Metric | Target | Alert Threshold |
|---|---|---|
| Metrics instrumentation flow success rate | >= 99.0% | < 98.0% over 15 min |
| Metric-definition publish integrity | >= 99.5% | < 99.0% over 15 min |
| Metrics instrumentation flow p95 latency | <= 2500ms | > 3500ms over 15 min |
| Audit-linkage completeness and canonicality | >= 99.5% | < 99.0% over 15 min |
| Release-gate verdict determinism | >= 99.5% | < 99.0% over 15 min |
| Telemetry quality/audit evidence continuity | >= 99.5% | < 99.0% over 15 min |
| Permission-tier and policy-denial anomaly detection | >= 99.0% | < 98.0% over 15 min |
3. Operational Playbooks#
3.1 Metric Definition Publish Failures#
Trigger: repeated METRIC_DEFINITION_PUBLISH_FAILED outcomes.
Actions:
- Validate metric identity (
metricId,metricName,owner) and threshold metadata. - Inspect schema/contract registry compatibility for publish payload drift.
- Re-run publish in preview channel with contract diagnostics enabled.
- Block production promotion until definition integrity stabilizes.
3.2 Flow Span Emission Degradation#
Trigger: repeated FLOW_SPAN_EMISSION_FAILED outcomes.
Actions:
- Validate requested/completed span pair payload and flow attribution fields.
- Inspect analytics sink backpressure and span lifecycle ordering.
- Re-run span emission with tracing enabled and bounded traffic replay.
- Escalate to Platform SRE if p95 latency or completion gaps persist.
3.3 Audit-Linkage Contract Failures#
Trigger: repeated AUDIT_ANALYTICS_LINKAGE_FAILED outcomes.
Actions:
- Validate canonical audit event name and lineage mapping fields.
- Inspect governance mapping services for version or schema drift.
- Re-run linkage in staging with compliance observer sign-off.
- Freeze production linkage updates until CAB-approved remediation is applied.
3.4 KPI Evaluation and Threshold Contract Failures#
Trigger: repeated KPI_EVALUATION_FAILED outcomes.
Actions:
- Validate warning/critical thresholds and KPI composition inputs.
- Inspect threshold evaluation engine for rounding or window drift.
- Re-run KPI evaluation against known-good fixtures.
- Disable promotion gates that depend on invalid KPI verdicts until fixed.
3.5 Telemetry Quality Release-Gate Failures#
Trigger: repeated TELEMETRY_QUALITY_GATE_FAILED outcomes.
Actions:
- Validate coverage, linkage, and quality-state prerequisites.
- Inspect completeness and schema-violation streams for regression sources.
- Re-run release gate with compliance auditor supervision.
- Hold production release path until gate returns deterministic verdict.
3.6 Permission and Policy Denial Spike#
Trigger: sustained increase in ROLE_POLICY_DENIED,
PERMISSION_TIER_ROLE_DENIED, FLOW_PERMISSION_TIER_INSUFFICIENT,
POLICY_JUSTIFICATION_REQUIRED, RESTRICTED_DOMAIN_REQUIRES_RESTRICTED_TIER,
PRODUCTION_APPROVAL_TICKET_REQUIRED, or
PRODUCTION_RELEASE_GATE_REQUIRES_COMPLIANCE_AUDITOR.
Actions:
- Compare active role/permission-tier policies with intended policy bundles.
- Inspect recent policy release deltas for unintended deny-path regressions.
- Roll back or patch policy bundle and verify expected allow/deny behavior.
- Confirm deny-rate normalization and recovery across all instrumentation flows.
3.7 Telemetry and Audit Evidence Degradation#
Trigger: missing or delayed studio_metrics_analytics_instrumentation_*
telemetry events.
Actions:
- Validate analytics sink health and backlog depth.
- Enable degraded-observability banner in metrics workspace operations.
- Replay buffered telemetry where replay semantics are supported.
- Verify SLO dashboards and governance evidence stream completeness.
4. Release Checklist#
- Validate
/studio/metrics-analytics-instrumentationrenders and executes required flows. - Validate optimistic, error, and retry behavior for all instrumentation flows.
- Validate permission-tier, restricted-domain, and production CAB policy enforcement.
- Validate telemetry/audit emissions and SLO panel updates.
- Validate integration/e2e/accessibility/performance evidence artifacts.
5. Incident Communication Templates#
Incident Start#
[P{severity}] Studio Metrics/Analytics Instrumentation Incident
Detected: {timestamp}
Impact: {metric-publish | span-emission | audit-linkage | kpi-evaluation | release-gate}
Suspected cause: {policy drift | schema regression | telemetry degradation | threshold contract drift}
Owner: {on-call}
Incident Update#
[P{severity}] Studio Metrics/Analytics Instrumentation Update
Time: {timestamp}
Mitigation: {rollback | hotfix | policy patch | release hold}
Affected flows: {publishMetricDefinition | emitFlowSpanPair | linkAuditAnalyticsOutcome | composeKpiThresholdEvaluation | runTelemetryQualityReleaseGate}
ETA: {time}
Incident Resolved#
[P{severity}] Studio Metrics/Analytics Instrumentation Incident Resolved
Resolved: {timestamp}
Root cause: {description}
Fix: {release/hotfix id}
Validation: {tests + telemetry checks}
Postmortem: {link}
6. Escalation Paths#
| Area | Primary Channel | Escalate To |
|---|---|---|
| Web runtime and instrumentation UX | #oshun-web |
Platform on-call |
| Metrics contracts and BFF APIs | #oshun-bff |
BFF lead |
| Telemetry ingestion and quality gating | #oshun-platform-sre |
SRE lead |
| Policy/entitlement/CAB governance rules | #oshun-security |
Security + Compliance |
| Analytics evidence and reporting | #oshun-data-platform |
Data platform/SRE lead |
7. Acceptance Criteria (OST-00224)#
OST-00224 is complete only when:
- Ownership and decision authority are documented.
- Metrics/analytics instrumentation SLOs and alert thresholds are documented.
- At least five playbooks are documented for major failure modes.
- Release checklist, communication templates, and escalation paths exist.