Status: Active
Owner: OSHUN Studio Architecture + OSHUN Web + OSHUN BFF + OSHUN Security +
Program Operations
Last Updated: 2026-02-23
Related:
docs/adr/ADR-0037-oshun-studio-asset-preview-pipeline.mddocs/releases/v1/specs/studio-asset-preview-pipeline-dependency-and-data-contract-map.mddocs/releases/v1/specs/studio-asset-preview-pipeline-user-journeys-and-edge-cases.mdapps/oshun/web/src/components/studio/StudioAssetPreviewPipelineWorkspace.tsxapps/oshun/web/src/analytics/studioAssetPreviewPipelineTelemetry.ts
Purpose#
Define operational ownership, SLOs, incident playbooks, and escalation paths for Studio asset preview pipeline workflows.
This document is the execution artifact for OST-00200.
1. Ownership Model#
| Surface | Primary Owner | Backup Owner | Dependencies |
|---|---|---|---|
| Asset preview workspace route | OSHUN Web | Platform Web | route contracts + preview UX orchestration |
| Preview request envelope persistence | OSHUN BFF | Platform SRE | preview APIs + revision metadata contracts |
| Render lifecycle orchestration | Platform SRE | Rendering Platform | render workers + lifecycle state services |
| Fidelity fallback policy governance | Security + Compliance | Studio Architecture | profile policy bundles + entitlement checks |
| Cache invalidation and refresh | Platform SRE + CDN Ops | BFF | cache services + revision-key lineage |
| Governance evidence export pipeline | Data Platform + Security | Program Operations | audit streams + retention/export controls |
2. Core SLOs#
| Metric | Target | Alert Threshold |
|---|---|---|
| Asset preview flow success rate | >= 99.0% | < 98.0% over 15 min |
| Render lifecycle reliability | >= 99.5% | < 99.0% over 15 min |
| Asset preview flow p95 latency | <= 2300ms | > 3300ms over 15 min |
| Fidelity fallback determinism | >= 99.5% | < 99.0% over 15 min |
| Cache invalidation/refresh correctness | >= 99.5% | < 99.0% over 15 min |
| Asset preview audit event completeness | >= 99.5% | < 99.0% over 15 min |
3. Operational Playbooks#
3.1 Preview Request Failures#
Trigger: repeated PREVIEW_REQUEST_FAILED outcomes.
Actions:
- Validate preview request envelope (
previewId,assetRef,profile,classification,revision) completeness. - Inspect request adapter output for missing revision or asset identity data.
- Re-run preview request flow in preview channel with diagnostics enabled.
- Block release promotion until request envelope validation stabilizes.
3.2 Render Lifecycle Failures#
Trigger: repeated PREVIEW_RENDER_LIFECYCLE_FAILED outcomes.
Actions:
- Validate queued -> rendering -> ready transition contracts and guardrails.
- Inspect render-worker traces for non-deterministic lifecycle transitions.
- Re-run lifecycle orchestration with state tracing enabled.
- Freeze production render promotions until lifecycle reliability recovers.
3.3 Fidelity Fallback Failures#
Trigger: repeated PREVIEW_FIDELITY_FALLBACK_FAILED outcomes.
Actions:
- Validate fallback profile mapping and deterministic downgrade rules.
- Inspect fallback execution traces for profile lineage drift.
- Re-run fallback flow with policy diagnostics enabled.
- Disable profile downgrade rollout when fallback integrity cannot be proven.
3.4 Cache Invalidation and Refresh Failures#
Trigger: repeated PREVIEW_CACHE_INVALIDATION_FAILED outcomes.
Actions:
- Validate invalidation reason, revision key, and refresh lineage contracts.
- Inspect cache service traces for stale-key retention or refresh races.
- Re-run cache invalidation flow with constrained blast radius.
- Escalate to CDN ops if refresh misses remain above alert threshold.
3.5 Governance Export Failures#
Trigger: repeated PREVIEW_GOVERNANCE_EXPORT_FAILED outcomes.
Actions:
- Validate evidence package completeness for lifecycle, fallback, and cache lineage metadata.
- Inspect export pipeline for schema or retention-tagging mismatches.
- Re-run evidence export with compliance reviewer oversight.
- Block production evidence export until package validation passes.
3.6 Policy Denial Spike#
Trigger: unexpected rise in ROLE_POLICY_DENIED,
CONFIDENTIAL_PREVIEW_POLICY_DENIED, PRODUCTION_HIGH_FIDELITY_POLICY_DENIED,
or PRODUCTION_GOVERNANCE_EXPORT_POLICY_DENIED.
Actions:
- Compare flow allowlists against active RBAC and policy bundles.
- Inspect recent policy release changes for drift.
- Apply policy hotfix or roll back drifted policy bundle.
- Confirm deny-rate normalization and impacted flow recovery.
3.7 Telemetry and Audit Degradation#
Trigger: missing or delayed asset preview telemetry and audit events.
Actions:
- Validate analytics sink health and ingestion backlog depth.
- Enable degraded-observability warning in preview workspace.
- Replay buffered events where supported.
- Confirm SLO and evidence stream completeness after recovery.
4. Release Checklist#
- Validate
/studio/asset-preview-pipelinerenders and executes required flows. - Validate optimistic, error, and retry behavior for each flow.
- Validate profile, fallback, cache, and governance policy enforcement.
- Validate telemetry/audit emissions and SLO panel updates.
- Validate integration/e2e/accessibility evidence artifacts.
5. Incident Communication Templates#
Incident Start#
[P{severity}] Studio Asset Preview Pipeline Incident
Detected: {timestamp}
Impact: {preview-request | render-lifecycle | fidelity-fallback | cache-refresh | governance-export}
Suspected cause: {policy drift | render pipeline regression | cache invalidation failure}
Owner: {on-call}
Incident Update#
[P{severity}] Studio Asset Preview Pipeline Update
Time: {timestamp}
Mitigation: {rollback | hotfix | degraded mode}
Affected flows: {requestPreviewRender | orchestrateRenderLifecycle | executeFidelityFallback | invalidateAndRefreshCache | exportPreviewGovernanceEvidence}
ETA: {time}
Incident Resolved#
[P{severity}] Studio Asset Preview Pipeline Incident Resolved
Resolved: {timestamp}
Root cause: {description}
Fix: {release/hotfix id}
Validation: {tests + telemetry checks}
Postmortem: {link}
6. Escalation Paths#
| Area | Primary Channel | Escalate To |
|---|---|---|
| Web runtime and preview workspace | #oshun-web |
Platform on-call |
| Preview request and metadata services | #oshun-bff |
BFF lead |
| Render lifecycle and cache services | #oshun-platform-sre |
SRE lead |
| Policy/fallback governance | #oshun-security |
Security lead |
| Audit/compliance evidence pipelines | #oshun-data-platform |
Data platform/SRE lead |
7. Acceptance Criteria (OST-00200)#
OST-00200 is complete only when:
- Ownership and decision authority are documented.
- Asset preview pipeline SLOs and alert thresholds are documented.
- At least five playbooks are documented for major failure modes.
- Release checklist, communication templates, and escalation paths exist.