Eve library

OSHUN Studio IA Runbook

Runbook

Revision 1

On this page

Status: Active
Owner: OSHUN Studio Web Team
Last Updated: 2026-02-23
Related:

  • docs/adr/ADR-0013-oshun-studio-information-architecture.md
  • docs/releases/v1/specs/studio-domain-dependency-and-data-contract-map.md
  • docs/releases/v1/specs/studio-ia-user-journeys-and-edge-cases.md
  • apps/oshun/web/src/components/studio/StudioInformationArchitectureWorkspace.tsx

Purpose#

Define operational ownership, playbooks, and response procedures for Studio IA flows, including optimistic actions, policy denials, cross-domain dependency failures, and SLO breaches.

1. Ownership Model#

1.1 Service Ownership#

Surface Primary Owner Backup Owner Domain Dependencies
Studio IA route (/studio) OSHUN Web OSHUN Platform Yemaya, Isis, Hathor, Aja, Bellona
Studio context contract OSHUN Platform OSHUN Web All studio domains
Policy enforcement (UI layer) OSHUN Web Security Platform Auth + entitlement contracts
Telemetry and audit events OSHUN Data/Platform OSHUN Web Analytics sinks, BFF telemetry ingress
Flow SLO dashboarding OSHUN SRE OSHUN Platform BFF metrics + domain health signals

1.2 Decision Authority#

  • UX/IA decisions: OSHUN Product + OSHUN Web
  • Contract changes: OSHUN Platform Architecture Council
  • Policy enforcement changes: Security + Domain leads
  • Incident command for P0/P1: OSHUN On-call IC

2. Core SLOs#

Metric Target Alert Threshold
Studio flow success rate >= 99.0% < 98.0% over 15 min
Studio flow p95 latency <= 2000ms > 3000ms over 15 min
Policy denial correctness 100% deterministic Any unauthorized action succeeds
Audit event completeness 100% for run/deny/complete < 99.5% audit emission

3. Operational Playbooks#

3.1 Policy Denial Spike#

Trigger: sudden increase in ROLE_POLICY_DENIED outcomes.

Actions:

  1. Verify active role-policy map in Studio UI config.
  2. Compare against expected permission matrix for current release.
  3. Check entitlement payload changes from auth/bff.
  4. If misconfigured policy map: hotfix UI config.
  5. If auth payload drift: rollback auth contract version or apply compatibility patch.

3.2 Dependency Failure Spike#

Trigger: DEPENDENCY_FAILURE errors exceed threshold.

Actions:

  1. Inspect latest audit log for failing flow and domain step.
  2. Check domain health for Isis/Hathor/Aja/Bellona in parallel.
  3. Enable degraded mode and queue retryable operations.
  4. Notify affected domain on-call with trace IDs.
  5. Validate replay success after domain recovery.

3.3 Context Incomplete Errors#

Trigger: repeated CONTEXT_INCOMPLETE flow errors.

Actions:

  1. Verify route hydration and query/context parser behavior.
  2. Confirm program/project/workspace selectors are loading defaults.
  3. Block destructive actions until context repair.
  4. Patch fallback context resolver if defaulting regressed.

3.4 Latency SLO Breach#

Trigger: p95 flow latency > 3000ms.

Actions:

  1. Identify bottleneck (UI orchestration vs domain response time).
  2. Inspect per-step latency from telemetry payloads.
  3. Reduce synchronous work and increase optimistic/deferred execution where safe.
  4. Enable temporary load-shedding for non-critical enrichment steps.
  5. Track post-mitigation p95 for 30 minutes before clearing incident.

4. Release Checklist (Studio IA)#

  • Validate route /studio renders all required flow cards.
  • Validate role-policy denial and allow paths.
  • Validate optimistic pending states and retry paths.
  • Validate telemetry events for studio_flow_requested, studio_flow_completed, and studio_audit_event.
  • Validate SLO indicator panel updates from latest executions.
  • Validate regression suite for Studio IA component tests.

5. Incident Communication Templates#

5.1 Incident Start#

text
[P{severity}] Studio IA Incident
Detected: {timestamp}
Impact: {flow/domain/role impact summary}
Suspected cause: {policy drift | dependency failure | context regression}
Owner: {on-call}

5.2 Incident Update#

text
[P{severity}] Studio IA Incident Update
Time: {timestamp}
Current mitigation: {degraded mode | rollback | hotfix}
Affected flows: {programBootstrap | pipelineRun | approvalSubmit}
ETA: {time}

5.3 Incident Resolution#

text
[P{severity}] Studio IA Incident Resolved
Resolved: {timestamp}
Root cause: {description}
Fix: {release/hotfix id}
Validation: {tests + telemetry confirmation}
Postmortem: {link}

6. Escalation Paths#

Area Primary Channel Escalate To
Studio web runtime #oshun-web Platform on-call
Domain integration #oshun-platform Domain on-call (Isis/Hathor/Aja/Bellona/Yemaya)
Security/policy #oshun-security Security lead
Program operations (Obsidian) #project-obsidian-ops Executive producer + program tech lead

7. Acceptance Criteria (OST-00008)#

OST-00008 is complete only when:

  1. Operational ownership is explicitly documented.
  2. Studio IA SLOs and alert thresholds are defined.
  3. Incident playbooks exist for policy, dependency, context, and latency issues.
  4. Release checklist exists for Studio IA flow validation.
  5. Escalation paths and communication templates are documented.