Eve library

OSHUN Studio Security Hardening Program Runbook

Runbook

Revision 1

On this page

Status: Active
Owner: OSHUN Studio Architecture + OSHUN Web + OSHUN BFF + OSHUN Security + Compliance + Platform SRE + OSHUN Data/Analytics Engineering + Program Operations
Last Updated: 2026-02-23
Related:

  • docs/adr/ADR-0049-oshun-studio-security-hardening-program.md
  • docs/releases/v1/specs/studio-security-hardening-program-dependency-and-data-contract-map.md
  • docs/releases/v1/specs/studio-security-hardening-program-user-journeys-and-edge-cases.md
  • apps/oshun/web/src/components/studio/StudioSecurityHardeningProgramWorkspace.tsx
  • apps/oshun/web/src/analytics/studioSecurityHardeningProgramTelemetry.ts

Purpose#

Define operational ownership, SLOs, incident playbooks, and escalation paths for Studio security hardening workflows.

This document is the execution artifact for OST-00296.

1. Ownership Model#

Surface Primary Owner Backup Owner Dependencies
Security hardening workspace route OSHUN Web Platform Web workflow UX orchestration + incident navigation
Identity/session and scope validation engine OSHUN BFF + Security Studio Architecture auth/session contracts + scope-policy packages
Runtime trust and abuse controls Security Engineering + SRE BFF platform attestation sources + abuse heuristics + deny policy bundles
Transport pinning transition controls Platform SRE + Security Release Operations verifier support + cert lifecycle + rollout controls
Sensitive policy/approval governance Security + Compliance Program Operations role/tier enforcement + CAB controls + waiver lifecycle
Telemetry and governance audit continuity Data Platform + Security Platform SRE analytics sinks + retention/export + replay capabilities

2. Core SLOs#

Metric Target Alert Threshold
Security hardening flow success rate >= 99.0% < 98.0% over 15 min
Identity/session and scope validation fidelity >= 99.5% < 99.0% over 15 min
Runtime trust/pinning/recovery flow p95 <= 2500ms > 3500ms over 15 min
Runtime integrity deny-path correctness >= 99.5% < 99.0% over 15 min
Policy-tier/CAB enforcement correctness >= 99.5% < 99.0% over 15 min
Incident recovery closure determinism >= 99.5% < 99.0% over 15 min
Telemetry/audit event delivery continuity >= 99.5% < 99.0% over 15 min

3. Operational Playbooks#

3.1 Identity and Domain-Scope Validation Failures#

Trigger: repeated SECURITY_HARDENING_IDENTITY_SCOPE_FAILED outcomes.

Actions:

  1. Validate actor/session context (actorId, workspaceId, requestId, traceId, domainScope, token expiry).
  2. Inspect scope-policy version drift and auth provider envelope mismatches.
  3. Re-run validation in preview with diagnostics enabled.
  4. Block sensitive and restricted operations until validation integrity recovers.

3.2 Runtime Trust and Abuse-Control Denials#

Trigger: repeated SECURITY_HARDENING_RUNTIME_TRUST_FAILED outcomes.

Actions:

  1. Validate integrity status, attestation freshness, and request-rate metadata.
  2. Inspect abuse-threshold configuration and signal-ingestion stability.
  3. Re-run runtime trust flow with bounded traffic and known-good attestation.
  4. Escalate to Security Engineering and SRE for sustained deny-path anomalies.

3.3 Sensitive Security Policy Decision Failures#

Trigger: repeated SECURITY_HARDENING_POLICY_DECISION_FAILED outcomes.

Actions:

  1. Validate role/tier/CAB/waiver prerequisites and reason codes.
  2. Inspect policy bundle drift, role assignment drift, and approval evidence.
  3. Re-run decision with compliance-auditor oversight.
  4. Freeze production-sensitive decisions while policy integrity is degraded.

3.4 Transport Pinning Transition Failures#

Trigger: repeated SECURITY_HARDENING_PINNING_TRANSITION_FAILED outcomes.

Actions:

  1. Validate selected transport policy metadata and verifier capability.
  2. Inspect mismatch-rate trend and pinset/cert lifecycle state.
  3. Re-run transition in report-only or staged mode before enforcement.
  4. Hold production rollout until mismatch and verification baselines recover.

3.5 Incident Recovery and Closure Failures#

Trigger: repeated SECURITY_HARDENING_RECOVERY_FAILED outcomes.

Actions:

  1. Validate incident state, severity policy, and remediation-action payload.
  2. Verify post-fix identity/runtime/pinning checks before closure.
  3. Re-run recovery flow under incident command oversight.
  4. Prevent closure until deterministic revalidation passes.

3.6 Telemetry and Audit Degradation#

Trigger: missing or delayed studio_security_hardening_program_* telemetry events.

Actions:

  1. Validate analytics sink health and ingestion backlog.
  2. Enable degraded-observability banner in security hardening workspace.
  3. Replay buffered events where replay semantics are supported.
  4. Confirm SLO panel and audit-evidence continuity after recovery.

4. Release Checklist#

  • Validate /studio/security-hardening-program renders and executes required flows.
  • Validate optimistic, error, and retry behavior across all security hardening flows.
  • Validate permission-tier, waiver, and production CAB policy enforcement.
  • Validate telemetry/audit emissions and SLO panel updates.
  • Validate integration/e2e/accessibility/performance evidence artifacts.

5. Incident Communication Templates#

Incident Start#

text
[P{severity}] Studio Security Hardening Incident
Detected: {timestamp}
Impact: {identity-scope | runtime-trust | policy-decision | pinning-transition | incident-recovery}
Suspected cause: {policy drift | attestation drift | cert lifecycle regression | abuse-threshold drift | telemetry degradation}
Owner: {on-call}

Incident Update#

text
[P{severity}] Studio Security Hardening Update
Time: {timestamp}
Mitigation: {rollback | hotfix | policy patch | traffic constraint | release hold}
Affected flows: {validateIdentityAndDomainScope | enforceRuntimeTrustAndAbuseControls | decideSensitiveSecurityPolicyAction | executeTransportSecurityPinningTransition | recoverSecurityHardeningIncident}
ETA: {time}

Incident Resolved#

text
[P{severity}] Studio Security Hardening Incident Resolved
Resolved: {timestamp}
Root cause: {description}
Fix: {release/hotfix id}
Validation: {tests + telemetry checks}
Postmortem: {link}

6. Escalation Paths#

Area Primary Channel Escalate To
Security hardening workspace UX #oshun-web Platform on-call
Identity/runtime/pinning orchestration #oshun-bff BFF lead
Runtime trust and transport operations #oshun-platform-sre SRE lead
Policy-tier/CAB/waiver governance control #oshun-security Security + Compliance
Telemetry and audit reporting continuity #oshun-data-platform Data platform lead

7. Acceptance Criteria (OST-00296)#

OST-00296 is complete only when:

  1. Ownership and decision authority are documented.
  2. Security hardening SLOs and alert thresholds are documented.
  3. At least five playbooks are documented for major failure modes.
  4. Release checklist, communication templates, and escalation paths exist.