# OSHUN Studio Security Hardening Program Runbook

Status: Active  
Owner: OSHUN Studio Architecture + OSHUN Web + OSHUN BFF + OSHUN Security +
Compliance + Platform SRE + OSHUN Data/Analytics Engineering + Program
Operations  
Last Updated: 2026-02-23  
Related:

- `docs/adr/ADR-0049-oshun-studio-security-hardening-program.md`
- `docs/releases/v1/specs/studio-security-hardening-program-dependency-and-data-contract-map.md`
- `docs/releases/v1/specs/studio-security-hardening-program-user-journeys-and-edge-cases.md`
- `apps/oshun/web/src/components/studio/StudioSecurityHardeningProgramWorkspace.tsx`
- `apps/oshun/web/src/analytics/studioSecurityHardeningProgramTelemetry.ts`

## Purpose

Define operational ownership, SLOs, incident playbooks, and escalation paths for
Studio security hardening workflows.

This document is the execution artifact for `OST-00296`.

## 1. Ownership Model

| Surface                                      | Primary Owner              | Backup Owner        | Dependencies                                                 |
| -------------------------------------------- | -------------------------- | ------------------- | ------------------------------------------------------------ |
| Security hardening workspace route           | OSHUN Web                  | Platform Web        | workflow UX orchestration + incident navigation              |
| Identity/session and scope validation engine | OSHUN BFF + Security       | Studio Architecture | auth/session contracts + scope-policy packages               |
| Runtime trust and abuse controls             | Security Engineering + SRE | BFF platform        | attestation sources + abuse heuristics + deny policy bundles |
| Transport pinning transition controls        | Platform SRE + Security    | Release Operations  | verifier support + cert lifecycle + rollout controls         |
| Sensitive policy/approval governance         | Security + Compliance      | Program Operations  | role/tier enforcement + CAB controls + waiver lifecycle      |
| Telemetry and governance audit continuity    | Data Platform + Security   | Platform SRE        | analytics sinks + retention/export + replay capabilities     |

## 2. Core SLOs

| Metric                                         | Target    | Alert Threshold      |
| ---------------------------------------------- | --------- | -------------------- |
| Security hardening flow success rate           | >= 99.0%  | < 98.0% over 15 min  |
| Identity/session and scope validation fidelity | >= 99.5%  | < 99.0% over 15 min  |
| Runtime trust/pinning/recovery flow p95        | <= 2500ms | > 3500ms over 15 min |
| Runtime integrity deny-path correctness        | >= 99.5%  | < 99.0% over 15 min  |
| Policy-tier/CAB enforcement correctness        | >= 99.5%  | < 99.0% over 15 min  |
| Incident recovery closure determinism          | >= 99.5%  | < 99.0% over 15 min  |
| Telemetry/audit event delivery continuity      | >= 99.5%  | < 99.0% over 15 min  |

## 3. Operational Playbooks

### 3.1 Identity and Domain-Scope Validation Failures

Trigger: repeated `SECURITY_HARDENING_IDENTITY_SCOPE_FAILED` outcomes.

Actions:

1. Validate actor/session context (`actorId`, `workspaceId`, `requestId`,
   `traceId`, `domainScope`, token expiry).
2. Inspect scope-policy version drift and auth provider envelope mismatches.
3. Re-run validation in preview with diagnostics enabled.
4. Block sensitive and restricted operations until validation integrity
   recovers.

### 3.2 Runtime Trust and Abuse-Control Denials

Trigger: repeated `SECURITY_HARDENING_RUNTIME_TRUST_FAILED` outcomes.

Actions:

1. Validate integrity status, attestation freshness, and request-rate metadata.
2. Inspect abuse-threshold configuration and signal-ingestion stability.
3. Re-run runtime trust flow with bounded traffic and known-good attestation.
4. Escalate to Security Engineering and SRE for sustained deny-path anomalies.

### 3.3 Sensitive Security Policy Decision Failures

Trigger: repeated `SECURITY_HARDENING_POLICY_DECISION_FAILED` outcomes.

Actions:

1. Validate role/tier/CAB/waiver prerequisites and reason codes.
2. Inspect policy bundle drift, role assignment drift, and approval evidence.
3. Re-run decision with compliance-auditor oversight.
4. Freeze production-sensitive decisions while policy integrity is degraded.

### 3.4 Transport Pinning Transition Failures

Trigger: repeated `SECURITY_HARDENING_PINNING_TRANSITION_FAILED` outcomes.

Actions:

1. Validate selected transport policy metadata and verifier capability.
2. Inspect mismatch-rate trend and pinset/cert lifecycle state.
3. Re-run transition in report-only or staged mode before enforcement.
4. Hold production rollout until mismatch and verification baselines recover.

### 3.5 Incident Recovery and Closure Failures

Trigger: repeated `SECURITY_HARDENING_RECOVERY_FAILED` outcomes.

Actions:

1. Validate incident state, severity policy, and remediation-action payload.
2. Verify post-fix identity/runtime/pinning checks before closure.
3. Re-run recovery flow under incident command oversight.
4. Prevent closure until deterministic revalidation passes.

### 3.6 Telemetry and Audit Degradation

Trigger: missing or delayed `studio_security_hardening_program_*` telemetry
events.

Actions:

1. Validate analytics sink health and ingestion backlog.
2. Enable degraded-observability banner in security hardening workspace.
3. Replay buffered events where replay semantics are supported.
4. Confirm SLO panel and audit-evidence continuity after recovery.

## 4. Release Checklist

- [ ] Validate `/studio/security-hardening-program` renders and executes
      required flows.
- [ ] Validate optimistic, error, and retry behavior across all security
      hardening flows.
- [ ] Validate permission-tier, waiver, and production CAB policy enforcement.
- [ ] Validate telemetry/audit emissions and SLO panel updates.
- [ ] Validate integration/e2e/accessibility/performance evidence artifacts.

## 5. Incident Communication Templates

### Incident Start

```text
[P{severity}] Studio Security Hardening Incident
Detected: {timestamp}
Impact: {identity-scope | runtime-trust | policy-decision | pinning-transition | incident-recovery}
Suspected cause: {policy drift | attestation drift | cert lifecycle regression | abuse-threshold drift | telemetry degradation}
Owner: {on-call}
```

### Incident Update

```text
[P{severity}] Studio Security Hardening Update
Time: {timestamp}
Mitigation: {rollback | hotfix | policy patch | traffic constraint | release hold}
Affected flows: {validateIdentityAndDomainScope | enforceRuntimeTrustAndAbuseControls | decideSensitiveSecurityPolicyAction | executeTransportSecurityPinningTransition | recoverSecurityHardeningIncident}
ETA: {time}
```

### Incident Resolved

```text
[P{severity}] Studio Security Hardening Incident Resolved
Resolved: {timestamp}
Root cause: {description}
Fix: {release/hotfix id}
Validation: {tests + telemetry checks}
Postmortem: {link}
```

## 6. Escalation Paths

| Area                                      | Primary Channel        | Escalate To           |
| ----------------------------------------- | ---------------------- | --------------------- |
| Security hardening workspace UX           | `#oshun-web`           | Platform on-call      |
| Identity/runtime/pinning orchestration    | `#oshun-bff`           | BFF lead              |
| Runtime trust and transport operations    | `#oshun-platform-sre`  | SRE lead              |
| Policy-tier/CAB/waiver governance control | `#oshun-security`      | Security + Compliance |
| Telemetry and audit reporting continuity  | `#oshun-data-platform` | Data platform lead    |

## 7. Acceptance Criteria (OST-00296)

`OST-00296` is complete only when:

1. Ownership and decision authority are documented.
2. Security hardening SLOs and alert thresholds are documented.
3. At least five playbooks are documented for major failure modes.
4. Release checklist, communication templates, and escalation paths exist.
