# OSHUN Studio Commenting and Annotation System Runbook

Status: Active  
Owner: OSHUN Studio Architecture + OSHUN Web + OSHUN BFF + OSHUN Security +
Program Operations  
Last Updated: 2026-02-23  
Related:

- `docs/adr/ADR-0034-oshun-studio-commenting-and-annotation-system.md`
- `docs/releases/v1/specs/studio-commenting-and-annotation-system-dependency-and-data-contract-map.md`
- `docs/releases/v1/specs/studio-commenting-and-annotation-system-user-journeys-and-edge-cases.md`
- `apps/oshun/web/src/components/studio/StudioCommentingAnnotationSystemWorkspace.tsx`
- `apps/oshun/web/src/analytics/studioCommentingAnnotationSystemTelemetry.ts`

## Purpose

Define operational ownership, SLOs, incident playbooks, and escalation paths for
Studio commenting and annotation workflows.

This document is the execution artifact for `OST-00176`.

## 1. Ownership Model

| Surface                                    | Primary Owner            | Backup Owner        | Dependencies                                         |
| ------------------------------------------ | ------------------------ | ------------------- | ---------------------------------------------------- |
| Commenting/annotation workspace route      | OSHUN Web                | Platform Web        | route contracts + UI orchestration                   |
| Annotation/thread persistence services     | OSHUN BFF                | Platform SRE        | thread persistence + lifecycle state transitions     |
| Target binding and domain adapters         | Domain Integrators       | Studio Architecture | entity/frame/token adapters across domain boundaries |
| Moderation and policy decisioning          | Security + Compliance    | Program Operations  | policy bundles + moderation workflows                |
| Annotation telemetry and governance export | Data Platform + Security | Platform SRE        | analytics ingestion + retention controls             |

## 2. Core SLOs

| Metric                                  | Target             | Alert Threshold         |
| --------------------------------------- | ------------------ | ----------------------- |
| Annotation flow success rate            | >= 99.0%           | < 98.0% over 15 min     |
| Thread lifecycle transition reliability | >= 99.5%           | < 99.0% over 15 min     |
| Commenting/annotation flow p95 latency  | <= 2200ms          | > 3200ms over 15 min    |
| Thread ordering determinism             | 100% deterministic | any ordering divergence |
| Moderation decision completeness        | >= 99.5%           | < 99.0% over 15 min     |
| Annotation audit event completeness     | >= 99.5%           | < 99.0% over 15 min     |

## 3. Operational Playbooks

### 3.1 Create Annotation Failures

Trigger: repeated `ANNOTATION_CREATE_FAILED` outcomes.

Actions:

1. Validate envelope requirements (`annotationId`, `threadId`, `targetRef`,
   `actorId`, `content`, `occurredAt`) are present and correctly typed.
2. Inspect target-binding adapter output for missing/invalid target references.
3. Re-run creation flow in preview with diagnostics enabled.
4. Block promotion until envelope validation stability normalizes.

### 3.2 Reply Thread Expansion Failures

Trigger: repeated `ANNOTATION_REPLY_FAILED` outcomes.

Actions:

1. Validate parent-thread continuity and deterministic ordering semantics.
2. Inspect reply payload for invalid parent linkage or thread mismatch.
3. Re-run reply flow with ordering diagnostics enabled.
4. Freeze release when thread continuity invariants fail.

### 3.3 Resolve Lifecycle Failures

Trigger: repeated `ANNOTATION_RESOLVE_FAILED` outcomes.

Actions:

1. Validate resolution rationale and owner attribution metadata.
2. Inspect lifecycle transition logs for illegal state transitions.
3. Re-run resolve flow with lifecycle trace logging enabled.
4. Escalate to Studio architecture if resolution contract drift persists.

### 3.4 Reopen Lifecycle Failures

Trigger: repeated `ANNOTATION_REOPEN_FAILED` outcomes.

Actions:

1. Validate resolved-state prerequisites and reopen reason metadata.
2. Inspect reopen history and closure evidence lineage.
3. Re-run reopen flow with history-integrity checks enabled.
4. Lock rollout when reopen integrity checks fail repeatedly.

### 3.5 Moderation Governance Failures

Trigger: repeated `ANNOTATION_MODERATION_FAILED` outcomes.

Actions:

1. Validate moderation action, policy ID, and rationale metadata.
2. Inspect moderation decision pipeline and policy evaluation traces.
3. Re-run moderation flow with security reviewer oversight.
4. Block production moderation rollout until policy evidence checks pass.

### 3.6 Policy Denial Spike

Trigger: unexpected rise in `ROLE_POLICY_DENIED`,
`DOMAIN_VISIBILITY_POLICY_DENIED`, `REOPEN_ESCALATION_POLICY_DENIED`, or
`PRODUCTION_MODERATION_POLICY_DENIED`.

Actions:

1. Compare flow role allowlists with active RBAC/policy bundles.
2. Inspect entitlement/policy release diffs for drift.
3. Apply policy hotfix or roll back drifted policy bundle.
4. Confirm deny-rate normalization and operator impact resolution.

### 3.7 Telemetry and Audit Degradation

Trigger: missing or delayed commenting/annotation telemetry/audit events.

Actions:

1. Validate analytics sink health and ingestion backlog depth.
2. Enable degraded-observability warning in workspace operations.
3. Replay buffered events where supported.
4. Confirm evidence stream completeness after recovery.

## 4. Release Checklist

- [ ] Validate `/studio/commenting-annotation-system` renders and executes
      required flows.
- [ ] Validate optimistic, error, and retry behavior for each flow.
- [ ] Validate role-policy and moderation governance enforcement.
- [ ] Validate telemetry/audit emissions and SLO panel updates.
- [ ] Validate integration/e2e/accessibility evidence artifacts.

## 5. Incident Communication Templates

### Incident Start

```
[P{severity}] Studio Commenting/Annotation Incident
Detected: {timestamp}
Impact: {annotation-create | thread-reply | lifecycle-resolve | lifecycle-reopen | moderation-review}
Suspected cause: {policy drift | thread continuity failure | lifecycle transition regression}
Owner: {on-call}
```

### Incident Update

```
[P{severity}] Studio Commenting/Annotation Update
Time: {timestamp}
Mitigation: {rollback | hotfix | degraded mode}
Affected flows: {createContextBoundAnnotation | replyThreadExpansion | acknowledgeResolveAnnotation | reopenResolvedAnnotation | moderationGovernanceReview}
ETA: {time}
```

### Incident Resolved

```
[P{severity}] Studio Commenting/Annotation Incident Resolved
Resolved: {timestamp}
Root cause: {description}
Fix: {release/hotfix id}
Validation: {tests + telemetry checks}
Postmortem: {link}
```

## 6. Escalation Paths

| Area                                   | Primary Channel        | Escalate To            |
| -------------------------------------- | ---------------------- | ---------------------- |
| Web runtime and annotation workspace   | `#oshun-web`           | Platform on-call       |
| Annotation/thread persistence services | `#oshun-bff`           | BFF lead               |
| Lifecycle reliability and replay       | `#oshun-platform-sre`  | SRE lead               |
| Moderation and policy governance       | `#oshun-security`      | Security lead          |
| Audit/compliance evidence pipelines    | `#oshun-data-platform` | Data platform/SRE lead |

## 7. Acceptance Criteria (OST-00176)

`OST-00176` is complete only when:

1. Ownership and decision authority are documented.
2. Commenting/annotation SLOs and alert thresholds are documented.
3. At least five playbooks are documented for major failure modes.
4. Release checklist, communication templates, and escalation paths exist.
