# OSHUN Studio API Gateway and BFF Composition Runbook

Status: Active  
Owner: OSHUN Studio Architecture + OSHUN Web + OSHUN BFF + Platform SRE + OSHUN
Security + OSHUN Data/Analytics Engineering + Program Operations  
Last Updated: 2026-02-23  
Related:

- `docs/adr/ADR-0046-oshun-studio-api-gateway-and-bff-composition.md`
- `docs/releases/v1/specs/studio-api-gateway-and-bff-composition-dependency-and-data-contract-map.md`
- `docs/releases/v1/specs/studio-api-gateway-and-bff-composition-user-journeys-and-edge-cases.md`
- `apps/oshun/web/src/components/studio/StudioApiGatewayBffCompositionWorkspace.tsx`
- `apps/oshun/web/src/analytics/studioApiGatewayBffCompositionTelemetry.ts`

## Purpose

Define operational ownership, SLOs, incident playbooks, and escalation paths for
Studio API gateway and BFF composition workflows.

This document is the execution artifact for `OST-00272`.

## 1. Ownership Model

| Surface                                         | Primary Owner            | Backup Owner        | Dependencies                                         |
| ----------------------------------------------- | ------------------------ | ------------------- | ---------------------------------------------------- |
| API gateway/BFF composition workspace route     | OSHUN Web                | Platform Web        | route contracts + composition UX orchestration       |
| Gateway route registry and contract lifecycle   | OSHUN BFF                | Studio Architecture | schema/version governance + route ownership          |
| Domain adapter composition and aggregation      | OSHUN BFF + domain teams | Platform SRE        | adapter contracts + transformation pipelines         |
| Policy/entitlement and approval controls        | Security + Compliance    | Program Operations  | role/tier policy bundles + production approval flows |
| Resilience/fallback and degraded-mode execution | Platform SRE + BFF       | Domain operations   | timeout/retry/fallback controls + dependency health  |
| Telemetry and governance audit continuity       | Data Platform + Security | Platform SRE        | analytics sinks + retention/export controls          |

## 2. Core SLOs

| Metric                                         | Target    | Alert Threshold      |
| ---------------------------------------------- | --------- | -------------------- |
| Gateway composition flow success rate          | >= 99.0%  | < 98.0% over 15 min  |
| Route contract integrity                       | >= 99.5%  | < 99.0% over 15 min  |
| Composition execution and recovery flow p95    | <= 2500ms | > 3500ms over 15 min |
| Downstream dependency health signal continuity | >= 99.5%  | < 99.0% over 15 min  |
| Policy-tier/CAB enforcement correctness        | >= 99.5%  | < 99.0% over 15 min  |
| Degraded-mode fallback determinism             | >= 99.5%  | < 99.0% over 15 min  |
| Telemetry/audit event delivery continuity      | >= 99.5%  | < 99.0% over 15 min  |

## 3. Operational Playbooks

### 3.1 Route Contract Publish Failures

Trigger: repeated `API_GATEWAY_ROUTE_PUBLISH_FAILED` outcomes.

Actions:

1. Validate route identity (`routePath`, `method`, `workflowRef`, `version`).
2. Inspect schema compatibility and contract version drift.
3. Re-run publish in preview with contract diagnostics enabled.
4. Block sensitive production actions until route integrity recovers.

### 3.2 Composed Request Execution Failures

Trigger: repeated `API_GATEWAY_COMPOSITION_EXECUTION_FAILED` outcomes.

Actions:

1. Validate request envelope (`requestId`, `traceId`, actor/workspace context).
2. Inspect adapter fan-out order and downstream error patterns.
3. Re-run composed request against known-good dependency snapshots.
4. Escalate to Platform SRE on sustained execution latency/error regressions.

### 3.3 Degraded-Mode Strategy Failures

Trigger: repeated `API_GATEWAY_DEGRADED_MODE_FAILED` outcomes.

Actions:

1. Validate degraded strategy (`fallback_cached`, `fallback_partial`,
   `fallback_minimal`) prerequisites.
2. Inspect dependency health signals and timeout budget drift.
3. Re-run degradation strategy with bounded fallback diagnostics.
4. Enforce release hold while degraded-mode determinism is uncertain.

### 3.4 Sensitive Route Decision Failures

Trigger: repeated `API_GATEWAY_SENSITIVE_ROUTE_DECISION_FAILED` outcomes.

Actions:

1. Validate role/tier/CAB prerequisites and reason codes.
2. Inspect policy bundle drift and deny-path anomalies.
3. Re-run decision with compliance auditor supervision.
4. Freeze production promotion path while policy integrity is degraded.

### 3.5 Composition Health Recovery Failures

Trigger: repeated `API_GATEWAY_COMPOSITION_RECOVERY_FAILED` outcomes.

Actions:

1. Validate remediation action payload and incident-state preconditions.
2. Verify post-fix composition validation before closure.
3. Re-run recovery flow under incident command oversight.
4. Prevent closure until deterministic revalidation is complete.

### 3.6 Policy Denial Spike

Trigger: sustained increase in `ROLE_POLICY_DENIED`,
`PERMISSION_TIER_ROLE_DENIED`, `FLOW_PERMISSION_TIER_INSUFFICIENT`,
`POLICY_JUSTIFICATION_REQUIRED`, `CRITICAL_ROUTE_RESTRICTED_REQUIRED`,
`PRODUCTION_APPROVAL_REQUIRED`, or
`PRODUCTION_DECISION_REQUIRES_COMPLIANCE_AUDITOR`.

Actions:

1. Compare active role/tier/CAB policy bundles against intended controls.
2. Inspect recent policy releases for deny-path regressions.
3. Roll back or patch policy bundle and revalidate decision paths.
4. Confirm denial-rate normalization across all composition flows.

### 3.7 Telemetry and Audit Degradation

Trigger: missing or delayed `studio_api_gateway_bff_composition_*` telemetry
events.

Actions:

1. Validate analytics sink health and ingestion backlog.
2. Enable degraded-observability notice in gateway composition workspace.
3. Replay buffered events where replay semantics are supported.
4. Confirm SLO panel and audit-evidence continuity after recovery.

## 4. Release Checklist

- [ ] Validate `/studio/api-gateway-bff-composition` renders and executes
      required flows.
- [ ] Validate optimistic, error, and retry behavior for all composition flows.
- [ ] Validate permission-tier and production CAB policy enforcement.
- [ ] Validate telemetry/audit emissions and SLO panel updates.
- [ ] Validate integration/e2e/accessibility/performance evidence artifacts.

## 5. Incident Communication Templates

### Incident Start

```text
[P{severity}] Studio API Gateway/BFF Composition Incident
Detected: {timestamp}
Impact: {route-publish | request-execution | degraded-mode | policy-decision | recovery}
Suspected cause: {policy drift | schema regression | dependency degradation | telemetry degradation}
Owner: {on-call}
```

### Incident Update

```text
[P{severity}] Studio API Gateway/BFF Composition Update
Time: {timestamp}
Mitigation: {rollback | hotfix | policy patch | release hold}
Affected flows: {publishGatewayRouteContract | executeComposedRequest | applyDegradedCompositionStrategy | decideSensitiveRouteAccess | recoverCompositionHealth}
ETA: {time}
```

### Incident Resolved

```text
[P{severity}] Studio API Gateway/BFF Composition Incident Resolved
Resolved: {timestamp}
Root cause: {description}
Fix: {release/hotfix id}
Validation: {tests + telemetry checks}
Postmortem: {link}
```

## 6. Escalation Paths

| Area                                     | Primary Channel        | Escalate To           |
| ---------------------------------------- | ---------------------- | --------------------- |
| Gateway composition workspace UX         | `#oshun-web`           | Platform on-call      |
| Route contracts and composition APIs     | `#oshun-bff`           | BFF lead              |
| Dependency health and fallback runtime   | `#oshun-platform-sre`  | SRE lead              |
| Policy-tier/CAB governance controls      | `#oshun-security`      | Security + Compliance |
| Composition analytics and audit evidence | `#oshun-data-platform` | Data platform lead    |

## 7. Acceptance Criteria (OST-00272)

`OST-00272` is complete only when:

1. Ownership and decision authority are documented.
2. API gateway/BFF composition SLOs and alert thresholds are documented.
3. At least five playbooks are documented for major failure modes.
4. Release checklist, communication templates, and escalation paths exist.
