Eve library

OSHUN Studio Search and Global Discovery Runbook

Runbook

Revision 1

On this page

Status: Active
Owner: OSHUN Studio Product Design + OSHUN Web + OSHUN BFF
Last Updated: 2026-02-23
Related:

  • docs/adr/ADR-0021-oshun-studio-search-and-global-discovery.md
  • docs/releases/v1/specs/studio-search-and-global-discovery-dependency-and-data-contract-map.md
  • docs/releases/v1/specs/studio-search-and-global-discovery-user-journeys-and-edge-cases.md
  • apps/oshun/web/src/components/studio/StudioSearchDiscoveryWorkspace.tsx
  • apps/oshun/bff/src/routes/search.ts

Purpose#

Define operational ownership, SLOs, incident playbooks, and escalation paths for Studio search and global discovery governance workflows.

This document is the execution artifact for OST-00072.

1. Ownership Model#

Surface Primary Owner Backup Owner Dependencies
Search/discovery workspace route OSHUN Web Platform Web route + analytics contracts
Search API and ranking logic OSHUN BFF Data Platform BFF route + ranking modules
Discovery UI and action routing OSHUN Web + Navigation Product Design route mapping + context hydration
Policy and scope enforcement Security + Product Ops Platform Identity entitlement/session payloads
Telemetry and audit pipeline Data Platform SRE sinks and ingestion pipelines

2. Core SLOs#

Metric Target Alert Threshold
Search/discovery flow success rate >= 99.0% < 98.0% over 15 min
Ranking determinism validation integrity 100% deterministic any false pass
Cross-domain parity sync success rate >= 99.5% < 99.0% over 15 min
Search/discovery flow p95 latency <= 2200ms > 3200ms over 15 min
Search/discovery audit completeness >= 99.5% < 99.0% over 15 min

3. Operational Playbooks#

3.1 Discovery Index Publish Failures#

Trigger: repeated DISCOVERY_INDEX_SCHEMA_FAILED outcomes.

Actions:

  1. Inspect index schema and query contract diagnostics.
  2. Validate compatibility with route payload contracts.
  3. Re-run publish in preview with diagnostics enabled.
  4. Roll back to last certified discovery bundle if needed.

3.2 Relevance Calibration Failures#

Trigger: repeated RELEVANCE_CALIBRATION_FAILED outcomes.

Actions:

  1. Compare ranking outputs to deterministic baseline fixtures.
  2. Validate tie-break behavior and domain-intent weighting.
  3. Apply ranking calibration adjustments.
  4. Re-run validation and block promotion until pass.

3.3 Cross-Domain Discovery Drift#

Trigger: DISCOVERY_PARITY_DRIFT findings.

Actions:

  1. Generate drift report by domain and result-action mapping.
  2. Re-apply canonical discovery contracts.
  3. Re-run parity checks.
  4. Freeze release channel promotion until parity is restored.

3.4 Policy Denial Spike#

Trigger: unexpected increase in ROLE_POLICY_DENIED outcomes.

Actions:

  1. Compare discovery policy map against entitlement contracts.
  2. Verify recent identity payload/version changes.
  3. Apply policy hotfix or rollback entitlement drift.
  4. Confirm deny-rate normalization after mitigation.

3.5 Telemetry and Audit Degradation#

Trigger: missing or delayed search/discovery telemetry events.

Actions:

  1. Validate analytics sink health and queue depth.
  2. Enable degraded-observability warning in workspace.
  3. Reconcile missed events from buffered traces.
  4. Confirm event completeness and SLO recovery.

4. Release Checklist#

  • Validate /studio/search-discovery renders and executes all required flows.
  • Validate optimistic, error, and retry behavior for each flow.
  • Validate role-policy allow/deny behavior.
  • Validate telemetry/audit emissions and SLO panel updates.
  • Validate integration/e2e/accessibility evidence artifacts.

5. Incident Communication Templates#

Incident Start#

text
[P{severity}] Studio Search/Discovery Incident
Detected: {timestamp}
Impact: {index/calibration/parity impact}
Suspected cause: {schema issue | ranking regression | parity drift | policy drift}
Owner: {on-call}

Incident Update#

text
[P{severity}] Studio Search/Discovery Update
Time: {timestamp}
Mitigation: {rollback | hotfix | degraded mode}
Affected flows: {discoveryIndexPublish | relevanceCalibrationValidation | crossDomainSearchParitySync}
ETA: {time}

Incident Resolved#

text
[P{severity}] Studio Search/Discovery Incident Resolved
Resolved: {timestamp}
Root cause: {description}
Fix: {release/hotfix id}
Validation: {tests + telemetry checks}
Postmortem: {link}

6. Escalation Paths#

Area Primary Channel Escalate To
Web runtime and workspace #oshun-web Platform on-call
Search API/ranking behavior #oshun-bff Search architecture lead
Navigation and result routing #oshun-navigation Navigation architecture lead
Policy and permissions #oshun-security Security lead
Telemetry pipeline #oshun-data-platform SRE lead

7. Acceptance Criteria (OST-00072)#

OST-00072 is complete only when:

  1. Ownership and decision authority are documented.
  2. Search/discovery SLOs and alert thresholds are documented.
  3. At least five playbooks are documented for major failure modes.
  4. Release checklist, communication templates, and escalation paths exist.