Eve library

OSHUN Studio Observability and Operational Dashboards Journeys and Edge Cases

Spec

Revision 1

On this page

Status: Active
Owner: OSHUN Studio Architecture + OSHUN Web + Platform SRE + OSHUN Data/Analytics Engineering
Last Updated: 2026-02-23
Related:

  • docs/adr/ADR-0041-oshun-studio-observability-and-operational-dashboards.md
  • docs/releases/v1/specs/studio-observability-and-operational-dashboards-dependency-and-data-contract-map.md

Purpose#

Define primary journeys and deterministic edge-case behavior for Studio observability and operational dashboard workflows.

This document is the execution artifact for OST-00227.

Personas#

  • Program Operator
  • Collaboration Lead
  • Platform SRE
  • Security Reviewer
  • Compliance Auditor

Journey Principles#

  1. Every operational signal must map to canonical SLI/SLO semantics.
  2. Alert triage must preserve full domain/workflow/ownership attribution.
  3. Remediation actions must enforce role, permission-tier, and release policy.
  4. Dashboard actions must emit telemetry and audit records deterministically.

1. Primary Journeys#

J1. Monitor Cross-Domain Operational Health#

Goal: View Studio-wide SLI/SLO posture across Yemaya, Isis, Hathor, Aja, Bellona, and OSHUN control surfaces.

Entry: Observability and operational dashboards workspace

Steps:

  1. Load dashboard signal cards by domain and window.
  2. Review status (healthy, warning, degraded, critical) and trend indicators.
  3. Compare current values vs SLO targets.
  4. Identify highest-impact breached signals.
  5. Persist operational snapshot for shift handoff.

Success State:

  • Operator can prioritize the top risks using one canonical dashboard view.

J2. Drill Down from Signal to Dependency Root Cause#

Goal: Isolate likely root cause from a degraded signal card.

Entry: Dashboard signal card in warning/degraded/critical state

Steps:

  1. Open signal drill-down panel.
  2. Review workflow identity, source domain, and dependency graph.
  3. Inspect upstream/downstream impact and correlated alerts.
  4. Open trace/evidence references for the selected window.
  5. Produce root-cause hypothesis for triage action.

Success State:

  • Incident hypothesis is generated with attributable dependency evidence.

J3. Execute Policy-Aware Remediation Action#

Goal: Trigger safe remediation from dashboard context.

Entry: Alert/incident card requiring intervention

Steps:

  1. Select remediation action (retry, rollback, gate hold, escalation).
  2. Validate role and permission-tier eligibility.
  3. Supply CAB approval ticket for production-impacting actions.
  4. Execute remediation and track action status.
  5. Emit remediation audit evidence linked to alert and signal.

Success State:

  • Remediation executes only when policy constraints are satisfied.

J4. Drive Alert Lifecycle to Resolution#

Goal: Move alerts through deterministic triage states.

Entry: Active alert queue

Steps:

  1. Acknowledge alert and assign an owner.
  2. Transition state to investigating with contextual notes.
  3. Record remediation and verification evidence.
  4. Resolve and close alert once SLO/signal posture stabilizes.
  5. Export closure evidence for governance reporting.

Success State:

  • Alert lifecycle is complete with deterministic state transitions and evidence.

J5. Evaluate Operational Readiness for Release#

Goal: Gate production promotion on operational readiness.

Entry: Release checkpoint in Project Obsidian control flow

Steps:

  1. Evaluate SLO rollup snapshot for all participating domains.
  2. Validate telemetry freshness and completeness indicators.
  3. Confirm no unresolved critical alerts or policy violations.
  4. Approve or block release with explicit reason codes.
  5. Emit release-readiness decision event with attribution.

Success State:

  • Release decision is deterministic, auditable, and evidence-backed.

2. Critical Edge Cases#

E1. Signal Card Staleness#

  • Trigger: updatedAt exceeds freshness window.
  • Required behavior: mark signal stale/degraded and block readiness decisions.

E2. Missing SLO Target#

  • Trigger: signal card has no target value.
  • Required behavior: quarantine card and raise configuration defect alert.

E3. Domain Not in Registry#

  • Trigger: signal source domain absent from canonical domain registry.
  • Required behavior: block drill-down and emit domain-mapping violation event.
  • Trigger: alert exists without signalId.
  • Required behavior: block state transitions and route to data-quality queue.

E5. Invalid Alert State Transition#

  • Trigger: attempted transition skips lifecycle rules (for example, new -> resolved).
  • Required behavior: deny transition and emit policy violation audit event.

E6. Remediation Action Without Permission Tier#

  • Trigger: action requested without valid permission-tier context.
  • Required behavior: deny action and require explicit tier selection.

E7. Production Remediation Without CAB Ticket#

  • Trigger: production-impacting action has missing/invalid CAB evidence.
  • Required behavior: hard deny action and emit approval-policy denial event.

E8. Drill-Down Dependency Graph Timeout#

  • Trigger: dependency graph query exceeds timeout budget.
  • Required behavior: show partial graph + fallback diagnostics + retry action.

E9. Conflicting Concurrent Alert Updates#

  • Trigger: two operators mutate the same alert lifecycle simultaneously.
  • Required behavior: enforce optimistic concurrency and preserve causal order.

E10. Release-Gate Decision Without Telemetry Completeness#

  • Trigger: readiness gate executed with low telemetry completeness score.
  • Required behavior: auto-block release and emit readiness-quality denial.

3. Journey-to-Surface Mapping#

Journey Primary Surface Supporting Surfaces
J1 Monitor Operational Health Observability/operational dashboards Metrics instrumentation + SLO rollup feeds
J2 Drill Down to Root Cause Dashboard drill-down panels Domain dependency graph + trace services
J3 Execute Policy-Aware Remediation Dashboard action panel RBAC/policy engine + CAB governance
J4 Drive Alert Lifecycle to Resolution Dashboard alert queue Incident routing + governance exports
J5 Evaluate Operational Release Readout Dashboard readiness gate panel Release orchestration + quality services

4. Instrumentation Requirements#

Required events:

  • observability_dashboards.journey.started
  • observability_dashboards.journey.step.completed
  • observability_dashboards.journey.blocked
  • observability_dashboards.journey.recovered
  • observability_dashboards.journey.completed

Required dimensions:

  • signalId, alertId, sourceDomain, workflowId, dependencyRef
  • actorId, actorRole, permissionTier, workspaceId, releaseChannel
  • journeyId, stepId, outcome, latencyMs, errorCode, reasonCode
  • sloTarget, sloActual, telemetryCompletenessRate, approvalTicketRef

5. Acceptance Criteria (OST-00227)#

OST-00227 is complete only when:

  1. At least five observability/dashboard journeys are documented.
  2. At least ten deterministic edge cases are documented.
  3. Journey-to-surface mapping is explicit.
  4. Instrumentation requirements and dimensions are documented.