# OSHUN V1 Metis Generation And Tutoring SLOs

Status: Active

Date: April 10, 2026

Owners: OSHUN Platform, OSHUN Metis Engineering, OSHUN Safety

Related:

- `docs/releases/v1/reports/metis-v1-scorecard.md`
- `docs/releases/v1/runbooks/metis-operator-workflows-runbook.md`
- `docs/releases/v1/audits/metis/media-evaluation.md`
- `docs/releases/v1/specs/todos.md`

## 1. Purpose

This document defines the canonical SLO layer for Metis generation and tutoring.

It exists so Metis release decisions are driven by explicit cost, latency,
quality, and failure targets rather than generic uptime or ad hoc operator
judgment.

These SLOs are the thresholds surfaced in the Metis admin monitoring workspace.

## 2. Scope

This SLO set covers two launch-critical surfaces:

1. `generation` Prerecorded lecture package generation, packaging, export, and
   release readiness for grounded educational outputs.
2. `tutoring` Live and text tutoring runtime behavior, learner-facing response
   cadence, evaluation quality, and failure recovery.

## 3. Canonical Objectives

| SLO ID               | Surface      | Dimension | Target                          | Warning     | Critical    | Primary evidence                                               |
| -------------------- | ------------ | --------- | ------------------------------- | ----------- | ----------- | -------------------------------------------------------------- |
| `generation_cost`    | `generation` | `cost`    | average package cost `<= $6.50` | `> $6.50`   | `> $8.00`   | persisted lecture package complexity model                     |
| `generation_latency` | `generation` | `latency` | P95 build latency `<= 6000 ms`  | `> 6000 ms` | `> 9000 ms` | build-job observability events                                 |
| `generation_quality` | `generation` | `quality` | average score `>= 85/100`       | `< 85`      | `< 70`      | lecture media evaluation plus release-readiness fallback       |
| `generation_failure` | `generation` | `failure` | failure rate `<= 5%`            | `> 5%`      | `> 10%`     | blocked or failed build, export, and rights posture            |
| `tutoring_cost`      | `tutoring`   | `cost`    | average session cost `<= $1.50` | `> $1.50`   | `> $2.50`   | persisted session estimated cost                               |
| `tutoring_latency`   | `tutoring`   | `latency` | P95 tutor response `<= 450 ms`  | `> 450 ms`  | `> 700 ms`  | tutoring response-gap timing                                   |
| `tutoring_quality`   | `tutoring`   | `quality` | average score `>= 85/100`       | `< 85`      | `< 70`      | tutor evaluation, embodiment evaluation, satisfaction fallback |
| `tutoring_failure`   | `tutoring`   | `failure` | failure rate `<= 4%`            | `> 4%`      | `> 8%`      | escalated, abandoned, or stale tutoring sessions               |

## 4. Signal Mapping

### 4.1 Generation

- `generation_cost` Derived from the persisted lecture package itself, using a
  deterministic complexity model over lessons, slides, diagrams, notes, locale
  variants, render jobs, and export bundles.
- `generation_latency` Reads `processingTimeMs` from build-job observability
  events. When recent event data is absent, the admin workspace falls back to
  current build readiness evidence so the lane remains explicit rather than
  silently empty.
- `generation_quality` Prefers persisted lecture media evaluation scores. When
  those are absent, it uses build-readiness evidence as a lower-fidelity
  fallback so release posture still degrades visibly.
- `generation_failure` Combines explicit build or export event failures with
  derived blockers from review status, grounded lesson coverage, assessment
  support, and source-rights posture.

### 4.2 Tutoring

- `tutoring_cost` Uses persisted session `estimated_cost` from the real tutoring
  service.
- `tutoring_latency` Uses learner-to-assistant message gaps and falls back to
  session duration per message when direct gaps are unavailable.
- `tutoring_quality` Prefers canonical tutor evaluation reports and tutoring
  embodiment-quality evaluations, then falls back to learner satisfaction when
  no persisted evaluations exist yet.
- `tutoring_failure` Counts escalated, abandoned, or stale active sessions as
  failures because these reflect real learner support breakdowns or recovery
  gaps.

## 5. Release Interpretation

- Any `critical` quality or failure SLO is release-blocking.
- `warning` cost or latency SLOs require operator review and remediation
  ownership before release approval.
- A release is not considered Metis-green if the SLO layer is unproven because
  evidence is missing; missing evidence must remain visible as degraded posture.

## 6. Operational Notes

- These thresholds are intentionally Metis-specific and should not be replaced
  by generic shell or Studio uptime targets.
- If the cost model, quality suites, or tutoring latency instrumentation
  changes, this document and the admin monitoring workspace must change in the
  same task.
- Any future per-model or per-course exception must be audit-backed and linked
  from the Metis incident or release-gate workspace.
