Status: Active
Date: April 10, 2026
Owners: OSHUN Platform, OSHUN Metis Engineering, OSHUN Safety
Related:
docs/releases/v1/reports/metis-v1-scorecard.mddocs/releases/v1/runbooks/metis-operator-workflows-runbook.mddocs/releases/v1/audits/metis/media-evaluation.mddocs/releases/v1/specs/todos.md
1. Purpose#
This document defines the canonical SLO layer for Metis generation and tutoring.
It exists so Metis release decisions are driven by explicit cost, latency, quality, and failure targets rather than generic uptime or ad hoc operator judgment.
These SLOs are the thresholds surfaced in the Metis admin monitoring workspace.
2. Scope#
This SLO set covers two launch-critical surfaces:
generationPrerecorded lecture package generation, packaging, export, and release readiness for grounded educational outputs.tutoringLive and text tutoring runtime behavior, learner-facing response cadence, evaluation quality, and failure recovery.
3. Canonical Objectives#
| SLO ID | Surface | Dimension | Target | Warning | Critical | Primary evidence |
|---|---|---|---|---|---|---|
generation_cost |
generation |
cost |
average package cost <= $6.50 |
> $6.50 |
> $8.00 |
persisted lecture package complexity model |
generation_latency |
generation |
latency |
P95 build latency <= 6000 ms |
> 6000 ms |
> 9000 ms |
build-job observability events |
generation_quality |
generation |
quality |
average score >= 85/100 |
< 85 |
< 70 |
lecture media evaluation plus release-readiness fallback |
generation_failure |
generation |
failure |
failure rate <= 5% |
> 5% |
> 10% |
blocked or failed build, export, and rights posture |
tutoring_cost |
tutoring |
cost |
average session cost <= $1.50 |
> $1.50 |
> $2.50 |
persisted session estimated cost |
tutoring_latency |
tutoring |
latency |
P95 tutor response <= 450 ms |
> 450 ms |
> 700 ms |
tutoring response-gap timing |
tutoring_quality |
tutoring |
quality |
average score >= 85/100 |
< 85 |
< 70 |
tutor evaluation, embodiment evaluation, satisfaction fallback |
tutoring_failure |
tutoring |
failure |
failure rate <= 4% |
> 4% |
> 8% |
escalated, abandoned, or stale tutoring sessions |
4. Signal Mapping#
4.1 Generation#
generation_costDerived from the persisted lecture package itself, using a deterministic complexity model over lessons, slides, diagrams, notes, locale variants, render jobs, and export bundles.generation_latencyReadsprocessingTimeMsfrom build-job observability events. When recent event data is absent, the admin workspace falls back to current build readiness evidence so the lane remains explicit rather than silently empty.generation_qualityPrefers persisted lecture media evaluation scores. When those are absent, it uses build-readiness evidence as a lower-fidelity fallback so release posture still degrades visibly.generation_failureCombines explicit build or export event failures with derived blockers from review status, grounded lesson coverage, assessment support, and source-rights posture.
4.2 Tutoring#
tutoring_costUses persisted sessionestimated_costfrom the real tutoring service.tutoring_latencyUses learner-to-assistant message gaps and falls back to session duration per message when direct gaps are unavailable.tutoring_qualityPrefers canonical tutor evaluation reports and tutoring embodiment-quality evaluations, then falls back to learner satisfaction when no persisted evaluations exist yet.tutoring_failureCounts escalated, abandoned, or stale active sessions as failures because these reflect real learner support breakdowns or recovery gaps.
5. Release Interpretation#
- Any
criticalquality or failure SLO is release-blocking. warningcost or latency SLOs require operator review and remediation ownership before release approval.- A release is not considered Metis-green if the SLO layer is unproven because evidence is missing; missing evidence must remain visible as degraded posture.
6. Operational Notes#
- These thresholds are intentionally Metis-specific and should not be replaced by generic shell or Studio uptime targets.
- If the cost model, quality suites, or tutoring latency instrumentation changes, this document and the admin monitoring workspace must change in the same task.
- Any future per-model or per-course exception must be audit-backed and linked from the Metis incident or release-gate workspace.