Status: Active
Date: April 10, 2026
Owners: OSHUN Metis Product, OSHUN Metis Admin Engineering, OSHUN Safety, OSHUN Support, OSHUN Program Operations
Related:
docs/releases/v1/reports/metis-v1-scorecard.mddocs/releases/v1/reports/metis-v1-workstream-tracking.mddocs/releases/v1/reports/metis-subsystem-ownership-matrix.mddocs/releases/v1/specs/todos.mdapps/metis/admin/src/app/page.tsxapps/metis/admin/src/app/reviews/page.tsxapps/metis/admin/src/app/rights/page.tsxapps/metis/admin/src/app/content/page.tsxapps/metis/admin/src/app/complaints/page.tsxapps/metis/admin/src/app/users/page.tsxapps/metis/admin/src/app/monitoring/page.tsxapps/metis/admin/src/app/incidents/page.tsxapps/metis/admin/src/app/audit/page.tsx
1. Purpose#
This document is the canonical runbook and training guide for Metis operator workflows in OSHUN V1.
It exists to answer five operating questions in one place:
- Which Metis admin surfaces are considered launch-critical operator tooling?
- Which operator role owns each queue, lane, or escalation path?
- What is the correct step-by-step flow for review, rights, moderation, support, learner operations, monitoring, and incident command?
- What training and signoff must an operator complete before working without supervision?
- What evidence must be present before an operator closes, escalates, or approves a Metis action?
This is a Metis-specific execution artifact, not a generic Studio operations document.
2. Operator Surface Map#
| Surface | Route | Primary users | Core purpose | Must produce |
|---|---|---|---|---|
| Dashboard | / |
shift lead, incident commander | start-of-shift posture, quick routing, queue pressure review | queue routing decision |
| Reviews | /reviews |
curriculum reviewer, safety reviewer | approve or reject publication packages, source bundles, prompt/runtime artifacts, avatar/voice artifacts | review decision with rationale |
| Source Rights | /rights |
rights reviewer, compliance reviewer | inspect BYOM provenance, license posture, and allowed-use restrictions | rights decision or blocker |
| Content | /content |
moderation operator, safety reviewer | inspect generated educational content and tutor abuse incidents | moderation decision |
| Complaints | /complaints |
learner-support operator | triage content, tutor, grading, and export complaints | acknowledgement, investigation, escalation, or resolution |
| Users | /users and /users/[userId] |
learner-support lead | pause/resume progression, inspect study history, reset progress, manage investigations | user action with audit note |
| Monitoring | /monitoring |
operations lead, on-call operator | inspect tutoring latency, grounding failures, build failures, export failures, and cost anomalies | alert tuning or incident escalation |
| Incidents | /incidents |
incident commander | unify monitoring, complaints, moderation, rights, and learner investigations in one shell | runbook execution and workspace routing |
| Operations | /operations |
platform operator, safety reviewer | inspect transcripts, grounded build evidence, and model cost footprints | investigation evidence |
| Audit Trail | /audit |
support lead, compliance reviewer | validate operator actions, queue history, and escalation chronology | audit confirmation or escalation |
3. Role Model and Decision Rights#
| Role | Can decide directly | Must escalate when | Required companion surface |
|---|---|---|---|
| Curriculum reviewer | publication package, grounded course build, pedagogical prompt change | rights posture is unclear, grounded evidence is incomplete, or integrity concerns exist | /reviews, /rights, /operations |
| Rights reviewer | allowed-use posture, provenance sufficiency, licensing blocker status | source ownership is disputed, consent is missing, or legal interpretation is required | /rights, /audit |
| Moderation operator | generated-content takedown, tutor abuse acknowledgement, moderation escalation | learner safety risk, repeated abuse pattern, or policy ambiguity exists | /content, /incidents, /audit |
| Learner-support operator | complaint acknowledgement, investigation open, complaint resolution recommendation | grade dispute affects certification, learner trust risk is severe, or complaint links to live incident | /complaints, /users, /audit |
| Learner-support lead | progression pause/resume, progress reset, investigation resolution | action could affect entitlements, institutional reporting, or legal/compliance review | /users, /audit |
| Operations lead | alert threshold update, monitoring lane triage, operational routing | multi-lane degradation, cost anomaly with launch impact, or grounding failure is widespread | /monitoring, /incidents, /operations |
| Incident commander | runbook selection, lane ownership, downgrade/close decision | rights, safety, or compliance signoff is still unresolved | /incidents, /monitoring, /complaints, /content, /rights, /audit |
No operator may approve a Metis artifact or close a Metis incident without leaving an auditable rationale in the relevant workspace.
4. Shift-Start and Shift-End Routine#
4.1 Shift Start#
- Open
/and inspect quick actions, active counts, and recent operator activity. - Open
/monitoringand confirm whether any critical panel shows open or escalated incidents. - Open
/incidentsand confirm whether an incident commander is already assigned for open cross-lane issues. - Open
/reviews,/rights, and/complaintsand note any queue with breached SLA or unresolved escalations. - Open
/auditand confirm the previous shift left explicit rationale for any queue item still in progress.
4.2 Shift End#
- Ensure every changed item has an audit-backed decision, note, or escalation.
- Ensure open incidents have an owner, next action, and destination workspace.
- Ensure paused learner cases or rights blockers are visible in
/audit. - Record any exception-approved unresolved issue in the launch-readiness lane.
- Hand off unresolved high-severity work in the incident commander channel.
5. Standard Workflow Playbooks#
5.1 Review Queues#
Use /reviews when handling:
- publication packages
- source bundles
- grounded course builds
- tutor persona changes
- pedagogy prompt changes
- model-routing policy changes
- avatar teacher profiles
- voice packs
- likeness consent records
Required sequence:
- Confirm the queue type and current status.
- Inspect the detail payload and evidence summary.
- If the artifact references external material, verify rights posture in
/rights. - If the artifact affects tutor behavior or learner safety, inspect related
traces in
/operations. - Record an approve or reject decision with a rationale that would make sense to the next reviewer without extra context.
Approval must not proceed when grounding evidence, rights posture, or approval history is incomplete.
5.2 Source Rights#
Use /rights whenever imported or generated educational material depends on
third-party source rights.
Required sequence:
- Confirm provenance coverage and allowed-use posture.
- Inspect unknown-rights counts, licensing compatibility, and linked source artifacts.
- If the material is already referenced from a review queue, return to
/reviewsonly after a rights decision exists. - If the item blocks publication, escalate it into
/incidents. - Record whether the item is approved for release, review-only, or blocked.
5.3 Moderation#
Use /content for generated educational content risk, tutor abuse reports, and
academic-integrity incidents.
Required sequence:
- Confirm incident type, severity, and current operator owner.
- Inspect generated output, learner context, and any policy signals.
- If the issue affects an active learner trust event, jump into
/complaintsor/incidents. - Record the moderation outcome and note the policy basis.
- Verify the action appears in
/audit.
5.4 Learner Complaints#
Use /complaints when a learner reports content accuracy, tutor behavior,
grading, or export issues.
Required sequence:
- Filter to the complaint lane and open the case detail.
- Confirm the learner, course/session context, and current status.
- Decide whether the case can be acknowledged and resolved locally or needs an investigation or cross-workspace escalation.
- If the case affects progression or history, open
/users/[userId]. - If the case indicates broad platform degradation, open
/incidents. - Record the action and confirm it appears in
/audit.
5.5 Learner Operations#
Use /users and /users/[userId] for operator actions that directly affect a
learner account.
Required sequence:
- Confirm the learner identity and active investigations.
- Inspect study history and current course activity before pausing or resetting anything.
- Use pause/resume only when continuity or safety posture requires it.
- Use progress reset only with explicit operator rationale.
- Confirm every user action is visible in
/audit.
5.6 Monitoring and Incident Command#
Use /monitoring to identify platform risk and /incidents to run the shared
runbook across queue types.
Required monitoring sequence:
- Inspect critical panels for tutoring latency, grounding failures, build failures, export failures, and model cost anomalies.
- If a lane is locally recoverable, tune alerts or assign follow-up work.
- If the issue spans multiple lanes or creates customer trust risk, route it
into
/incidents.
Required incident-command sequence:
- Open the incident detail and confirm category, severity, owner, and source workspace.
- Work the runbook steps in order.
- Use the escalation routes to return to the resolving workspace instead of tracking work in chat only.
- Close the incident only after the resolving workspace action and audit entry both exist.
6. Training Program#
6.1 Initial Qualification#
An operator is not qualified for unsupervised Metis work until all of the following are complete:
- Read this runbook and the current Metis scorecard.
- Walk the live route map in the admin app with a trainer.
- Complete one shadow session for review/rights work.
- Complete one shadow session for complaints/user operations.
- Complete one shadow session for monitoring/incident command.
- Demonstrate audit-trail verification after every operator action.
6.2 Scenario Drills#
Every new operator must complete and pass the following drills:
| Drill | Required route flow | Pass condition |
|---|---|---|
| Publication review with rights blocker | /reviews -> /rights -> /audit |
operator blocks approval until rights posture is explicit |
| Tutor behavior complaint escalation | /complaints -> /incidents -> /users/[userId] -> /audit |
complaint becomes investigation-backed with clear owner |
| Monitoring spike to shared incident | /monitoring -> /incidents -> source workspace -> /audit |
incident runbook is followed and routed back into the correct lane |
| Moderation action with learner impact | /content -> /incidents -> /audit |
moderation decision includes policy rationale and incident linkage |
| Learner progression intervention | /users/[userId] -> /audit |
operator leaves durable rationale for pause/reset action |
6.3 Ongoing Calibration#
Weekly:
- review one closed incident and one closed complaint in
/audit - compare approval/rejection rationales across at least two operators
- confirm alert tuning changes still align with actual failure posture
Monthly:
- rerun at least one cross-lane escalation drill
- review rights and moderation decisions for policy drift
- confirm this runbook still matches the live admin route set
7. Evidence Requirements#
An operator action is incomplete unless all applicable evidence exists:
| Action type | Required evidence |
|---|---|
| Review approval or rejection | queue item, rationale, linked evidence summary, audit trail entry |
| Rights decision | provenance posture, allowed-use posture, blocker state, audit trail entry |
| Moderation decision | policy basis, affected content/session reference, audit trail entry |
| Complaint action | learner context, operator note, resulting complaint state, audit trail entry |
| Learner management action | user context, operator rationale, resulting status/history change, audit trail entry |
| Incident closure | runbook completion, destination-workspace action, audit trail entry |
If the evidence is missing, the operator must reopen or escalate the item rather than silently closing it.
8. Escalation Paths#
| Area | Primary owner | Escalate to |
|---|---|---|
| Review queues and course publication | Metis Product + Metis Admin Engineering | OSHUN Platform and relevant substrate owner |
| Source rights and provenance | Compliance + rights reviewer | Legal/compliance lead |
| Moderation and tutor behavior | Safety + moderation operator | Safety lead and incident commander |
| Learner complaints and user operations | Support operations | Support lead and product owner |
| Monitoring and incidents | Platform operations | Program operations and incident commander |
| Audit-trail gaps or permission ambiguity | Security + platform owner | OSHUN Security lead |
When a workflow crosses subsystem boundaries, the escalation target must follow the Metis ownership matrix rather than local convenience.
9. Currency Rule#
This runbook must be reviewed whenever any of the following changes:
- a new Metis admin route becomes launch-critical
- a queue type, incident type, or user action type changes
- scorecard status for operator documentation and training changes
- release-gate ownership or escalation policy changes
If the live app and this runbook disagree, the disagreement is a launch-readiness issue and must be resolved before the operator workflow is treated as current.