Eve library

OSHUN Metis V1 Scorecard

Guide

Revision 1

On this page

Status: Active

Date: April 9, 2026

Owners: OSHUN Product, OSHUN Engineering, OSHUN Platform, OSHUN Safety

Related:

  • docs/releases/v1/reports/metis-domain-gap-analysis-and-full-integration-report-2026-03-23.md
  • docs/releases/v1/reports/metis-subsystem-ownership-matrix.md
  • docs/releases/v1/specs/metis-mobile-strategy.md
  • docs/releases/v1/runbooks/metis-operator-workflows-runbook.md
  • docs/releases/v1/specs/metis-generation-and-tutoring-slos.md
  • docs/releases/v1/specs/metis-media-runtime-tiers.md
  • docs/releases/v1/specs/metis-persistence-migration-backfill-plan.md
  • docs/releases/v1/specs/metis-learning-source-bundle-contract.md
  • docs/releases/v1/specs/metis-grounding-pack-contract.md
  • docs/releases/v1/specs/metis-course-build-contract.md
  • docs/releases/v1/specs/metis-lesson-asset-bundle-contract.md
  • docs/releases/v1/specs/metis-publication-package-contract.md
  • docs/releases/v1/specs/metis-avatar-teacher-profile-contract.md
  • docs/releases/v1/specs/metis-voice-likeness-consent-record-contract.md
  • docs/domains/metis/METIS_KNOWLEDGE_GRAPH_AND_RETRIEVAL_ALGORITHM_STATUS.md
  • services/metis/openapi/metis.openapi.json
  • libs/openapi/src/generated/metis.ts
  • docs/releases/v1/specs/program-charter.md
  • docs/releases/v1/specs/studio-launch-readiness-governance-dependency-and-data-contract-map.md
  • docs/releases/v1/specs/studio-launch-readiness-governance-user-journeys-and-edge-cases.md
  • docs/releases/v1/specs/todos.md

1. Purpose#

This document is the canonical Metis readiness scorecard for OSHUN V1.

It consolidates the learner, admin, platform, governance, and release-gate dimensions that must be green before Metis can be treated as V1-complete.

This is not a feature inventory and not a marketing status page. It is a launch control artifact that should stay aligned with verified runtime truth.

2. Status Scale#

  • Green: launch-grade, verified in code/runtime, and covered by applicable automation and operational evidence
  • Amber: meaningful foundation exists, but launch criteria are still unmet
  • Red: launch-blocking capability is missing, mock-driven, simulated, or insufficiently governed

3. Current Snapshot#

Scorecard lane Current status Why
Learner completeness Red The shell-level Metis entry points now exist, but apps/metis/web remains materially mock-backed across auth, browse, progress, tutoring, assessment, and authoring flows.
Admin completeness Red The Metis admin surface is still largely mock-driven, and the required review, moderation, audit, support, and urgent-queue paths are not yet live and verified.
Platform completeness Amber Real libraries, gateway, worker, and service scaffolding exist, but the platform still lacks complete contract publication, service-test depth, and end-to-end API/runtime proof.
Governance completeness Red Ownership is now explicit, but launch-grade rights/integrity workflows, approval controls, and remaining governance gates are still incomplete even with operator runbooks now published.
Release-gate completeness Amber The Studio readiness model already requires a dedicated Metis lane, but Metis-specific quality, rights, safety, cost, pedagogy, and provenance gates are not yet all operationally green.

4. Learner Completeness#

Learner completeness is green only when all of the following are true:

Dimension Current status Green criteria Primary backlog drivers
Live learner auth and session truth Red learner identity, session hydration, and continuity are API-backed rather than mock-backed V1-MET-015
Live course browse, detail, and progress Red browse, detail, resume, and progress state are live and durable V1-MET-016, V1-MET-017, V1-MET-024, V1-MET-027
Live tutoring and assessment flows Red tutoring and assessments are session-backed, grounded where required, and recoverable after interruption V1-MET-018, V1-MET-019, V1-MET-069 through V1-MET-085
Shell and mobile continuity Amber shell entry, mobile strategy, resume paths, reminders, and narrow-width behavior are coherent and verified V1-MET-021 through V1-MET-028
Trust, accessibility, and degraded-state UX Red grounded evidence, accessibility, offline/degraded, and export/share behavior exist and are learner-legible V1-MET-025, V1-MET-026, V1-MET-029, V1-MET-030
Learner automation and performance Red Playwright coverage and route-level performance/PWA checks pass for critical learner journeys V1-MET-031, V1-MET-032, V1-MET-107

5. Admin Completeness#

Admin completeness is green only when all of the following are true:

Dimension Current status Green criteria Primary backlog drivers
Live admin dashboards and data truth Red dashboards, users, courses, and flagged content are live, not mock-backed V1-MET-033
Review and approval queues Red source bundles, grounded outputs, tutor personas, prompts, avatars, voice, and rights all have operator-grade review queues V1-MET-034 through V1-MET-039
Monitoring, incidents, and support tooling Red latency/grounding/failure panels, learner complaint tooling, and incident workflows are live and actionable V1-MET-040, V1-MET-043, V1-MET-044
Audit, permissions, and user actions Red Metis admin actions have durable audit trails, user-management controls, and contract/permission verification V1-MET-041, V1-MET-042, V1-MET-047
Admin automation and mobile urgent queues Red browser automation exists for review/moderation/monitoring, and admin mobile escalation coverage exists for urgent Metis queues V1-MET-045, V1-MET-046
Operator documentation and training Green runbooks, training, and operator-facing documentation are published and current V1-MET-048

6. Platform Completeness#

Platform completeness is green only when all of the following are true:

Dimension Current status Green criteria Primary backlog drivers
Canonical contracts Amber learner, admin, media, and workflow contracts are published and versioned in canonical Metis OpenAPI/contract surfaces V1-MET-049 through V1-MET-053, V1-MET-101
API-backed runtime truth Amber ingest, retrieval, generation, tutoring, assessment, and publication paths execute against live backend contracts V1-MET-054 through V1-MET-067, V1-MET-102, V1-MET-103
Service and persistence reliability Amber durable state, migrations, job reliability, and dependency behavior are verified for service-critical paths V1-MET-104, V1-MET-105, V1-MET-109; migration baseline in OSHUN_V1_METIS_PERSISTENCE_MIGRATION_BACKFILL_PLAN.md
Interoperability and export readiness Amber LMS/import/export promises are only exposed where implemented, tested, and operationally supported V1-MET-064, V1-MET-099, V1-MET-157 through V1-MET-195
Platform observability and rollback posture Amber build-job, tutor-session, publication, and interoperability telemetry are wired to rollback-aware operational signals V1-MET-109, V1-MET-196

7. Governance Completeness#

Governance completeness is green only when all of the following are true:

Dimension Current status Green criteria Primary backlog drivers
Ownership and subsystem boundaries Amber Metis-to-substrate ownership is explicit and current in canonical docs V1-MET-012
Source, rights, and integrity review Red grounded source approval, licensing review, originality checks, moderation, and appeals are live and auditable V1-MET-039, V1-MET-074, V1-MET-081, V1-MET-114, V1-MET-145
Tutor and pedagogy governance Red persona, pedagogy mode, escalation, and sensitive-teaching controls are reviewable and approval-backed V1-MET-035, V1-MET-073, V1-MET-076, V1-MET-082
Avatar, voice, and consent governance Red avatar teacher profiles, likeness consent, safety, fallback rules, and approvals are operational V1-MET-036, V1-MET-090, V1-MET-095, V1-MET-098
Documentation and operational currency Amber scorecards, ownership, runbooks, operator docs, and truth-in-documentation stay aligned with verified runtime state V1-MET-012, V1-MET-013, V1-MET-048, V1-MET-197

8. Release-Gate Completeness#

Release-gate completeness is green only when all of the following are true:

Dimension Current status Green criteria Primary backlog drivers
Dedicated Metis lane in readiness governance Amber Metis learner/tutoring/admin scope always generates a dedicated scorecard lane and blocker queue V1-GOV-027, V1-MET-013
Quality, rights, safety, cost, and pedagogy gates Amber launch gates evaluate educational quality, rights, safety, cost, and pedagogy rather than generic uptime only V1-MET-040, V1-MET-082, V1-MET-096, V1-MET-109, V1-MET-110, V1-MET-116
Provenance and confidence evidence Amber release decisions carry claim-level provenance and confidence evidence for grounded educational outputs V1-MET-063, V1-MET-068, V1-MET-116
Benchmark and adversarial evidence Red benchmark suites, red-team cases, prompt-injection resistance, and governed holdout evaluation are attached to gate review V1-MET-068, V1-MET-079, V1-MET-081, V1-MET-082, V1-MET-131, V1-MET-196
Exception handling and rollback triggers Amber open blockers, exceptions, and rollback triggers are explicit, time-bounded, and audit-backed for Metis scope V1-MET-044, V1-MET-109, V1-MET-196

9. Green-Only Exit Rule#

Metis should be considered V1-complete only when:

  • learner completeness is green
  • admin completeness is green
  • platform completeness is green
  • governance completeness is green
  • release-gate completeness is green or has only time-bounded, approval-backed exceptions consistent with the OSHUN V1 charter

If any lane is red, Metis is launch-blocking.

If any lane is amber, Metis is not yet complete even if portions of the stack look production-adjacent.

10. Full Integration Hold Rule#

Metis must not be marked fully integrated into OSHUN V1 while any launch- critical learner or admin path remains mock-driven, stub-dependent, simulated, or verified only against non-canonical local data.

This rule is fail-closed.

If any of the following are true, the canonical answer is still "not fully integrated":

  • apps/metis/web still depends on material mock-backed learner flows
  • apps/metis/admin still depends on material mock-backed operator flows
  • scorecard lanes remain Red
  • scorecard lanes remain Amber without time-bounded, approval-backed exceptions
  • applicable automated verification for learner, admin, API, release-gate, or operational paths is not passing

Current launch hold evidence:

  • learner completeness remains Red
  • admin completeness remains Red
  • governance completeness remains Red
  • apps/metis/web/src/lib/mock-data.ts and apps/metis/admin/src/lib/mock-data.ts still exist as material truth sources for current product surfaces
  • multiple Metis browser suites still rely on deterministic fixture layers rather than fully live end-to-end runtime truth

Metis may only be marked fully integrated after the scorecard itself shows all required lanes green and the linked verification evidence is current.