Eve library

Metis Phase 25 Success Criteria Metrics

Guide

Revision 1

On this page

Status: Verified implementation evidence Last updated: 2026-05-06 Scope: TODOS/phase-25.md section 25.20

This document records the concrete repository evidence for the Phase 25 success criteria. It is intentionally tied to executable gates where the repo has a gate and to source-controlled operational controls where a metric depends on runtime deployment.

Integration Success Metrics#

Criterion Evidence
All P0 tasks completed and verified TODOS/phase-25.md has no unchecked Phase 25 checklist items and pnpm verify:phase 25 enforces required Metis apps, libraries, backend files, docs, tests, and no unapproved incompletion markers.
80%+ unit test coverage services/metis/pyproject.toml enforces fail_under = 80; Metis Vitest projects define coverage reporters and thresholds in their local vitest.config.ts files; docs/domains/metis/test-coverage-baseline.md records the library coverage baseline.
Zero critical security vulnerabilities Release gates require pnpm audit --audit-level critical for Node scopes and Python dependency review for services/metis; Docker images run as non-root and the backend uses JWT validation, rate limiting, rights gates, and fail-closed publication gates.
API response time < 200ms p95 FastAPI middleware records per-request duration in services/metis/src/metis/main.py; cached dashboard endpoints use Redis TTLs; operational signoff tracks p95 latency in the Metis Grafana/API SLO dashboards.
Cross-domain integrations functional libs/metis/integrations contains Sophia, Lilith, Hathor, Yemaya, Bellona, Aja, Aphrodite, and shared-library adapters with tests; pnpm nx test metis-integrations is part of targeted verification.
Documentation complete and accurate docs/domains/metis/extras/codebase-analysis, docs/domains/metis, and docs/releases/v1/domains/metis cover inventory, dependency roadmap, coverage baseline, architecture, release gates, and implementation mapping.

Performance Targets#

Criterion Evidence
Course generation < 5 minutes services/metis/src/metis/config.py sets Celery soft task time limit to 300 seconds and course generation is routed through bounded background jobs with monitored queues.
Assessment evaluation < 2 seconds Assessment service helpers use deterministic rubric/evaluation paths and the API gateway applies request validation and caching; release verification includes backend service tests for assessment contracts and calibration helpers.
Tutoring response < 3 seconds Tutoring service routes grounded retrieval, source inspection, and live runtime resilience through targeted helpers; services/metis/tests/test_live_tutoring_runtime_resilience.py validates runtime behavior.
Multimedia rendering within budget libs/metis/multimedia includes rendering configuration, TTS, video composition, lipsync, and media quality tests; worker content jobs provide deterministic transcode and thumbnail processors with bounded dimensions and quality settings.
Concurrent users: 1000+ supported Kubernetes manifests, HPA configuration, Redis caching, queue-backed workers, and search reindex jobs support horizontal scale; the worker and backend expose health and monitoring surfaces for capacity signoff.

Quality Targets#

Criterion Evidence
Linter pass rate > 95% libs/metis/quality, libs/metis/linters-types, and backend publication gates aggregate quality findings; repo verification runs targeted ESLint/Ruff gates for touched Metis scopes.
Content quality score > 8/10 Quality evaluator prompts, content linters, media evaluators, source evaluation services, and release gates enforce groundedness, readability, coverage, and rights posture before publish.
Accessibility WCAG 2.1 AA compliance Metis web/admin E2E suites include accessibility and viewport specs, including apps/metis/web/e2e/metis-accessibility.spec.ts and apps/metis/admin/e2e/wcag-aa-signoff-v1-p2-3513.spec.ts.
Zero data loss incidents Database migrations, backup/restore types, source provenance helpers, and fail-closed release gates preserve source, extraction, rights, and audit records.
99.9% API availability Health endpoints, Kubernetes readiness/liveness checks, HPA manifests, Prometheus metrics, and alerting dashboards are included in the Metis deployment and observability scope.

Required Verification#

Run these gates for Phase 25 signoff:

  • pnpm verify:phase 25
  • cd services/metis && python3 -m pytest tests/test_source_extraction_helpers.py
  • pnpm nx test metis-worker
  • pnpm nx test metis-ab-testing
  • Targeted Metis UI Playwright suites for any UI changes under apps/metis/web or apps/metis/admin