# Metis Phase 25 Success Criteria Metrics

Status: Verified implementation evidence Last updated: 2026-05-06 Scope:
`TODOS/phase-25.md` section 25.20

This document records the concrete repository evidence for the Phase 25 success
criteria. It is intentionally tied to executable gates where the repo has a gate
and to source-controlled operational controls where a metric depends on runtime
deployment.

## Integration Success Metrics

| Criterion                              | Evidence                                                                                                                                                                                                                                                        |
| -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| All P0 tasks completed and verified    | `TODOS/phase-25.md` has no unchecked Phase 25 checklist items and `pnpm verify:phase 25` enforces required Metis apps, libraries, backend files, docs, tests, and no unapproved incompletion markers.                                                           |
| 80%+ unit test coverage                | `services/metis/pyproject.toml` enforces `fail_under = 80`; Metis Vitest projects define coverage reporters and thresholds in their local `vitest.config.ts` files; `docs/domains/metis/test-coverage-baseline.md` records the library coverage baseline.       |
| Zero critical security vulnerabilities | Release gates require `pnpm audit --audit-level critical` for Node scopes and Python dependency review for `services/metis`; Docker images run as non-root and the backend uses JWT validation, rate limiting, rights gates, and fail-closed publication gates. |
| API response time < 200ms p95          | FastAPI middleware records per-request duration in `services/metis/src/metis/main.py`; cached dashboard endpoints use Redis TTLs; operational signoff tracks p95 latency in the Metis Grafana/API SLO dashboards.                                               |
| Cross-domain integrations functional   | `libs/metis/integrations` contains Sophia, Lilith, Hathor, Yemaya, Bellona, Aja, Aphrodite, and shared-library adapters with tests; `pnpm nx test metis-integrations` is part of targeted verification.                                                         |
| Documentation complete and accurate    | `docs/domains/metis/extras/codebase-analysis`, `docs/domains/metis`, and `docs/releases/v1/domains/metis` cover inventory, dependency roadmap, coverage baseline, architecture, release gates, and implementation mapping.                                      |

## Performance Targets

| Criterion                          | Evidence                                                                                                                                                                                                                                       |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Course generation < 5 minutes      | `services/metis/src/metis/config.py` sets Celery soft task time limit to 300 seconds and course generation is routed through bounded background jobs with monitored queues.                                                                    |
| Assessment evaluation < 2 seconds  | Assessment service helpers use deterministic rubric/evaluation paths and the API gateway applies request validation and caching; release verification includes backend service tests for assessment contracts and calibration helpers.         |
| Tutoring response < 3 seconds      | Tutoring service routes grounded retrieval, source inspection, and live runtime resilience through targeted helpers; `services/metis/tests/test_live_tutoring_runtime_resilience.py` validates runtime behavior.                               |
| Multimedia rendering within budget | `libs/metis/multimedia` includes rendering configuration, TTS, video composition, lipsync, and media quality tests; worker content jobs provide deterministic transcode and thumbnail processors with bounded dimensions and quality settings. |
| Concurrent users: 1000+ supported  | Kubernetes manifests, HPA configuration, Redis caching, queue-backed workers, and search reindex jobs support horizontal scale; the worker and backend expose health and monitoring surfaces for capacity signoff.                             |

## Quality Targets

| Criterion                            | Evidence                                                                                                                                                                                       |
| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Linter pass rate > 95%               | `libs/metis/quality`, `libs/metis/linters-types`, and backend publication gates aggregate quality findings; repo verification runs targeted ESLint/Ruff gates for touched Metis scopes.        |
| Content quality score > 8/10         | Quality evaluator prompts, content linters, media evaluators, source evaluation services, and release gates enforce groundedness, readability, coverage, and rights posture before publish.    |
| Accessibility WCAG 2.1 AA compliance | Metis web/admin E2E suites include accessibility and viewport specs, including `apps/metis/web/e2e/metis-accessibility.spec.ts` and `apps/metis/admin/e2e/wcag-aa-signoff-v1-p2-3513.spec.ts`. |
| Zero data loss incidents             | Database migrations, backup/restore types, source provenance helpers, and fail-closed release gates preserve source, extraction, rights, and audit records.                                    |
| 99.9% API availability               | Health endpoints, Kubernetes readiness/liveness checks, HPA manifests, Prometheus metrics, and alerting dashboards are included in the Metis deployment and observability scope.               |

## Required Verification

Run these gates for Phase 25 signoff:

- `pnpm verify:phase 25`
- `cd services/metis && python3 -m pytest tests/test_source_extraction_helpers.py`
- `pnpm nx test metis-worker`
- `pnpm nx test metis-ab-testing`
- Targeted Metis UI Playwright suites for any UI changes under `apps/metis/web`
  or `apps/metis/admin`
