# Phase 91 continuous evaluation

The Phase 91 implementation is intentionally endpoint- and dataset-provider
neutral. Public datasets, model credentials, judge endpoints, code sandboxes,
PDF rendering, durable storage, and alert transports enter through explicit
interfaces; the core never invents benchmark data or silently executes generated
code in the application process.

## Standard suites

`createStandardBenchmarkRegistry` registers MMLU-Pro, HumanEval/BigCodeBench,
IFEval, Arena-Hard-Auto, GPQA Diamond, MATH-500, LiveBench, SWE-bench Verified,
WildBench, AAII, HLE/HLE-Rolling, multi-turn/MTRAG, and ToolBench/MCPAgentBench.
It records canonical dataset references and modality requirements. Code suites
require a resource-limited `CodeExecutor`; Arena-Hard requires a calibrated
`Judge`. `runLightEval` is the validated LightEval process boundary.
`BlindArena` captures anonymous pairwise votes and `bradleyTerryRatings` ranks
them.

## Domain suites

`DOMAIN_EVALUATIONS` is the ownership and metric manifest. Concrete metric
implementations cover RAGAS, the retrieval/classification/clustering/reranking
MTEB subset, typed Ghanaian NER, NPC personality and memory consistency, TTS
MOS/speaker/intelligibility, image FID/CLIP/aesthetic and alignment, 3D
geometry/texture/PBR, music quality/genre/alignment, per-harm moderation, and
per-language NLU/NLG/ChrF for Twi, Ewe, Ga, Hausa, and Dagbani.

## Operations and governance

- `UnifiedBenchmarkRunner` executes any registered suite against any
  `ModelEndpoint`.
- `EvaluationOrchestrator` supports production-daily, checkpoint, and manual
  runs; persists artifacts, selects a last-known-passing baseline, detects
  significant regressions, and alerts.
- `judgeCalibration`, randomized human sampling, normalized Likert summaries,
  and confidence intervals connect judge and human evaluation.
- Exact SHA-256 and embedding-cosine contamination checks cover exact and near
  matches.
- HTML and injected-renderer PDF reports are deterministic and escaped.
- `oshunBenchRotation`, approved-item validity windows, production redaction,
  stratified sampling, and independent submitter/reviewer curation implement
  rotating Oshun-Bench and production sets.
- `benchmarkConcurrentInference` captures TTFT, end-to-end p50/p95/p99,
  token/request throughput, concurrency, and errors. `costPerformance` and
  `paretoFront` support quantization/GPU result dimensions supplied as model
  metadata.
- The admin model workspace exposes score history, suite filtering, regressions,
  and the three-objective quality/cost/latency Pareto front. It has unit and
  Playwright/axe coverage.

## Integration contract

Production deployments must provide durable `EvaluationStore` and
`EvalCurationRepository` implementations, an alert sink, authenticated model
endpoints, a network-isolated code executor, a calibrated judge, and dataset
loaders licensed for the selected benchmark. Local in-memory implementations are
supplied for deterministic tests and development, not represented as production
persistence.
