The Phase 92 implementation is provider-neutral and fail-closed. Durable stores, MLflow/W&B clients, model routers, queues, evaluators, flag providers, alerting, Prometheus exporters, artifact storage, and scanners enter through interfaces; the core does not claim an in-memory adapter is production persistence.
Capability map#
mlops-experiments.ts: unified run SDK, parameters, metrics, artifacts, tags/search, lineage DAG, side-by-side/Pareto model comparison, and training job dashboard summaries.mlops-lifecycle.ts: sliding-window output/calibration/latency drift, retraining and promotion gates, stable A/B assignment, shadow variants, feature-flag routing, staged canaries, transactional rollback with health checks, deprecation retention, endpoint SLA evaluation, and model incident severity/playbooks aligned to NIST AI 600-1 and MITRE ATLAS.mlops-governance.ts: Hugging Face model cards, RBAC approval stages, bias disparity audits, hash-chained operation logs, license compatibility, reproducibility manifests, tiered risk policies, inventory summaries, chargeback/budget alerts, GPU FinOps recommendations, five-part adversarial gates, and security governance validation.infra/nous/phase-92: pinned, persistent MLflow + PostgreSQL + MinIO deployment with loopback-only exposure and health checks.- The existing
/modelsadmin workspace is the operational UI boundary. Its model comparison, evaluation and governance panels consume registry evidence; Phase 92 stores must project experiment/job, lifecycle/SLA, approval/audit, and cost summaries into that authenticated workspace rather than creating a second control plane.
Production invariants#
All model-changing operations require authenticated actor identity and append to both the immutable model audit and deployment audit sinks. Customer-facing and safety-critical models cannot reach production without human approval, current adversarial evidence, a rollback plan, and continuous/daily monitoring. Canary assessment and rollback health failures fail closed. Artifact content is hashed; reproduction manifests pin data, code, config, seeds, and environment.
Prometheus adapters should export endpoint latency p95/p99, availability,
throughput, errors, drift statistics, rollout stage, GPU utilization, and cost.
Monthly reporters aggregate evaluateSla, allocateMlCosts, and governance
inventory results; alert sinks own delivery and escalation policy.