Disciplines · Runbooks

Concordia Pilot Evidence and Exit Gate

Each domain owner must freeze and attest five distinct artifacts before the deployment timestamp:

5sections3 minread
Markdown source
On this page

title: Concordia Pilot Evidence and Exit Gate owner: Concordia program lead version: 1.0.0 last_reviewed: 2026-07-15 next_review: 2026-10-13 phase_task: 179.10.8#

This runbook defines the evidence required to close §179.10.8. Repository fixtures, synthetic benchmarks, staging sessions, rollout flags, and unsigned dashboard screenshots do not prove a deployment or pilot value. The criterion closes only when evaluatePilotProgram passes one signed production-pilot report for each of Maat, Themis, Kuanyin, Iris, Aje, and Yemaya.

Before deployment#

Each domain owner must freeze and attest five distinct artifacts before the deployment timestamp:

  1. a passing dated research refresh;
  2. the current Concordia governance approval;
  3. that domain's populated policy-bundle card;
  4. a passing release-gate report for the exact release digest; and
  5. the current Concordia incident-response runbook.

Record the release version and SHA-256 digest, pseudonymous tenant reference, production-pilot environment, deployment record, and two distinct operator signoffs. A card or gate approved after launch is remediation, not evidence that the pilot launched compliantly; restart the observation window after the prerequisite is satisfied.

Pre-register metrics#

Before launch, select at least one primary value metric with a positive minimum effect and sample floor. Define supporting metrics and non-regression guardrails at the same time. Do not lower thresholds or swap an unfavorable primary after observing results. Baselines must be comparable historical/manual cohorts or a pre-registered contemporaneous control, with definitions, exclusions, and source artifact retained.

Recommended domain plans are below; owners may add metrics but should not omit the safety, privacy, authority, and review-routing guardrails applicable to the domain.

Domain Primary value measures Supporting measures Required guardrails
Maat procurement savings or realized value cycle time, DPO change, supplier satisfaction dispute rate, supplier fairness, review-routing misses, privacy incidents
Themis mediated proposal pass-rate delta or consensus improvement quorum, participation, amendment durability minority fairness, authority bypass, governance-review misses, privacy incidents
Kuanyin moderation-recurrence reduction or obligation completion participant fairness, community health, repair completion coercion/safety events, dignity/voice, privacy incidents
Iris agreement/plan completion or cycle-time reduction willingness to reuse, relationship preservation unsuitable-case routing, consent failures, private-context leakage
Aje settlement execution completion or execution-time reduction failed milestone reduction, reconciliation effort unauthorized acceptance, execution mismatch, finance-review misses, privacy/security incidents
Yemaya first-pass delivery acceptance or revision-round reduction schedule adherence, obligation completion, participant fairness labor/safety routing, rights/authority violations, privacy incidents

The typed aggregators in outcome-metrics, subjective-value, domain-metrics, cost-metrics, and safety-metrics should produce the source exports. A metric observation records only the comparable baseline, observed value, sample size, timestamp, unit, and immutable source reference; the source artifact retains cohort detail and uncertainty analysis.

Observation and incident coverage#

Start observation no earlier than the signed deployment. Keep the window open until every registered metric reaches its sample floor and domain-appropriate durability horizon. Measurements must fall inside the closed window.

Export the complete incident inventory after the window closes, with coverage from window start through window end and a safety-reviewer attestation. Include closed, mitigated, duplicate, and false-positive records so an auditor can reconcile counts. Follow concordia-incident-response.md. Any P0/P1 not in closed status blocks the pilot; mitigated is unresolved for this gate.

Evaluation and signoff#

  1. Parse each report with PilotEvaluationInputSchema.
  2. Run evaluatePilot; archive the structured result with the input digest.
  3. Remedy insufficient_evidence by obtaining missing evidence, never by imputing observations. A rolled-back deployment is not_deployed.
  4. Extend or redesign a no_measurable_value pilot and pre-register a new window. Do not merge windows with changed thresholds.
  5. Close every blocking incident under dual control and begin a fresh post-remediation observation window when the control failure could affect outcome validity.
  6. Run evaluatePilotProgram over exactly one passing result per named domain. Duplicate pilots do not substitute for a missing domain.

Only the final six-domain program result, linked evidence manifest, and program lead signoff authorize checking §179.10.8. Evidence artifacts containing party data remain in the governed evidence store; the repository receives only redacted identifiers, digests, schemas, and the signed evaluation result.