---
artifact_kind: model-card
artifact_name: pairwise-preference-ranker
version: 1.0.0
owner: Nous · preference-inference team
last_reviewed: 2026-04-25
next_review: 2026-07-25
concordia_components:
  [
    '@nous/preference-inference',
    '@iris/concordia-assistant',
    '@nous/agreement-search',
  ]
related_phase_179_tasks:
  [
    179.3.2.1,
    179.3.2.2,
    179.3.2.3,
    179.3.2.4,
    179.3.2.5,
    179.3.2.6,
    179.5.1.1,
    179.5.1.2,
  ]
related_cards:
  system: ../systems/concordia-workbench.md
  evaluators:
    - ../evaluators/preference-stability.md
    - ../evaluators/baseline-benchmark-gate.md
    - ../evaluators/fairness-suite.md
---

# Model Card — pairwise-preference-ranker v1.0.0

## 1. Identity

- **Model name:** pairwise-preference-ranker
- **Provider / developer:** Concordia · preference-inference (this repo)
- **Base model and version:** Tenant-configurable. Default routing tier is
  Anthropic `claude-opus-4-7` for high-stakes classes, `claude-sonnet-4-6` for
  moderate-stakes, and a local zero-retention reasoning model for
  `low_stakes_personal`, `community_restorative`, and `creative_collaboration`
  classes when zero-retention is required.
- **Purpose:** Given one party's private intake snippets and two candidate
  agreements (A and B), emits a pairwise preference: choice ∈ {A, B, tie,
  abstain}, posterior probability, citations to the snippets used, and an
  optional redline flag. The output is consumed by the utility-fitters
  ([../models/utility-fitters.md](../models/utility-fitters.md)) and feeds every
  search kernel under `@nous/agreement-search`.
- **Intended use cases (Concordia use-case classes):** all 14
  `ConcordiaUseCaseClass` values are eligible for some preference inference, but
  the routing matrix in `USE_CASE_CLASS_PROFILES` (see
  `libs/contracts/concordia/src/use-case/use-case-class.ts`) restricts which
  base model is allowed per class and which classes block autonomous acceptance.
- **Out-of-scope uses:** model output is never the basis for any decision the
  hard-boundary registry refuses
  (`libs/contracts/concordia/src/boundaries/hard-boundary.ts`): binding legal
  claim resolution, custody, domestic-violence or coercive control, employment
  adverse action, medical settlement, financial suitability, immigration,
  criminal matters, or rights waiver without qualified human review.

## 2. Development context

- **Training data summary:** the underlying LLMs are vendor-supplied; their
  training data is whatever the chosen provider documented. Concordia adds no
  fine-tuning weights to the base models in v1.0.0. The local zero-retention
  reasoning model is the vendor base with no Oshun fine-tune.
- **Fine-tuning:** none. Concordia operates the model purely via prompting.
  Future fine-tuning enters the §179.5.5.5 training-data-governance pipeline
  (see `libs/contracts/concordia/src/training-data-governance/`).
- **Prompt family:** `buildPairwiseComparisonPrompt` and
  `parsePairwiseComparisonResponse` in
  `libs/nous/preference-inference/src/pairwise-prompt.ts`. Single-source
  template; perturbation variants in `prompt-perturbations.ts` are used only for
  stability probing, never as the live ranker.
- **Concordia-specific customization:** strict JSON schema
  (`PairwiseComparisonResponseSchema`) for the response, mandatory citations to
  private-intake snippet ids, citation-quote verification (the post-processor
  confirms a quoted span exists in the cited snippet), explicit abstention
  states (`low_confidence`, `unstable`, `refused`, `redline_blocked`,
  `non_tradeable`, `insufficient_evidence`), and rationale length cap (320
  chars) to discourage fabrication.

## 3. Inputs and outputs

- **Input schema:** `BuildPairwisePromptInput` from
  `libs/nous/preference-inference/src/pairwise-prompt.ts`. Owning party's intake
  snippets only — the function signature physically excludes opposing-party
  private fields. Two `AgreementSummaryForComparison` records (10 clause lines
  each, max).
- **Output schema:** `PairwiseComparisonResponseSchema` (Zod-validated):
  `{ schemaVersion: 1, choice, probability ∈ [0, 1], rationale ≤ 320 chars, citations: ≥ 1, abstentionReason?, redlineFlag }`.
  Free-form text is treated as a refusal.
- **Deterministic settings used in production:** temperature 0.0 by default
  (overridable per use-case class to a maximum of 0.3 for `low_stakes_personal`
  only); top-p 1.0; max tokens 512; per-comparison deadline 6000 ms
  (`pairwise-plan.ts` default). Seed is always pinned and recorded in the
  reproducibility manifest (§179.5.3.3).

## 4. Evaluation

| Metric                                            | Value                         | Evaluator card                                                      | Date       |
| ------------------------------------------------- | ----------------------------- | ------------------------------------------------------------------- | ---------- |
| Preference-stability — paraphrase agreement       | ≥ 0.80 (release-gate floor)   | [preference-stability](../evaluators/preference-stability.md)       | 2026-04-25 |
| Preference-stability — template agreement         | ≥ 0.80 (release-gate floor)   | [preference-stability](../evaluators/preference-stability.md)       | 2026-04-25 |
| Preference-stability — model-version agreement    | ≥ 0.85 (release-gate floor)   | [preference-stability](../evaluators/preference-stability.md)       | 2026-04-25 |
| Order-effect agreement (A↔B swap)                 | ≥ 0.85 (release-gate floor)   | [preference-stability](../evaluators/preference-stability.md)       | 2026-04-25 |
| Adversarial-framing agreement                     | ≥ 0.70 (release-gate floor)   | [preference-stability](../evaluators/preference-stability.md)       | 2026-04-25 |
| 95% credible-interval width on fitted utility     | ≤ 0.20 (release-gate floor)   | [preference-stability](../evaluators/preference-stability.md)       | 2026-04-25 |
| Citation-grounding pass rate (quote-in-snippet)   | 1.00 (parser-enforced)        | n/a — schema gate                                                   | 2026-04-25 |
| Citation-coverage pass rate (every rationale)     | 1.00 (parser-enforced)        | n/a — schema gate                                                   | 2026-04-25 |
| Baseline-benchmark gate — wins vs llm-mediator    | ≥ 6 of 8 suites (gate config) | [baseline-benchmark-gate](../evaluators/baseline-benchmark-gate.md) | 2026-04-25 |
| Baseline-benchmark gate — wins vs nash-genetic    | ≥ 6 of 8 suites (gate config) | [baseline-benchmark-gate](../evaluators/baseline-benchmark-gate.md) | 2026-04-25 |
| Baseline-benchmark gate — wins vs static-template | ≥ 6 of 8 suites (gate config) | [baseline-benchmark-gate](../evaluators/baseline-benchmark-gate.md) | 2026-04-25 |
| Baseline-benchmark gate — wins vs seed-offer-only | ≥ 6 of 8 suites (gate config) | [baseline-benchmark-gate](../evaluators/baseline-benchmark-gate.md) | 2026-04-25 |
| Per-comparison deadline                           | 6000 ms (`pairwise-plan.ts`)  | n/a — runtime contract                                              | 2026-04-25 |
| Cost per scoring call                             | tenant-billed; bounded by     | n/a — orchestrator scoring-cost trace                               | 2026-04-25 |
|                                                   | observability budget guard    |                                                                     |            |

The release-gate floors are the defaults declared in
`libs/nous/preference-inference/src/pairwise-plan.ts`
(`StabilityThresholdsSchema`) and the `minSuiteWinsPerBaseline = 6` default in
`libs/contracts/concordia/src/baseline-benchmark-gate/baseline-benchmark-gate.ts:113`.
Tenants may _raise_ (never lower) the floors via configuration.

## 5. Privacy posture

- **Data visibility at the model:** the owning party's intake snippets, the two
  candidate-agreement summaries (≤ 10 clause lines each), the prompt template
  version, and the use-case class. No opposing-party private fields, no
  reviewer-only annotations, no consent ledger entries, no audit trail.
- **Per-party isolation:** §179.5.1.1 physically separate prompt contexts.
  Enforced at the function-signature level — `buildPairwiseComparisonPrompt`
  accepts only the owning party's snippets. Cross-party leakage is impossible
  through this surface; integration tests in
  `libs/iris/concordia-assistant/src/party-isolated-context.test.ts` and
  `libs/iris/concordia-assistant/src/privacy-red-team.test.ts` assert that.
- **Training-data reuse eligibility:** off by default. Reuse only after the
  trace passes `recordDeidentificationReview` (§179.5.5.5,
  `libs/contracts/concordia/src/training-data-governance/`) AND the granting
  party has an active `training_data_reuse` consent purpose.
- **Zero-retention mode:** required for `legal_claim`, `medical_sensitive`,
  `regulated_financial`, `family_sensitive`, `employment_sensitive`, and any
  case where the party requested it via the consent ledger. Routing layer
  enforces the local zero-retention model for these classes.
- **Secure-enclave / confidential-compute posture:** §179.5.1.4 enclave channel
  implemented in `libs/nous/concordia-sealed-memory/`. Used today for utility
  aggregation only; preference scoring inside enclave is a roadmap item for
  v1.1.

## 6. Safety posture

- **Hard-boundary coverage:** the candidate-filter gate
  (`libs/nous/agreement-search/src/candidate-filter.ts`) refuses any candidate
  that triggers a hard boundary BEFORE pairwise scoring. The ranker therefore
  never scores a hard-boundary-violating candidate. Tested by
  `candidate-filter.test.ts` and the boundary unit tests in
  `libs/contracts/concordia/src/boundaries/hard-boundary.test.ts`.
- **Adversarial robustness:** prompt-injection, fishing-for-private-info, and
  redline-bypass tests in
  `libs/iris/concordia-assistant/src/privacy-red-team.test.ts` and
  `libs/nous/agreement-search/src/spec-gaming-audit.test.ts`. Output schema
  rejects free-form text, which closes the most common injection path.
- **Coercive-choice refusal:** `coercive-choice-audit.ts` in
  `@nous/agreement-search` audits emitted candidate sets and explanations for
  countdown pressure, hidden anchors, asymmetric disclosure, and forced
  defaults. The ranker itself does not produce explanations to parties; it
  produces structured comparisons consumed by other components.
- **Prompt-injection resistance:** schema-gated output + citation verification
  - rationale length cap. The parse failure modes (`unparsable`,
    `missing_citation`, `quote_not_in_snippet`) are observed in
    `pairwise-prompt.test.ts`.

## 7. Fairness and bias posture

- **Protected-class disparate treatment:** the fairness-profile module
  (`libs/nous/preference-inference/src/fairness-profiles.ts`) blocks any
  fairness profile whose configuration would induce protected-class disparate
  treatment in scoring. Tested in `fairness-profiles.test.ts`.
- **Power-asymmetry handling:** BATNA plausibility (`batna-plausibility.ts`)
  refuses to take self-reported BATNA as ground truth — this is the structural
  defense against a high-leverage party gaming the optimizer. Inequity aversion
  appears as a separable signal in `multi-attribute-utility.ts` so it never
  silently dominates utility.
- **Language / locale coverage:** language-detector + language-drift modules in
  `@iris/concordia-assistant` (`language-drift.ts`) detect internal-
  reasoning-language drift relative to the party's intake language. Multiple
  language-summary and language-preference modules preserve party language
  end-to-end. Locales below the language-acceptance gate route to human mediator
  review rather than scoring.

## 8. Known limitations

- The vendor base models exhibit paraphrase-sensitivity that cannot be removed
  by prompting alone. The release-gate floor at 0.80 paraphrase agreement is a
  containment, not a fix.
- Order-effect bias is small but non-zero. The `order_swap` perturbation in
  `stability-probes.ts` is run on every released model version.
- Adversarial framing (loaded analogies, framing as "everyone agrees") can shift
  the ranker's choice. The 0.70 floor is a known weak point and is mitigated by
  the adversarial-framing perturbation in stability probing, which causes the
  abstention path to engage when framing is detected.
- Long-context degradation: the prompt is intentionally short (≤ 10 clause lines
  per agreement, ≤ 512-char private snippets). Cases with many clauses are
  decomposed via `proposal-decomposition.ts` rather than fed to the ranker as a
  single mega-prompt.
- BATNA is never accepted as self-reported; this is by design. A party-supplied
  BATNA is treated as a hypothesis until it is corroborated by evidence in
  `batna-plausibility.ts`.

## 9. Human-in-the-loop expectations

- **Required reviewers when this model is used:** determined by use-case class
  via `USE_CASE_CLASS_PROFILES`. `legal_claim`, `medical_sensitive`,
  `regulated_financial`, and `family_sensitive` always require a qualified human
  reviewer queue (`legal_counsel`, `compliance_officer`, `safety_reviewer`, or
  domain expert).
- **Escalation triggers specific to this model:** posterior `probability` below
  0.55 produces a low-confidence signal; abstention with reason `unstable`,
  `redline_blocked`, or `refused` triggers the §179.1.3 escalation policy via
  `deriveEscalationDecision`. Citation-quote verification failure escalates as a
  model-integrity incident.
- **Mediator-assist vs autonomous use:** the model is mediator-assist for every
  class except `low_stakes_personal`, `community_restorative`,
  `creative_collaboration`, `commerce_negotiation` (under explicit autonomous-
  acceptance authority bounded by `settlementLimit`), and
  `agent_to_agent_contract` (within the policy-constrained autonomous-
  acceptance contract of §179.7.6.3).

## 10. Incident history

- 2026-04-25 (v1.0.0 release review): no incidents on file. Future entries
  reference incident ids from `libs/contracts/concordia/src/incidents/`.

## 11. Regulatory posture

- **EU AI Act Article 11 / Annex IV alignment:** technical documentation
  obligations satisfied by this card together with the optimizer cards under
  `../optimizers/` and the system card at `../systems/concordia-workbench.md`.
  See `docs/compliance/eu-ai-act-mapping.md` §Art. 11 for the row-by-row
  mapping.
- **JAMS AI Rules expert-inspection readiness:** prompt template version,
  base-model id, deterministic settings, response schema, and citations are all
  preserved on every comparison via the reproducibility manifest
  (`libs/contracts/concordia/src/reproducibility/`). Trade-secret flags
  permitted on private-intake snippet content; the citation hash is exposed
  unredacted.
- **NIST AI RMF harm coverage:** H1 (privacy), H2 (security), H3
  (interpretability), H4 (fairness), H5 (safety), H6 (accountability) per
  `docs/compliance/nist-ai-rmf-mapping.md`. H7 (environmental) addressed at the
  system-card level via cost / latency observability.

## 12. References

- `libs/nous/preference-inference/src/pairwise-prompt.ts` — prompt builder &
  response schema
- `libs/nous/preference-inference/src/pairwise-plan.ts` — stability thresholds &
  per-comparison deadline defaults
- `libs/nous/preference-inference/src/stability-probes.ts` — perturbation runner
  used for §179.3.2.4 stability tests
- `libs/iris/concordia-assistant/src/privacy-red-team.test.ts` — privacy
  red-team coverage
- `libs/contracts/concordia/src/baseline-benchmark-gate/baseline-benchmark-gate.ts`
  — release-gate winning-suite arithmetic
- `docs/research/bibliography.md` §pairwise-preference — Bradley, Terry,
  Bradley-Terry-Luce, Davidson, ProMediate, LLMediator