Optimizer Card — psro-opponent-model v1.0.0

artifact_kind: optimizer-card artifact_name: psro-opponent-model version: 1.0.0 owner: Nous · agreement-search team last_reviewed: 2026-04-25 next_review: 2026-07-25 optimizer_kind: psro related_phase_179_tasks: [179.4.2.6, 179.7.6, 179.7.1]#

1. Identity#

• Optimizer name: psro-opponent-model
• Algorithm family: Policy-Space Response Oracles (Lanctot et al. 2017) — empirical-game best-response loop with a meta-solver and a best-response oracle.
• Implementation source: libs/nous/agreement-search/src/psro-opponent-model.ts.
• Intended use: agent-to-agent negotiation (§179.7.6) and competitive commercial negotiation (§179.7.1) where the counterparty will adapt strategically to whatever the platform proposes. The recommended candidate must survive the counterparty's best response, not just optimize Pareto-product against fixed utilities.

2. Scoring rule#

• Utility aggregation: game-theoretic — payoff tensor M_p[i, j] = party p's true utility under the deterministic proposal-arbitration simulator's outcome for policy pair (σ_A^i, σ_B^j).
• Meta-solver: replicator dynamics (general-sum convergent), fictitious play (zero-sum provable), or uniform (sanity control).
• Fairness metrics reported: Nash exploitability bound, per-party meta-strategy distribution, candidate marginal distribution.
• Handling of hard constraints: filter gate runs on every candidate considered by every policy.
• Handling of uncertainty: payoff modes posterior_mean | lcb | thompson against the §179.3.2.5 utility-uncertainty model. lcb penalizes over-fit to noisy preferences.

3. Inputs#

• Policy parameter space: caller-defined; default uses an acceptance-threshold + concession-rate parameterization.
• Best-response oracle: (1, λ)-evolution strategy with Gaussian perturbations; deterministic given seed.
• Game simulator: caller-supplied simulateGame(σ_A, σ_B) → outcome. Default Concordia simulator intersects accepted candidate sets, picks the survivor by joint policy-weighted score, and falls back to BATNA on no-overlap.
• Time / iteration budget: psroIterations × oracleSearchBudget.
• Randomness: seeded mulberry32 for both meta-solver and oracle.

4. Outputs#

• Accepted candidate contract: the most-robust candidate by marginal probability under Σ π_A(σ_A) π_B(σ_B) · 1[game(σ_A, σ_B) chose c].
• Per-party meta-strategy distribution: π_A, π_B, ..., showing how the equilibrium mixes over policies.
• Exploitability trace: Σ over parties of (BR payoff − meta payoff) decaying toward zero as the empirical game approaches an ε-Nash equilibrium.
• Empirical payoff tensor: preserved for audit.

5. Evaluation#

6. Known limitations#

• The proposal-arbitration simulator's fidelity bounds the result; a poorly-modeled simulator gives a misleading equilibrium.
• General-sum convergence under replicator dynamics is empirical, not guaranteed; use fictitious play for zero-sum subproblems.
• Best-response oracle is heuristic; may miss strict best responses if the policy space is poorly conditioned.
• Compute cost grows with policy population; the practical limit is ~10 policies per party per run.

7. Guardrails#

• Candidate safety gate: filterCandidates on every candidate evaluated under any policy.
• Clause static validation: inside filterCandidates.
• Redline separation: redlines apply per-party; PSRO cannot laundered them through equilibrium selection.
• Authority bounds in agent-to-agent runs: the simulator's acceptance check enforces the §179.7.6.3 policy-constrained authority bounds; agent acceptance outside its authority is rejected before reaching equilibrium computation.

8. References#

• libs/nous/agreement-search/src/psro-opponent-model.ts
• Lanctot et al. (2017), "A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning" (PSRO).
• McMahan, Gordon, Blum (2003), "Planning in the presence of cost functions controlled by an adversary".
• docs/research/bibliography.md §opponent-modeling