# @nous/self-play-loop

Self-improvement & self-play reasoning for the Phase-178 autonomous-research
stack (TODOS/phase-178 §178.3). Every learner here trains on **self-generated,
executor-verified** signal — nothing fabricates a reward.

## Components

| Module              | Summary                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `azr.py`            | **Absolute Zero Reasoner**: propose/solve **deduction / abduction / induction** tasks over an executable integer DSL, verified by execution; **learnability reward** (favour harder-but-solvable), **TRR++** advantages (baselined per role × task-type), and a self-improving synthesis solver whose op-priors sharpen from its own verified solutions. Trained on **zero external data**, held-out solve-rate rises (fresh 0.14 → trained 0.22 in the test). (§178.3.1.1) |
| `star.py`           | **STaR / V-STaR**: rationale (skill-choice) bootstrapping with **rationalisation** for unsolved problems; the V-STaR **verifier** learns from correct _and_ incorrect attempts and can select a correct skill the policy never sampled. Accuracy climbs from chance to ≥0.9. (§178.3.1.2)                                                                                                                                                                                   |
| `self_rewarding.py` | **Self-Rewarding & Meta-Rewarding**: LLM-as-judge → **DPO preference pairs** → policy improvement; a **meta-judge** recalibrates the judge against known anchors, provably reducing judgment error under reward drift. (§178.3.1.3)                                                                                                                                                                                                                                         |
| `spiral.py`         | **SPIRAL** zero-sum self-play RL on a Nim-like subtraction game with a **closed-form optimum** (Sprague–Grundy) for verification; tabular Monte-Carlo self-play reaches ≥0.99 win-rate vs random. Optional `search_policy` hook plugs in a `@nous/tree-search-reasoning` kernel. (§178.3.1.4)                                                                                                                                                                               |
| `voyager.py`        | **Voyager skill library**: embedding-indexed executable skills, cosine retrieval, pipeline **composition**, **test-gated** iterative addition, and a Phase-47 memory sink. (§178.3.2.1)                                                                                                                                                                                                                                                                                     |
| `curriculum.py`     | **Automatic curriculum** generator: per-skill mastery, **ZPD gap analysis**, difficulty escalation as skills saturate, and a **quality-gated retraining hook**. (§178.3.2.2)                                                                                                                                                                                                                                                                                                |
| `evoagentx.py`      | **EvoAgentX** agent-workflow evolver: mutate role/tool/prompt/kernel graphs, keep the **Pareto front over quality × cost × latency**. (§178.3.2.3)                                                                                                                                                                                                                                                                                                                          |

## Scope of local verification

The substrates are deliberately controllable so _learning_ is provable, not
asserted: AZR's induction DSL, STaR's skill-selection tasks, SPIRAL's
game-theoretically-solved game, self-rewarding's known-target quality. Swap an
LM policy + Python sandbox behind the same interfaces to run the papers'
math/code evals and the reasoning-transfer claims (which need the LM and are out
of local scope).

## Testing

```bash
python3 -m pytest tests/ -q     # 24 tests
python3 -m ruff check .
```