# M10.3 evaluation reports

Generated by `scripts/run_m10_3_evaluations.py report` from the committed runs.
Every corpus is synthetic and labelled by its construction; every threshold is
provisional; the launch gate refuses all four. Intervals are 95% Wilson.

## item_writing — m10-3-item-writing (run-m10-3-item-writing-20260911T171028Z)

- task: generation: the unit is a variant request; the right answer is a valid
  variant
- items 24, answered 24, abstained 0/24 = 0.00 [0.00, 0.14], errored 0/24 = 0.00
  [0.00, 0.14]
- validity 21/24 = 0.88 [0.69, 0.96]
- calibration: none (the writer states no confidence in its variants)
- harm: {'key_changed': 1, 'key_given_away': 1, 'operands_lost': 0}
- slices: 13 reported, 13 under 20 items and marked as unable to carry a
  conclusion
- launch: refused

## bias_assist — m10-3-bias-assist (run-m10-3-bias-assist-5e0edfc8947c6d05)

- task: detection: flag a passage that carries a biased claim, clear one that
  does not
- items 42, answered 42, abstained 0/42 = 0.00 [0.00, 0.08], errored 0/42 = 0.00
  [0.00, 0.08]
- precision 12/18 = 0.67 [0.44, 0.84]
- recall 12/30 = 0.40 [0.25, 0.58]
- false positive rate 6/12 = 0.50 [0.25, 0.75]
- false negative rate 18/30 = 0.60 [0.42, 0.75]
- expected calibration error 0.307 (measured only over predictions that stated a
  confidence, 18 of 42)
- harm: {'biased_passages_cleared': 18, 'cleared_items':
  ['bias-gender_trait-paraphrase-1', 'bias-gender_trait-paraphrase-2',
  'bias-gender_trait-paraphrase-3', 'bias-cultural_universal-paraphrase-1',
  'bias-cultural_universal-paraphrase-2',
  'bias-cultural_universal-paraphrase-3', 'bias-language_gate-paraphrase-1',
  'bias-language_gate-paraphrase-2', 'bias-language_gate-paraphrase-3',
  'bias-normality_default-paraphrase-1', 'bias-normality_default-paraphrase-2',
  'bias-normality_default-paraphrase-3', 'bias-sensory_default-paraphrase-1',
  'bias-sensory_default-paraphrase-2', 'bias-sensory_default-paraphrase-3',
  'bias-representation_limit-paraphrase-1',
  'bias-representation_limit-paraphrase-2',
  'bias-representation_limit-paraphrase-3']}
- slices: 11 reported, 11 under 20 items and marked as unable to carry a
  conclusion
- launch: refused

## tutor — m10-3-tutor-preview (run-m10-3-tutor-preview-2014a42a94b2af70)

- task: agreement: each scenario has one expected outcome and code
- items 46, answered 46, abstained 0/46 = 0.00 [0.00, 0.08], errored 0/46 = 0.00
  [0.00, 0.08]
- agreement 46/46 = 1.00 [0.92, 1.00]
- calibration: none (the preview engine states no confidence)
- harm: {'disagreements': []}
- slices: 7 reported, 7 under 20 items and marked as unable to carry a
  conclusion
- launch: refused

## integrity_classifier — m10-3-integrity (run-m10-3-integrity-91af9527b3282c81)

- task: detection: flag a copied, pasted or generated answer; clear the rest
- items 30, answered 30, abstained 0/30 = 0.00 [0.00, 0.11], errored 0/30 = 0.00
  [0.00, 0.11]
- precision 4/4 = 1.00 [0.51, 1.00]
- recall 4/22 = 0.18 [0.07, 0.39]
- false positive rate 0/8 = 0.00 [0.00, 0.32]
- false negative rate 18/22 = 0.82 [0.61, 0.93]
- expected calibration error 0.290 (measured only over predictions that stated a
  confidence, 4 of 30)
- harm: {'learners_wrongly_flagged': 0, 'wrongly_flagged_items': [],
  'missed_by_action': {'tutor_copy': 6, 'external_paste': 4, 'generated': 8}}
- slices: 8 reported, 6 under 20 items and marked as unable to carry a
  conclusion
- launch: refused
