# Identity tracking through hard conditions (2026-09-10)

Evidence for `182.C.19.03`: track identity through profile turns, partial
occlusion, camera motion, lighting changes, cuts, interactions, and multiple
people.

## Two failures, kept apart

**Association** fails when the tracker loses the face. It produces
**fragmentation** — more tracks than people. That is the safe failure: a broken
track loses continuity and whoever picks it up starts again.

**Identity** fails when a track continues and the person changes. That is a
**swap**, and it is the dangerous one: the take reports the wrong person with
full confidence.

Measuring them apart is the whole value, and the measurement shows they fail
under different conditions.

## Method

Six 24-frame sequences built from the committed frame
`../2026-09-10-openrouter-face-detection/two-foreground-subjects-plus-bystanders.png`
by real geometry and photometry — a moving crop window is a camera move, a black
rectangle is an occluder, an exposure ramp is a lighting change, a splice is a
cut, and two real face crops moving past each other is an interaction between
two people whose pixels are genuinely theirs. Nothing paints a synthetic face.

Detection with YuNet, identity with SFace (128-d cosine), association by IoU
alone — deliberately, because mixing the embedding into the association would
make the tracker paper over the identity failure being measured.

Two subjects were enrolled: the faces above 0.10 relative frame height. They
cross-score at 0.230, so they are genuinely different people.

## Results

| Sequence      | tracks | confident tracks | **swaps** | min cosine | tracks matching nobody |
| ------------- | ------ | ---------------- | --------- | ---------- | ---------------------- |
| camera-pan    | 49     | 2                | 0         | 0.889      | 47                     |
| camera-zoom   | 9      | 2                | 0         | 0.901      | 7                      |
| lighting-ramp | 6      | 2                | 0         | 0.643      | 4                      |
| cut           | 9      | 4                | 0         | 0.945      | 5                      |
| crossing      | 3      | 3                | 0         | 0.388      | 0                      |
| **occlusion** | 6      | 2                | **2**     | **0.099**  | 4                      |

- **Camera motion**: identity holds (0 swaps, 0.889 minimum) while association
  fragments into 49 tracks — a pan moves a small face further than its own width
  between frames and IoU cannot bridge it. One failure, not both.
- **Lighting**: holds through a −1 to +1 stop ramp, and costs 0.25 of cosine
  (0.643 against 0.889–0.945 elsewhere). A threshold fitted under even lighting
  will be tighter than a lit scene can support.
- **Cut**: correctly _breaks_ the tracks — 4 for 2 people, 0 swaps — which is
  what a cut should do rather than continuing one across it.
- **Interaction**: 0 swaps through the overlap; one track split, which is the
  safe failure.
- **Occlusion is the one that breaks.** Two identity swaps inside the foreground
  tracks and a minimum cosine of 0.099: as the occluder covers the face the
  embedding drifts until the nearest enrolled subject is the _other_ person, and
  the track carries on with them. Identity assignment is suppressed under
  occlusion rather than trusted.

## The finding that governs every assignment

**47 of the 49 pan tracks matched nobody** — their highest mean cosine against
the two enrolled subjects was 0.243 — and every one of them was still assigned a
subject name, because nearest-neighbour always has a nearest.

An identity assignment with no bound calibration names somebody for every face
in frame. That is the same conclusion `182.C.07.03` reached about pack identity
and it takes the same remedy: return `unassigned`, and let `182.C.19.08` fit the
cutoff on the `182.C.33` corpus.

## The enrolment floor

Enrolling all six detected faces instead of the two above 0.10 relative height
changed the swap counts in **both directions** — occlusion 2 → 5, camera-zoom 4
→ 2 — because four unusable bystander embeddings act as absorbers for ambiguous
frames. An enrolment below the measurability floor is a phantom identity, and
`assertEnrollable` refuses it.

## Not tested

**Profile turns.** Rotating a face away from camera is a 3D transformation of a
head, and a still gives no geometry to rotate; a sheared or perspective-warped
frontal face is a warped frontal face, and a tracker that survived it would have
proved nothing. Needs recorded footage — `182.C.33`. `assertConditionTested`
refuses to claim it was handled.

## Reproducing

```
python tools/human-video/evaluator/make_tracking_sequences.py --out /tmp/tracking
python tools/human-video/evaluator/identity_tracking.py \
  --enroll /tmp/tracking/enroll.png --sequence /tmp/tracking/occlusion-ramp
```

Embeddings are biometric data and are never committed — only the cosines
computed from them, following `tools/human-video/face-embedding.py`.

## Per-item status this evidence supports

| Item          | Was           | Is                                                                                            |
| ------------- | ------------- | --------------------------------------------------------------------------------------------- |
| `182.C.19.03` | `not-started` | done for six of seven conditions; profile turns need `182.C.33` footage                       |
| `182.C.19.08` | pending       | a third reason thresholds cannot be universal: lighting alone cost 0.25 of cosine on one ramp |
