Eve library

CPU occlusion and body-visibility probe — 2026-09-10

Guide

Revision 1

On this page

Evidence for the two metrics 182.C.06.06 was still missing after the earlier probe in ../2026-09-10-cpu-reference-quality/: occlusion and full-body visibility. Both are now measured on this machine's CPU. This folder records the numbers, the control, and the one approach that was tried and failed.

Machine and models#

Linux dev server, 15 GiB RAM, 4 cores, no GPU. mediapipe 1.0.1 and opencv-python-headless 5.0.0 in a venv at ~/.cache/oshun-human-video/venv — not the shared system interpreter, and Python packages do not touch the pnpm lockfile.

Model Licence Size sha256
pose_landmarker_lite.task Apache-2.0 5.5 MB 59929e1d…
selfie_multiclass_256x256.tflite Apache-2.0 16.4 MB c6748b12…
face_detection_yunet_2023mar.onnx Apache-2.0 227 KB 8f2383e4…

Both new models are Apache-2.0 and therefore commercially usable, the same test 182.C.02.06 applied to the insightface weights it rejected.

The approach that did not work#

MediaPipe Pose's own per-landmark visibility score does not respond to occlusion. It is the obvious candidate — the field is literally named visibility — so it was tried first and measured before being discarded:

Condition mouth landmarks wrist landmark
baseline 0.998 / 0.997 0.754
opaque block drawn over the mouth 0.999 / 0.999 —
opaque block drawn over that wrist — 0.758

Covering a landmark with a black rectangle moved its visibility score by +0.001 and +0.004. Whatever that field encodes, it is not "is this landmark covered", and a metric named occlusion must not be backed by it. This is why tools/human-video/body-geometry.py reports presence and in-frame coordinates, and does not report visibility at all.

Occlusion — segmentation, with a control#

tools/human-video/face-occlusion.py: MediaPipe's multiclass selfie segmenter labels every pixel background / hair / body-skin / face-skin / clothes / accessory; within the YuNet face box, the class shares are the profile.

Four images, all derived from the committed probe photo ../2026-09-10-openrouter-face-detection/two-foreground-subjects-plus-bystanders.png, each reproducible with one ffmpeg command (below). Raw output: face-occlusion.json.

Image Subject face-skin Obstruction (clothes+accessory) Control face-skin
baseline 0.835 0.000 0.736
lower-face blocked 0.434 0.487 0.733
eye band blocked 0.568 0.266 0.736
hand over face 0.248 0.281 0.840

The control column is the second, untouched face in the same photograph. It moves by at most 0.007 across every variant, so the changes in the first column are the occlusion and not the segmenter drifting.

Thresholds follow directly: minFaceSkinRatio 0.65 sits between 0.568 and 0.733; maxObstructionRatio 0.15 separates 0.000 from 0.266. One probe is a starting point, not a calibration — 182.C.34 re-fits on a corpus.

Body-skin is reported but not thresholded. A clear face carries 0.04–0.06 of it (the neck), and the hand case moved it only to 0.095, because a hand arrives with a sleeve that the segmenter reads as clothes. One reading is not enough to threshold on, so the share is recorded for later fitting and nothing is claimed from it now.

Reproducing the four images#

sh
SRC=docs/domains/isis/human-video/evidence/2026-09-10-openrouter-face-detection/two-foreground-subjects-plus-bystanders.png
ffmpeg -i "$SRC" -vf "drawbox=x=815:y=189:w=89:h=69:color=black@1:t=fill"  lower-face-blocked.png
ffmpeg -i "$SRC" -vf "drawbox=x=815:y=155:w=89:h=34:color=black@1:t=fill"  eye-band-blocked.png
ffmpeg -i "$SRC" -i "$SRC" -filter_complex \
  "[1:v]crop=90:90:876:618,scale=86:78[hand];[0:v][hand]overlay=552:165"   hand-over-face.png
ffmpeg -i "$SRC" -vf "crop=352:346:704:77"                                 head-and-shoulders-crop.png

The hand fixture is real hand pixels — a 90×90 crop taken from one subject's own wrist and composited over the other subject's face — rather than a synthetic skin patch, so the segmenter is classifying a photograph of a hand.

The images themselves are not committed: they are 2.5 MB each and regenerate byte-for-byte from the committed source with the commands above.

Full-body visibility — pose landmarks, presence AND in-frame#

tools/human-video/body-geometry.py. Raw output: body-geometry.json.

Image People Hips Knees Ankles
baseline (waist-up, subject 1) 2 0.992, in frame 0.452, y≈1.03 0.045, y=1.21, out
baseline (subject 2) 0.997, in frame 0.797, y≈0.99 0.074, y=1.19, out
head-and-shoulders crop 1 0.001, out 0.000, out 0.000, out

Two things this shows. First, the measurement tracks framing: cropping the same subject to head-and-shoulders drops the hips from 0.997 to 0.001. Second, presence alone is not sufficient — the model reports coordinates for landmarks it places below the bottom edge (y = 1.21), so a region counts as shown only when its presence clears the threshold and its coordinates fall inside [0,1]. The scorer applies both conditions.

The pose landmarker is also the person detector 182.C.06.06 asked for: it found exactly the two foreground subjects in a photograph containing five detectable faces, which is the same subject-versus-bystander separation the face-geometry probe produced.

What is delivered, and what is still an estimate#

All nine metrics the item lists are now real measurements, with one honestly labelled fallback:

  • fullBodyLikelyInFrame — head-height arithmetic against the 7.5-head canon — is kept for deployments with no pose provider bound. The scored record carries fullBody.source, which is measured or estimated, and the estimate never presents itself as the measurement.
  • The scorer only warns about a body not being in frame when the answer is measured. Most identity references are portraits, so warning from the estimate would fire on nearly every good reference.

Occlusion of the BODY (a desk in front of a person) is still not measured, and nothing claims it is: the segmenter's classes cover what is over a face, and the scorer's occlusion issue is scoped to the face box.