Eve library

Synthetic negative fixtures, with measured ground truth (182.C.33.06)

Guide

Revision 1

On this page

Generated 2026-09-11 on the Linux dev server, ffmpeg 6.1.1-3ubuntu5. Reproduce with bash generate.sh <outdir>; the machine-readable result is manifest.json. The media itself is not committed — it is regenerated, deterministic, and there is no reason to carry binaries for it.

Why these exist#

The real corpus has no labels. Nobody knows the true audiovisual offset of a clip a customer complained about, so an evaluator measured only against real failures cannot be told how sensitive it is. A fixture with a known defect is the only way to ask "does the detector find a 120 ms offset, and does it find a 40 ms one".

The finding: every fixture's achieved defect differs from the requested one#

Fixture Asked for Measured (audio − video duration) Note
offset-filter-120ms.mp4 120 ms audio lag +137 ms adelay is sample-accurate; AAC adds ~17 ms of encoder padding
offset-container-120ms.mp4 120 ms audio lag +23 ms, start_time 96 ms -itsoffset is quantised to an AAC frame
drift-1pct.mp4 audio at 1.01× −24 ms over 4 s
dropped-300ms.mp4 300 ms removed −262 ms
inserted-300ms.mp4 300 ms duplicated +286 ms
silent-mouth.mp4 audio replaced by silence −6 ms the defect is content, not duration
frozen-picture.mp4 picture frozen 0 ms likewise
stretch-1.05x.mp4 both streams at 1.05× −54 ms
edit-boundary.mp4 0.5 s cut at 2.0 s +20 ms

Two ways to build "a 120 ms offset" produce two different fixtures, and neither is 120 ms. A fixture labelled with the value that was requested teaches an evaluator the wrong number, and the error is systematic rather than random — every AAC fixture carries the same encoder padding in the same direction. So the manifest records what was measured, and the requested value is kept only as provenance.

Two of the nine have no duration signature#

silent-mouth and frozen-picture are defects in content: the durations match and the file is wrong anyway. An evaluator that checks stream lengths finds seven of these nine and reports a clean bill of health for the two where a person is visibly speaking silence.

What is not here#

Real speech. These are a test pattern and a sine tone, which is enough to measure an offset detector and not enough to measure an aligner — word-level defects (dropped-speech, inserted-speech) are durations here rather than words. Labelled speech needs the consented corpus of 182.C.33.01.

Addendum, 2026-09-11 — the two timing fixtures (182.C.35.06)#

generate-timing.sh builds the two kinds the original nine did not cover, and timing-manifest.json is its measured output. Both are synthesisable, which is exactly why they belong here and why wrong-speaker, face-loss and identity-swap do not: those are content signatures of a real person.

fixture duration r_frame_rate avg_frame_rate frames bytes
cfr-source.mp4 4.000 s 25/1 25/1 100 400,672
vfr.mp4 3.040 s 50/1 625/19 100 320,789
vfr-attempt-vsync.mp4 3.080 s 25/1 25/1 77 284,340
reencoded.mp4 4.017007 s 25/1 25/1 100 118,731

Three findings, all measured rather than reasoned:

  1. -vsync vfr does not produce a variable frame rate. It drops frames to fit a constant one: 77 frames in 3.08 s, reported as an ordinary 25/1 file. A fixture suite built on that recipe would report "VFR handled correctly" forever. -fps_mode passthrough keeps every frame with its modified timestamp and produces the real thing — 100 frames in 3.04 s, r_frame_rate 50/1 against avg_frame_rate 625/19. Both are built, and the failed one is asserted not to be variable, so the difference is a test rather than a comment. Same shape as this directory's -itsoffset finding, on another flag.

  2. The asset validator cannot see a variable frame rate. VideoStreamFacts.frameRate is read from avg_frame_rate, and r_frame_rate is not in the ffprobe shape at all. The VFR file averages ~32.9 fps, comfortably inside the 12–120 policy, and is admitted cleanly. Reading r_frame_rate instead would not help either — 50 fps is also inside the policy. The gap is not the range; it is that the probe carries nothing that says variable.

  3. A lossy re-encode moves the duration by 17.007 ms (4.000 s → 4.017007 s, same 100 frames). Against the 45 ms ATSC sync window that is 38% of the budget, spent by an operation nobody thinks of as a timing change, before a single frame of drift.

Reproducing#

bash
bash docs/domains/isis/human-video/evidence/2026-09-11-negative-fixtures/generate-timing.sh /tmp/fx

The .mp4 files are not committed, for the same reason the original nine are not: the manifest records the measured facts, and the script rebuilds the bytes.