Evidence for 182.C.18.08, which names nine: fast and slow speech, whisper,
shouting, laughter, singing, emotional delivery, background music, accent
variation, and overlapping speech. Those nine phrases expand to ten conditions
in code, because "fast/slow speech" is two and they behave differently.
Four of the ten were measured. Six were not, and could not be.
Why six could not be built#
The four that were built are TRANSFORMATIONS of a signal — a tempo change, a mix, an offset copy — and each produces audio that genuinely has the property. The six are different phonation or different speakers:
| Condition | Why a filter will not do it |
|---|---|
| whisper | whispered speech is unvoiced; filtering a voice gives a quiet voice |
| shouting | shouting changes formants and phonation, not only level |
| laughter | not speech; cannot come out of a speech clip |
| singing | pitch-shifting speech to a melody is not singing |
| emotional delivery | one Kokoro voice is cached and it has no emotion control |
| accent variation | an accent is another person |
A fixture that merely looks like the condition proves the detector reads the
filter. All six need recorded speech — the 182.C.33 corpus.
What the four measured, and the two holes they found#
Onset errors are against the KNOWN onsets scaled by the tempo factor, so a
correct aligner on a 1.25x clip reads near zero rather than 25% late. Onsets
carry the 182.C.18.02 calibration of −48.288 ms.
| Fixture | max onset error | stretch slope | minGop | est. SNR |
|---|---|---|---|---|
| normal | 32.9 ms | 0.994 | 0.00 | 81.3 dB |
| fast-1.25x | 52.7 ms | 0.992 | −4.99 | 81.6 dB |
| fast-1.60x | 68.4 ms | 0.992 | −4.84 | 81.2 dB |
| slow-0.80x | 63.5 ms | 0.990 | −10.20 | 81.5 dB |
| music-under-20db | 52.9 ms | 0.991 | 0.00 | 31.2 dB |
| music-under-10db | 52.9 ms | 0.991 | 0.00 | 21.5 dB |
| music-under-5db | 52.9 ms | 0.991 | 0.00 | 16.8 dB |
| overlap-0.5s | 513.8 ms | 1.016 | 0.00 | 81.5 dB |
| overlap-2.0s | 2014.4 ms | 1.406 | −14.02 | 81.5 dB |
Hole 1 — time stretch produces false mispronunciation findings#
The fast and slow clips contain identical words from an identical speaker.
Only the tempo changed. Yet 182.C.18.03's goodness-of-pronunciation falls to
−4.99 at 1.25x and −10.20 at 0.80x, past its −3.0 floor, so the gate would
report a mispronunciation that did not happen. A phase vocoder's transient
smearing looks like bad pronunciation to a frame-level acoustic model.
assertGopMeasurable now refuses a pronunciation score on time-stretched audio.
Refuses, not discounts: a discounted wrong number is still a wrong number.
The aligner, by contrast, held — 68.4 ms at 1.6x is well inside the ITU +45/−125 ms reject boundary — so timing claims are still allowed on it.
Hole 2 — overlapping speech breaks the aligner silently#
A second voice starting 0.5 s in gives a 513.8 ms onset error, four times the ITU reject boundary, and the aligner reports success. At 2.0 s it is 2014.4 ms with a 1.406 stretch slope. Forced alignment is given the words and asked where they are; it will always find them somewhere.
Worse, at the 0.5 s offset the GOP is exactly 0.00 — the take looks perfectly clean while its timing is half a second wrong. A clean-looking evaluator beside a broken one is how a bad take gets through.
assertAlignmentMeasurable and assertGopMeasurable both refuse on overlapping
speech.
Not a hole, but worth knowing — music trips the SNR gate#
The aligner and the GOP are completely unaffected by a harmonic bed even 5
dB under the speech. But the percentile-split SNR estimator reads 16.8 dB there
and the 182.C.18.04 low-snr gate fires at 20. Music under speech is not noise,
and that estimator cannot tell the difference. The finding is not suppressed;
snrFindingCaveat attaches the reason it may be the estimator rather than the
take.
Reproducing#
python tools/human-video/evaluator/make_delivery_fixtures.py --out /tmp/delivery
python tools/human-video/evaluator/ctc_align.py --audio /tmp/delivery/overlap-0.5s.wav \
--transcript "engine ships tuesday costs forty dollars"
The fixtures are seeded and not committed; the generator reproduces them.
Per-item status this evidence supports#
| Item | Was | Is |
|---|---|---|
182.C.18.08 |
not-started |
blocked-corpus: 4 of 10 conditions measured with real fixtures, 6 need 182.C.33 recorded speech |
182.C.18.03 |
done | now guarded — GOP refuses on time-stretched audio, after measuring false findings at every tempo factor |
182.C.18.02 |
done | now guarded — word timing refuses on overlapping speech, after a 513.8 ms silent error at a 0.5 s offset |