# Where the sync evaluators run (`182.C.17.13`)

Last updated: 2026-09-10

The decision this phase asked for, taken from the `182.C.17.12` numbers rather
than from a preference. The measurements are in
`apps/oshun/bff/src/generation/human-video-evaluator-throughput.ts`, kept as
data so a different host can be measured the same way and compared.

## The numbers the decision rests on

Measured on the Linux dev box — 4 cores, no GPU — over real clips of a real face
through the real pipeline, at two lengths to confirm linearity.

| Stage                          | 10 s clip | 20 s clip | Per video-second |
| ------------------------------ | --------- | --------- | ---------------- |
| prepare (decode/detect/crop)   | 7.09 s    | 14.99 s   | ~0.73 s          |
| SyncNet v2 scoring             | 9.07 s    | 18.26 s   | ~0.91 s          |
| speech activity + mouth motion | 0.47 s    | 0.91 s    | ~0.05 s          |
| **total**                      | 16.63 s   | 34.16 s   | **~1.70 s**      |

**0.59× realtime**, and linear: the same per-second figure at both lengths, so
capacity is multiplication rather than a curve fit.

And the concurrency curve, which is the part that decided the hosting shape:

| Concurrent evaluators | Video-seconds per wall-second | Against one |
| --------------------- | ----------------------------- | ----------- |
| 1                     | 0.42                          | —           |
| 2                     | 0.50                          | +19%        |
| 3                     | 0.52                          | +24%        |
| 4                     | 0.48                          | +14%        |

It **turns over inside the measured range**. Four evaluators — one per core —
are worse than three. Each process already saturates the cores with its own
threads (torch at 4, plus ffmpeg and OpenCV), so more processes contend instead
of parallelising.

## The decision

**CPU inside the BFF worker, one evaluator at a time, is the production path for
the current shape of the workload. No GPU worker is provisioned now.**

Three reasons, in order of weight.

**1. The throughput is sufficient for the workload this phase produces.** At
0.42 video-seconds per wall-second a ten-second take is evaluated in about 24
seconds. Generation itself takes a provider tens of seconds to minutes, so
evaluation is not the long pole — it is a fraction of a step that is already
slower than it is.

**2. A GPU worker would buy latency the pipeline cannot spend.** The evaluation
happens after the render and before the release gate. Shaving twenty seconds off
a step inside a multi-minute job changes nothing a creator can perceive, and it
adds a network hop, a queue, a second place for weights to be stale, and a
second thing to be down.

**3. The scaling lever is horizontal, not vertical.** The curve says a single
box gains almost nothing from concurrency, which means the way to evaluate more
takes is more workers rather than a bigger one — and more BFF workers are
something the deployment already knows how to make.

### What would reverse it

Named now so it is a threshold rather than a judgement call later:

- **Sustained demand above one worker's capacity.** 0.42 video-seconds per
  wall-second is about 25 video-minutes an hour per worker. Past the point where
  adding workers is more expensive than a GPU host, the arithmetic changes.
- **An evaluator that needs a GPU.** The current two run on CPU because that was
  measured, not assumed. A third — `182.C.17.01` lists the candidates — could
  require one, and then the hosting question is answered by the model rather
  than by throughput.
- **Long-form takes.** At 0.59× realtime a ten-minute take costs seventeen
  minutes of CPU. `182.C.22` is where that lands, and it is a different
  calculation from the shot-length work this phase measured.

The route is already in the repo when it does reverse: RunPod through
`libs/isis/runpod-surface/` and `runpod-comfy-client.ts`, which is how the
GPU-bound generation lanes already reach hardware.

## The CPU path is kept regardless

Whatever hosts production, the CPU path stays, and this is not a fallback — it
is what CI runs. Every measurement in `182.C.17.03` through `182.C.17.11` was
produced by it on this box, including the adversarial suite. An evaluation stack
that can only be exercised on hardware CI does not have is a stack whose
regressions are found in production.

## The scheduling policy that follows

Enforced by `assertDoesNotStarveRenders`, not left to configuration:

- **One evaluator at a time** on a host with four cores or fewer. Three would
  deliver 24% more work while taking three-quarters of the box from the renders
  and transcodes that are the product; that trade is only worth making where
  there are cores nothing else wants.
- **Three cores reserved** for render, transcode and the request path.
- **Threads capped as well as processes.** They are the same resource: one
  process with every thread starves the box exactly as thoroughly as four
  processes with one thread each, and a policy that counted only processes would
  have a hole the size of the box.
