# Model cost for tests and harnesses — the full record

The binding rule lives in CLAUDE.md ("Test & harness model binding"). This file
carries the measurements and the two corrections behind it, so the rule cannot
be "optimised" without re-reading the evidence.

**A model slug on OpenRouter is an auction, not an endpoint — price the ROUTE,
not the model.** `/v1/models` returns one `pricing` object per slug and it is
not what you pay: `deepseek/deepseek-v4-flash` has eighteen providers behind it
and OpenRouter picks per request. Measured against the live API on 2026-08-08
(`/v1/models/<author>/<slug>/endpoints` for the table,
`usage: { include: true }` on a real call for the billed rate):

| routing arm                             | in /1M     | out /1M    | who served it                    |
| --------------------------------------- | ---------- | ---------- | -------------------------------- |
| `deepseek-v4-flash`, no preference      | $0.074     | $0.162     | 3 different providers in 3 calls |
| `deepseek-v4-flash`, `sort: price`      | **$0.068** | **$0.137** | StreamLake ×3 (fp8)              |
| `deepseek-v4-flash-0731`, no preference | $0.127     | $0.246     | 3 different providers in 3 calls |
| `deepseek-v4-flash-0731`, `sort: price` | $0.102     | $0.180     | DeepInfra ×3 (fp4)               |
| `claude-opus-5`                         | $5.00      | $25.00     | —                                |

Against Opus 5 that is still ~74x on input and ~182x on output at the cheap end,
which is the whole reason for the rule.

**Two corrections, both measured rather than argued (2026-08-08):**

1. An earlier version of the rule claimed the bare `deepseek-v4-flash` alias
   cost $0.14/$0.28 against the `-0731` snapshot's $0.09/$0.18, and pinned
   `-0731` to "save 1.56x". **That was wrong** — those were `/v1/models` summary
   fields, not routed prices. Priced properly, the alias is the CHEAPER of the
   two, because it has cheaper third-party endpoints. The pin made the harness
   ~1.5x dearer.
2. **The dominant lever is not the slug, it is provider routing**, and we were
   not sending any. `sort: "price"` moves the same slug from $0.074/$0.162 to
   $0.068/$0.137 and keeps it on one provider instead of three.

**Why `-0731` stays bound anyway (user decision, 2026-08-08):** this repo closes
defects on Fisher exact tests over arms run in different sessions, and a slug
whose model can change underneath it makes those arms incomparable. Buying
reproducibility at 1.5x on a model already ~100x cheaper than the frontier is
the trade the rule wants. Do not "optimise" this back to the alias without
re-reading this paragraph.

**Route re-measured 2026-08-16 (SMX P0 baseline session) — stale a third time,
in our favor:** `/endpoints` for `-0731` now lists StreamLake fp8 at
$0.061/$0.123 and Baidu fp8 at $0.062/$0.123 as the cheap end (DeepInfra fp4
moved to $0.080/$0.180). With `sort: price` +
`OPENROUTER_PROVIDER_QUANTIZATIONS=fp8` the SMX k=1 deck smoke was served by
StreamLake + Baidu and billed **$0.0432 across 121 real assistant turns**
(~$0.00036/turn, `usage: {include: true}`). The fp8 measurement pin is currently
FREE on `-0731` — the cheapest endpoints are already fp8, so pinning no longer
costs the $0.130/$0.267 premium the 2026-08-08 note recorded.

**Route re-measured 2026-08-30 — price-only was no longer a quality pin.** The
official endpoint feed listed a new cheapest fp8 endpoint, OpenInference, at
$0.030/$0.140 per million. A current seven-case builder slice routed there and
fell to **4/7**: two required read tools were skipped and a card-gated write's
arguments arrived as plain assistant JSON. On the same code, restricting the
turn route to the three endpoints with retained successful builder measurements
(`baidu/fp8`, `deepinfra/fp8`, `streamlake/fp8`) passed **7/7**, served by Baidu
and DeepInfra, at $0.0029 total with 5.2 s median latency and 71.4% cache-read.
Baidu was the cheapest admitted endpoint at the capture ($0.04494/$0.08988 per
million). The conclusion is narrower than “one provider is bad”: an auction can
change quality when its price order changes, so the serving pin must include a
measured endpoint set as well as slug and quantization. The raw before/after
logs and hashes are retained in `docs/audits/eve-phase-2/`.

Operational notes:

- `OPENROUTER_PROVIDER_SORT=price` is read by
  `resolveOpenRouterProviderPreferences` (`libs/shared/ai`), OpenRouter-only,
  OFF unless set. The assistant registry separately supplies its serving
  default, and `OPENROUTER_PROVIDER_ONLY` is the explicit endpoint-set override.
  Environment fields merge over independent registry safety fields rather than
  deleting the whole default object. Verified reaching the wire with a positive
  control: `OPENROUTER_PROVIDER_QUANTIZATIONS=int2` comes back
  `400 provider.quantizations.0: Invalid option…`, which OpenRouter could only
  say if the field arrived.
- `sort: price` also buys provider STABILITY: `-0731` went to DeepInfra 3/3
  instead of three different providers, so quantization stops varying between
  arms without an explicit `OPENROUTER_PROVIDER_QUANTIZATIONS` floor (which on
  this slug would cost $0.130/$0.267 — more than plain `-0731`).
- **Pin quantization when MEASURING, not just spending.** A Fisher arm run
  against fp8 and its comparison against fp4 are not the same experiment. Set
  `OPENROUTER_PROVIDER_QUANTIZATIONS=fp8` for measurement work — free on the
  alias, whose cheapest endpoint is already fp8.
- **Price the route before trusting any table, including this one.** It has gone
  stale twice.
  `curl -s "https://openrouter.ai/api/v1/models/deepseek/deepseek-v4-flash/endpoints" -H "Authorization: Bearer $OPENROUTER_API_KEY"`
  lists every provider with its own pricing/context/quantization; adding
  `usage: { include: true }` to a chat request returns the exact billed cost.
- **First-party is not cheapest.** DeepSeek's own endpoint is $0.140/$0.280 on
  both slugs; the cheap end is third-party.
- **Start cheaper than you think you need** (`:free` models where rate limits
  allow), and **label findings by model** so a weird reply can be re-judged
  without re-running everything.

## Embeddings and reranking are on this route too — and the rules differ (2026-09-05)

Measured while pricing the retrieval leg (Eve SOTA task 3.2; full table and
record: `docs/audits/EVE_SOTA_EMBEDDING_PRICING_2026-09.md`).

**The default model listing is a CHAT catalogue.** `GET /v1/models` returns no
embedding and no rerank model at all, so a candidate list assembled by recalling
slugs is a list of whatever you happened to remember. Ask for them:

```bash
curl -s "https://openrouter.ai/api/v1/models?output_modalities=embeddings" -H "Authorization: Bearer $OPENROUTER_API_KEY"
curl -s "https://openrouter.ai/api/v1/models?output_modalities=rerank"      -H "Authorization: Bearer $OPENROUTER_API_KEY"
```

Both `POST /v1/embeddings` and `POST /v1/rerank` exist, serve, and return a real
`usage.cost`.

**The fp8 pin is a CHAT-leg pin. Do not copy it here.** On the embeddings route
`provider: { quantizations: ["fp8"] }` returns
`404 No endpoints found for the request with quantization: fp8` for most slugs —
including `openai/text-embedding-3-small` and `baai/bge-m3`, whose endpoints
declare `unknown` and `fp32`. On the one slug where an fp8 endpoint exists,
`qwen/qwen3-embedding-8b`, the pin forces SiliconFlow and costs **4×**: $0.04/1M
pinned versus $0.01/1M at Nebius or DeepInfra. A pin copied across routes out of
discipline is a disqualification and a cost wearing discipline's name; the
registry now carries an explicit `quantizationPinWaiver` so dropping it has to
be argued rather than noticed.

**The catalogue price is wrong here in a new way.** `qwen/qwen3-reranker-8b`
lists `pricing.prompt: "0"` and billed a real amount on a real call. Price the
route, as ever.

**Reranking is not a rounding error beside embedding.** Cohere bills per _search
unit_ (a flat $0.001 for one query against a document batch), token- priced
rerankers bill per token, and either way one rerank costs thousands of times
what embedding that same query costs. A reranker needs a quality argument, not a
footnote.

**Data posture is a routing filter you can measure.** `provider: { zdr: true }`
and `provider: { data_collection: "deny" }` either route — meaning at least one
endpoint meets them — or 404 with the reason, which the router can only answer
by checking every endpoint. `:free` routes are refused under both, and the
refusal says `Free model training`. That is the cheapest available evidence
about what a route does with submitted text.

**An embedding leg has a constraint the chat legs do not.** Its output is
persisted and compared against vectors produced months earlier, so the endpoint
and the query-side calling convention are part of the model's identity. Two
endpoints serving one checkpoint at different quantizations return different
vectors; several models publish an asymmetric query prefix and are being called
wrong without it. Bind the slug, the endpoint set, and the protocol together.

## Video generation is on this route too — and the pricing shape is different again (2026-09-09)

Phase 182 routes video through the same OpenRouter account, and the chat rules
above do not transfer. There is no per-token input/output pair, no `/endpoints`
auction to price, and no `sort: price` lever: `GET /api/v1/videos/models` is
public (no auth needed), returns 28 models, and gives one `pricing_skus` object
per model that is **heterogeneous by design**. A single per-second rate cannot
represent it, so cost estimation needs a normalizer.

**The unit rule is derivable from the key name and nothing else: a key beginning
`cents_per_` is denominated in US cents; every other key is denominated in US
dollars.** `runway/aleph-2`'s `cents_per_second_output: "28"` is $0.28/s;
`alibaba/wan-2.7`'s `duration_seconds: "0.1"` is $0.10/s. Mixing these up is a
100x error in either direction, and the catalog carries no unit field to check
against.

### The six SKU families (every key observed on 2026-09-09)

| Family                       | Key stem                                                                                          | Unit                   | Qualifier order                                | Seen on                                           |
| ---------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------- | ---------------------------------------------- | ------------------------------------------------- |
| A. Per output second (USD)   | `duration_seconds`                                                                                | USD / s                | `[mode_]stem[_audio][_resolution]`             | Wan, Veo, Kling, Hailuo, Sora, HeyGen, Happyhorse |
| B. Per output second (cents) | `cents_per_second_output`, `cents_per_video_output_second`, `cents_per_second_video_continuation` | cents / s              | `stem[_resolution]`                            | Flux 3 Video, Runway, Grok Imagine                |
| C. Per megapixel-second      | `cents_per_megapixel_second`                                                                      | cents / MP·s           | `stem_qualitymode`                             | Flux Video Upscale                                |
| D. Per video token           | `video_tokens`                                                                                    | USD / token            | `stem[_resolution][_audio][_with_video_input]` | Every Seedance model                              |
| E. Per input unit            | `cents_per_image_input`, `reference_images`                                                       | cents / img, USD / img | none                                           | Grok Imagine, Hailuo 3                            |
| F. Per-generation floor      | `minimum_cents_per_generation`                                                                    | cents / generation     | none                                           | Runway Aleph 2                                    |

Qualifier vocabulary, in the order it appears in real keys:

- mode prefix — `text_to_video_`, `image_to_video_` (family A only)
- audio — `_with_audio`, `_without_audio`
- resolution — `_480p`, `_720p`, `_768p`, `_1024p`, `_1080p`, `_4k`
- input — `_with_video_input` (family D only)
- quality mode — `_precise`, `_creative` (family C only)
- continuation is not a suffix: it is a different stem
  (`cents_per_second_video_continuation_720p`)

Note that families A and D put the qualifiers in **different orders** —
`duration_seconds_with_audio_4k` is audio-then-resolution, while
`video_tokens_1080p_with_video_input` is resolution-then-input. A normalizer
that builds keys by concatenating suffixes in one fixed order will silently miss
half the catalog, so resolution is by an ordered candidate list per family, most
specific first, falling back to the bare stem.

### The honest gap: token-priced models cannot be estimated before the call

Family D prices per `video_tokens`, and **the catalog carries no tokens-per-
second, tokens-per-frame, or tokens-per-pixel constant**. The token count is
only known from the usage figures on the completed generation. Every Seedance
model is priced this way, including `bytedance/seedance-2.5` and
`bytedance/seedance-1-5-pro`.

So an admission-time estimate for a token-priced model has exactly two honest
options, and inventing a rate is not among them:

1. Refuse to admit the request against a spend ceiling that requires a
   pre-flight estimate, and say why.
2. Use a **dated, measured** tokens-per-second constant for that exact
   `canonical_slug` at that resolution and audio setting, recorded as a manual
   record with an expiry, and widen the estimate by the measured spread.

The second option needs a canary before it exists. Until then a token-priced
route is admissible only under a hard per-job ceiling enforced after the fact,
never as a "cheap" default.

### Prices for the lanes Phase 182 actually routes (fetched 2026-09-09)

| Model                        | `canonical_slug`                      | Priced as                                                                                           | 10 s cost                         |
| ---------------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------------- | --------------------------------- |
| `alibaba/wan-2.7`            | `alibaba/wan-2.7-20260414`            | flat `duration_seconds` $0.10/s                                                                     | **$1.00**                         |
| `alibaba/wan-2.6`            | `alibaba/wan-2.6-20260327`            | mode × resolution: t2v 480p $0.04, t2v 720p $0.08, i2v 720p $0.10, t2v 1080p $0.12, i2v 1080p $0.15 | $0.80–$1.50 (i2v 1080p $1.50)     |
| `bytedance/seedance-1-5-pro` | `bytedance/seedance-1-5-pro-20260320` | tokens: $2.4e-6 with audio, $1.2e-6 without                                                         | **not estimable pre-call**        |
| `bytedance/seedance-2.5`     | `bytedance/seedance-2.5-20260807`     | tokens: $1.07e-5, $6.4e-6 with video input                                                          | **not estimable pre-call**        |
| `google/veo-3.1`             | `google/veo-3.1-20260320`             | $0.40/s with audio, $0.20/s without; 4K $0.60/$0.40                                                 | $4.00 with audio (8 s max: $3.20) |
| `google/veo-3.1-fast`        | `google/veo-3.1-fast-20260320`        | $0.12/s with audio, $0.10/s at 720p                                                                 | $1.20                             |
| `google/veo-3.1-lite`        | `google/veo-3.1-lite-20260331`        | $0.08/s with audio, $0.05/s at 720p with audio                                                      | $0.80                             |
| `heygen/avatar-iv`           | `heygen/avatar-iv-20260625`           | flat `duration_seconds` $0.05/s                                                                     | $0.50                             |
| `kwaivgi/kling-v3.0-pro`     | `kwaivgi/kling-v3.0-pro-20260429`     | $0.112/s, $0.168/s with audio                                                                       | $1.68 with audio                  |
| `openai/sora-2-pro`          | `openai/sora-2-pro-20260320`          | $0.30/s at 720p, $0.50/s at 1024p/1080p                                                             | $3.00–$5.00                       |
| `runway/aleph-2`             | `runway/aleph-2-20260729`             | $0.28/s with a $0.56 per-generation floor                                                           | $2.80                             |
| `minimax/hailuo-3`           | `minimax/hailuo-03-20260730`          | $0.13/s **plus $0.04 per reference image**                                                          | $1.30 + $0.04·n                   |

**The scale that matters for budgeting.** One 10-second Wan 2.7 candidate costs
$1.00. At the measured chat rate above (~$0.00036 per assistant turn on
`deepseek-v4-flash-0731`), that single video candidate costs about as much as
**2,700 harness turns**. The chat rule saves ~100x against the frontier; the
video rule is about not generating candidates you did not need. Production and
hero tiers generate several candidates and may retry, so the per-request
estimate is `candidates × retries × seconds × rate`, and the number that belongs
on a dashboard is cost per _releasable_ second, not cost per attempt
(`182.C.37.06`).

**Re-fetch before trusting any number above:**

```
curl -sS https://openrouter.ai/api/v1/videos/models | jq '.data[] | {id, canonical_slug, pricing_skus}'
```

The catalog is mutable, the max age for a pricing fact is 24 h, and the
revalidation cadence that binds it is in
`docs/domains/isis/human-video/implementation-tracker.md`.
