Eve model-leg contract — 2026-09

Task 15.1. Evaluated 2026-09-15. Record eve.model-leg-contract.v1, digest 14f1a370cbd5.

Every model leg Eve has, what it can do, what it costs, what posture it routes under, how it refuses, and what the product does instead. The legs come from the shipped registry; capability and posture values are checked against the retained multi-leg probe plus Task 6.2's supplemental computer-use provider receipt before this record renders; the seam inventory is the shipped scanner run over Eve's real runtime.

Coverage#

The probe billed $0.002727 over 21 calls on 2026-09-05; the key was read from the environment and is not recorded (keyValueRecorded: false). Task 6.2 adds the later planning/vision evidence at docs/audits/eve-sota-computer-use-provider/2026-09-08.json: anthropic/claude-haiku-4.5, google/gemini-3.5-flash-lite, and rejected-candidate control NO_ZERO_DATA_RETENTION_ENDPOINT. The speech supplement at docs/audits/eve-sota-model-leg-contract/2026-09-15.speech.probe.json made 5 live model calls with no retained media or key value. It measured the dedicated speech endpoints' privacy-route controls as not enforced, so those server routes remain production-blocked while their model pins, prices and rollback are complete.

Legs#

turn — deepseek/deepseek-v4-flash-0731#

• Status: bound, pinned by dated-slug
• Route pins: sort price, quantizations fp8, endpoints baidu/fp8, deepinfra/fp8, streamlake/fp8
• Context: 1,024,000 guaranteed / 1,048,576 best across admitted endpoints
• Tool calling: available (live-call, 3/3 admitted endpoints)
• Structured output: available (live-call, 2/3 admitted endpoints)
• Requires: tool-calling
• Price: $0.04998/M tokens in, $0.09996/M tokens out at Baidu (endpoint-feed, 2026-09-05)
• Data posture: training denied available, zero-data-retention available (live-call)
• Fails loud: resolveAssistantAgentBinding in apps/oshun/bff/src/assistant/agent-provider-config.ts — returns null with no provider credential, and the assistant routes answer 503 assistant_not_configured rather than composing a reply without a model.
• Rollback: no model-backed agent; the deterministic engine remains available — disable the agent or remove the admitted OpenRouter credential. A prior slug or direct provider requires a new dated provider-posture review before it can be selected.
• Overrides: OSHUN_ASSISTANT_OPENROUTER_MODEL, OPENROUTER_PROVIDER_SORT / _QUANTIZATIONS / _ONLY, OSHUN_ASSISTANT_PROVIDER
• Resolved by Eve's runtime: 3 site(s), declared resolved

escalation — deepseek/deepseek-v4-pro-0813#

• Status: bound, pinned by dated-slug
• Route pins: sort price, quantizations fp8
• Context: 1,048,575 guaranteed / 1,048,576 best across admitted endpoints
• Tool calling: available (live-call, 7/8 admitted endpoints)
• Structured output: available (live-call, 5/8 admitted endpoints)
• Requires: nothing beyond the base modality
• Price: $1.1088/M tokens in, $3.3264/M tokens out at Novita (endpoint-feed, 2026-09-05); capability-complete at Baidu $1.11936/M in
• Data posture: training denied available, zero-data-retention available (live-call)
• Fails loud: ESCALATION_ENABLED_FAMILIES in apps/oshun/bff/src/routes/assistant.ts — escalation is offered only for the measured families and only on the openrouter provider; with no escalation configured the turn ends on the tier-2 result rather than fabricating a stronger answer.
• Rollback: no escalation — the tier-2 (checker-corrected) reply stands — disable the OpenRouter agent. The escalation model is exact and a different model is not accepted without a new review.
• Overrides: OSHUN_ASSISTANT_ESCALATION_MODEL, OPENROUTER_PROVIDER_SORT / _QUANTIZATIONS / _ONLY
• Resolved by Eve's runtime: 0 site(s), declared resolved

judge — deepseek/deepseek-v4-pro-0813#

• Status: bound, pinned by dated-slug
• Route pins: sort price, quantizations fp8
• Context: 1,048,575 guaranteed / 1,048,576 best across admitted endpoints
• Tool calling: available (live-call, 7/8 admitted endpoints)
• Structured output: available (live-call, 5/8 admitted endpoints)
• Requires: structured-output
• Price: $1.1088/M tokens in, $3.3264/M tokens out at Novita (endpoint-feed, 2026-09-05); capability-complete at Baidu $1.11936/M in
• Data posture: training denied available, zero-data-retention available (live-call)
• Fails loud: assistantModelForLeg in apps/oshun/bff/src/assistant/model-registry.ts — the leg has no consumer; asking for an unbound leg throws rather than returning a default, so a future judge cannot silently run on an unmeasured model.
• Rollback: no judging — nothing consumes this leg — set OSHUN_ASSISTANT_JUDGE_MODEL, or leave the leg unconsumed as it is today.
• Overrides: OSHUN_ASSISTANT_JUDGE_MODEL
• Resolved by Eve's runtime: 0 site(s), declared unresolved

embedding — perplexity/pplx-embed-v1-0.6b#

• Status: bound, pinned by vendor-versioned-name
• Route pins: sort price, quantizations none — waived on measurement, endpoints perplexity/int8
• Context: 32,000 guaranteed / 32,000 best across admitted endpoints
• Tool calling: not-applicable (endpoint-metadata, 0/1 admitted endpoints)
• Structured output: not-applicable (endpoint-metadata, 0/1 admitted endpoints)
• Requires: nothing beyond the base modality
• Price: $0.004/M tokens in, $0/M tokens out at Perplexity (billed-call, 2026-09-05)
• Data posture: training denied available, zero-data-retention available (live-call)
• Fails loud: resolveEmbeddingRoute in apps/oshun/bff/src/assistant/retrieval/embedding-client.ts — returns {configured:false, reason} for a missing key, an unknown protocol, or a missing endpoint pin; the dense index loader returns null rather than searching with vectors from an unpinned route.
• Rollback: lexical BM25F retrieval over the same corpus — the dense index is a separate artifact; with no index, no key, or a stale manifest, search_docs is served by docs-search.ts exactly as it was before task 3.3.
• Overrides: OSHUN_ASSISTANT_EMBEDDING_MODEL, OPENROUTER_PROVIDER_ONLY, OSHUN_ASSISTANT_DOCS_INDEX_DIR
• Resolved by Eve's runtime: 3 site(s), declared resolved

reranker — not admitted#

Nothing reranks. Task 3.2 priced the whole 7-model rerank field from billed calls and measured the cost shape rather than assuming it: a rerank of a 50-candidate set costs between 5,000× and 81,500× a single query embedding, and one candidate advertises pricing.prompt="0" while billing a real amount. Task 3.4 admits a reranker only "if priced and beneficial", and no relevance measurement exists yet (task 3.5), so a bound reranker today would be a cost with no measured benefit.

Admission trigger: task 3.5 measuring a reranked arm as better on the relevance set, and task 3.7 promoting it without a cost or security regression. Admission means a bound entry here with a price from a billed call, not a catalogue field.

Enforced by: model-leg-inventory.ts: any call to the /rerank route, or any import of a reranking client, inside Eve's runtime fails the inventory scan while this entry says not-admitted.

Resolution sites in Eve's runtime: 0.

speech-to-text — operator-configured#

• Status: operator-configured
• Providers: openrouter (microsoft/mai-transcribe-2, OPENROUTER_API_KEY); deepgram (nova-2, DEEPGRAM_API_KEY); whisper (whisper-1, OPENAI_API_KEY)
• Context: none — a transcription request has no token context window; the real limits are audio duration and file size, which are the provider client's to enforce and are not benchmarked here; the assistant upload route caps recordings at 10 MiB.
• Tool calling: not-applicable (vendor-documentation)
• Structured output: not-applicable (vendor-documentation)
• Requires: nothing beyond the base modality
• Price: $27.77777777777778/M audio-seconds in, $0/M audio-seconds out at Azure (billed-call, 2026-09-15)
• Data posture: training denied unavailable, zero-data-retention unavailable (live-call)
• Fails loud: resolveAssistantVoiceBinding in apps/oshun/bff/src/assistant/voice-config.ts — returns {stt: null} with no credential, an unknown explicit provider, or a provider named without its key; the route answers HTTP 503 assistant_stt_not_configured and never returns a transcript it did not receive.
• Rollback: the browser's own Web Speech recognition — the web client falls back when the route answers 503, so voice input degrades to the browser rather than disappearing.
• Overrides: OSHUN_ASSISTANT_STT_PROVIDER, OSHUN_ASSISTANT_STT_MODEL, OSHUN_ASSISTANT_STT_ENDPOINT
• Resolved by Eve's runtime: 1 site(s), declared resolved

text-to-speech — operator-configured#

• Status: operator-configured
• Providers: openrouter (qwen/qwen-audio-3.0-tts-plus, OPENROUTER_API_KEY); elevenlabs (no model pin, ELEVENLABS_API_KEY); cartesia (no model pin, CARTESIA_API_KEY)
• Context: none — synthesis is bounded by per-request character limits, not a token context window, and the assistant route caps each synthesis request at 1,500 characters.
• Tool calling: not-applicable (vendor-documentation)
• Structured output: not-applicable (vendor-documentation)
• Requires: nothing beyond the base modality
• Price: $20/M characters in, $0/M characters out at OpenRouter (endpoint-feed, 2026-09-15)
• Data posture: training denied unavailable, zero-data-retention unavailable (live-call)
• Fails loud: resolveAssistantVoiceBinding in apps/oshun/bff/src/assistant/voice-config.ts — returns {tts: null} with no credential, an unknown explicit provider, or Cartesia without a voice id; the route answers HTTP 503 assistant_tts_not_configured.
• Rollback: the browser's own speech synthesis — the web client falls back when the route answers 503.
• Overrides: OSHUN_ASSISTANT_TTS_MODEL, OSHUN_ASSISTANT_TTS_ENDPOINT, OSHUN_ASSISTANT_TTS_PROVIDER, OSHUN_ASSISTANT_TTS_VOICE_ID
• Resolved by Eve's runtime: 1 site(s), declared resolved

computer-use-planning — anthropic/claude-haiku-4.5#

• Status: bound, pinned by vendor-versioned-name
• Route pins: sort price, quantizations none — waived on measurement, endpoints amazon-bedrock/global
• Context: 200,000 guaranteed / 200,000 best across admitted endpoints
• Tool calling: available (live-call, 1/1 admitted endpoints)
• Structured output: available (endpoint-metadata, 1/1 admitted endpoints)
• Requires: tool-calling
• Price: $1/M tokens in, $5/M tokens out at Amazon Bedrock (endpoint-feed, 2026-09-08)
• Data posture: training denied available, zero-data-retention available (live-call)
• Fails loud: resolveEveComputerUseModelRegistry in apps/oshun/bff/src/assistant/computer-use-model-registry.ts — Task 14.2 makes credentials alone return no registry because the measured Bedrock pin is global. Controlled provider injections can still validate the capability contract.
• Rollback: no native computer use — Disable OSHUN_COMPUTER_USE_ENABLED or remove the provider credential; the native surface remains unadmitted and no deterministic or browser fallback impersonates it.
• Overrides: OSHUN_ASSISTANT_COMPUTER_USE_PLANNING_MODEL, OSHUN_COMPUTER_USE_ENABLED
• Resolved by Eve's runtime: 1 site(s), declared resolved

vision — operator-configured#

• Status: operator-configured
• Providers: openrouter (google/gemini-3.5-flash-lite, OPENROUTER_API_KEY)
• Context: 1,048,576 guaranteed / 1,048,576 best across admitted endpoints
• Tool calling: not-applicable (vendor-documentation)
• Structured output: not-applicable (vendor-documentation)
• Requires: nothing beyond the base modality
• Price: $0.3/M tokens in, $2.5/M tokens out at Google (billed-call, 2026-09-08)
• Data posture: training denied available, zero-data-retention available (live-call)
• Fails loud: resolveAssistantPageVisionBinding in apps/oshun/bff/src/assistant/page-vision.ts — No key or disabled vision omits inspect_page. Invalid captures, stale pages and provider errors return an honest tool failure.
• Rollback: read_page text context — Set OSHUN_ASSISTANT_VISION_ENABLED=false or disable Page screenshots in chat.
• Overrides: OSHUN_ASSISTANT_VISION_MODEL, OSHUN_ASSISTANT_VISION_ENABLED
• Resolved by Eve's runtime: 2 site(s), declared resolved

media — not admitted#

Eve generates no image or video. The BFF does carry OpenRouter image and video providers (src/generation/openrouter-image-provider.ts and openrouter-video-provider.ts), and they are PRODUCT surfaces: their callers are the hero-image and hero-video gates, not the assistant. Eve's legs are the models Eve's own runtime resolves and calls, and no assistant tool, route or turn path reaches those providers.

Admission trigger: an Eve tool that generates media — at which point the leg becomes Eve's, and needs its own price, posture, provenance and rollback here rather than inheriting the product gate's configuration.

Enforced by: model-leg-inventory.ts: an import of either generation provider from Eve's runtime fails the inventory scan while this entry says not-admitted.

Resolution sites in Eve's runtime: 0.

Seam inventory#

107 runtime files scanned, 170 seams classified, 0 findings.

Boundary: Eve's runtime: the assistant module, the assistant routes, and the assistant's build scripts, excluding specs and golden fixtures. Depth 1: a workspace package classified as non-model could begin calling a provider internally without changing these imports.

Blockers#

None.

Closure#

CLOSED — Every leg carries a measured contract; nothing in this task is blocked.

Honest limits#

• A data posture here is measured ROUTING behaviour under each leg's own pins on the capture date — OpenRouter still selected an endpoint when the filter was sent. It is not an audit of any provider's retention or training practice.
• The context window is read from endpoint metadata, not exercised: no request in this probe filled a million-token context. What IS measured is that the floor across admitted endpoints is the number a caller can rely on.
• The speech legs carry measured negative privacy postures: OpenRouter dedicated TTS routed both a non-ZDR model and the wrong requested upstream, and dedicated STT routed a non-ZDR model. Production remains fail-closed under Task 14.2 and uses the browser rollbacks.
• The TTS endpoint feed listed $20 per million characters and returned usable MP3 bytes, but current-key usage stayed unchanged during the ten-second settlement window. The contract records the feed rate and the observed zero delta separately; it does not call the route free.
• The seam inventory is depth 1 over imports and URL literals. It refuses an unclassified dependency and refutes a not-admitted leg, and it cannot see a provider call that appears inside an already-classified workspace package.
• Prices are a dated auction snapshot. Both chat legs moved between their previous snapshot and this one (turn input +11.2%, escalation output +36.6%) under unchanged pins; the snapshot is a fact about a date, not a rate card.