Status: v1.0 Date: 2026-04-26 Owner: eng-isis Backlog item:
V1-P2-0021
This document defines what callers can rely on when requesting deterministic
generation through IsisLLMClient. Determinism in LLM inference is
provider-bounded — the gateway can guarantee its own contribution is
deterministic, but the upstream model's reproducibility depends on the
provider's API surface and infrastructure.
1. Caller-side contract (gateway guarantees)#
Given two LLMRequest objects with byte-identical:
failover.primary.{provider, model}messages(including content order and parts)systemtools(if any)structuredOutput(if any)sampling.temperaturesampling.topP,sampling.topKsampling.seedsampling.stopSequencessampling.maxOutputTokenssampling.presencePenalty,sampling.frequencyPenaltysampling.reasoningEffort
IsisLLMClient.complete() MUST emit a byte-identical request body to the
upstream provider on every call. The verification fixture in
libs/isis/llm-providers/src/client/llm-client.test.ts (V1-P2-0021 suite)
issues five identical requests and asserts the captured fetch bodies are equal.
Fields that intentionally vary across calls — and therefore are NOT part of the determinism boundary — include:
requestId(UUID minted per call by the gateway)traceIdif not supplied by the calleridempotencyKey(caller-supplied; absent by default)metadata(caller-controlled; sent verbatim if set)- HTTP-level timing fields like
Date,User-Agent(set by the fetch implementation)
2. Provider-side determinism#
For each supported provider, deterministic mode requires:
| Provider | Deterministic settings | Notes |
|---|---|---|
| Anthropic | temperature=0, fixed top_p, fixed top_k (optional) |
Anthropic does not currently expose a request seed. Identical requests may still produce slightly varying outputs across versions of the underlying model server. Pin to a versioned model id (e.g., claude-sonnet-4-20250514) to maximise reproducibility. |
| OpenAI | temperature=0, seed=N |
OpenAI honors seed per their API docs; identical seed + temperature 0 produces the same output for the SAME system_fingerprint (returned in the response). When system_fingerprint changes, identity is not guaranteed. |
| Google (Gemini) | temperature=0, topP, topK |
Gemini does not expose a request seed in the public REST API. Reproducibility is best-effort under temperature 0. |
The gateway documents these guarantees but cannot enforce upstream
reproducibility. Callers depending on byte-identical OUTPUT must pin to OpenAI
with a fixed seed and assert system_fingerprint stability across calls.
3. Cost / quota bookkeeping#
Deterministic mode does not affect cost or quota arithmetic. Identical requests still consume one quota unit per call and accumulate in the ledger normally.
4. Verification fixture#
V1-P2-0021 is verified by the test suite
IsisLLMClient — V1-P2-0021 deterministic-mode contract in
libs/isis/llm-providers/src/client/llm-client.test.ts. The fixture:
- Builds a
buildRequest(...)envelope withtemperature=0, fixedseed=42,topP=0.95,topK=40, fixedmaxOutputTokens, and fixed messages. - Issues five
complete()calls back-to-back against a fetch mock. - Asserts the five captured request bodies (the bytes sent to the provider) are pairwise identical.
The mock returns the same recorded response on every call so no upstream non-determinism is involved; the test isolates the gateway's own determinism contract.