Eve SOTA closure baseline — September 2026

Status: point-in-time baseline for closure task 0.2
Source checkout: 778b9c874163fe9f958cfb44b54b23ab2b444eff (C0)
Local observation window: 2026-09-01T15:06:40Z through 2026-09-01T15:09:53Z (L1–L5)
Machine-readable record: eve-sota-closure-baseline/2026-09-01.json

This is a baseline, not a readiness verdict. Source facts mean “present in the named checkout”; historical measurements mean “recorded by the named committed artifact”; local observations mean only this developer host in the observation window. No production or staging system was queried. A command identifier in the Evidence column resolves to the exact command and commit in the command catalog below and in the machine-readable record.

Planes, seams, tools, and views#

The prompt inventory is a union of every definition that can be offered under some valid scope. It is deliberately not the size of any one turn's served toolset: family routing, release scope, client capabilities, builder tier, and deployment bindings reduce the definitions a particular turn receives.

Stable prompt boundary#

The hash covers stable conduct variants, exact served skill text, and complete offerable tool definitions. It excludes dynamic engine, persona, daypart, memory, page, and live tour-catalog bytes. Component byte counts therefore do not claim to be the total size of a live request.

Evaluation cases, runs, and floors#

The baseline family floor is case-level pass^k, with k = 3; it is not a Wilson interval. The builder reliability table is a different statistic: the lower endpoint of a pooled run-level Wilson 95% interval. Keeping those columns separate prevents a stronger-looking number from silently changing the unit of analysis.

The baseline arm used 363 billed runs and cost $0.1109; its ratchet record does not retain a cache-read rate or median latency, so those fields are explicitly null in the machine-readable record (C2 @ 778b9c874163fe9f958cfb44b54b23ab2b444eff).

Cost, cache, and latency observations#

These are different dated samples and must not be pooled. The targeted Phase 11 checks did not constitute a full-deck rerun.

The telemetry cache floor is 70% (C2 @ 778b9c874163fe9f958cfb44b54b23ab2b444eff). The first post-initiative weekly observation recorded 3,233 turns, only 246 with reported cost, $0.000727 per reporting turn, and a 26.5% cache rate; the same-day fixed battery then measured 92% across 150 runs after serving-route pinning (C3 @ 778b9c874163fe9f958cfb44b54b23ab2b444eff; weekly evidence commit cb5d630822f85933ab63b228d5336be3b8b0a6b2). That is a historical drift and remedy observation, not the current host's live assistant telemetry.

Queue and lease state#

The current observation is the local oshun_dev Postgres database. It is not production or staging state, and the aggregate query intentionally records no titles, identifiers, or holder names.

Operator-memory posture#

Source posture is default-on only when a database URL is bound. Absence of a database intentionally becomes session-only memory; a false-valued kill switch disables the durable slice. The slice rejects member identifiers and email-shaped values, stores confirmed text verbatim, and has bounded categories.

The current shell had neither accepted database URL set and no explicit memory kill switch (L1 at 2026-09-01T15:09:53Z, checkout 778b9c874163fe9f958cfb44b54b23ab2b444eff). Therefore, a BFF launched from that unchanged shell would be session-only even though the local Postgres service and empty memory table are reachable. This does not make a claim about an already-running deployment.

Invocation, tour, voice, and selection surfaces#

All 3 customer tours are member-visible; the 1 studio plan appears only at builder tier; all 3 admin plans use the admin player (C6 @ 778b9c874163fe9f958cfb44b54b23ab2b444eff). Provider-backed voice credentials were configured for 0 of the 4 checked voice providers in the restricted local secret file, while browser speech fallbacks remain implemented (L5, C9 @ 778b9c874163fe9f958cfb44b54b23ab2b444eff). This is name-only availability; no credential value was printed or copied.

Model legs#

The registry contains 4 legs: 3 bound and 1 explicitly unbound (C7 @ 778b9c874163fe9f958cfb44b54b23ab2b444eff). Prices are dated snapshots in USD per million tokens, not live quotes.

The restricted local secret file was mode 0600; OpenRouter was set and the four checked provider-backed voice keys were unset (L5 at 2026-09-01T15:06:41Z, checkout 778b9c874163fe9f958cfb44b54b23ab2b444eff). This proves only local configuration presence, not provider health or authorization.

Dependency and runtime availability#

The mutable MinIO tag, absent Qdrant process, unattached mobile device, and Unreal graphics failure are recorded as limitations, not passes. Task 0.7 owns the expiring capability record and will re-probe these plus native desktop permissions and external channel credentials; this baseline intentionally does not pre-approve or infer those capabilities.

Claim boundary and open baseline gaps#

• No full-deck model run was performed for this document. Family floors remain the committed ratchet values.
• The baseline deck has no retained cache or latency statistic. Null is the only honest value.
• Queue and lease counts describe one local development database only.
• Credential checks establish presence by variable name only; they do not test provider reachability, quota, account policy, or data residency.
• Dependency observations do not expire themselves. Task 0.7 must replace them with stale-expiring probes before later runtime admission relies on them.
• The current model registry has no bound embedding leg. That is an explicit architecture choice for lexical docs retrieval, not evidence that retrieval quality is sufficient. Superseded 2026-09-05 by task 3.2 — see "Declared drift since capture" below. The sentence above remains the fact as captured on 2026-09-01 and is not rewritten.

Declared drift since capture#

This record is a dated capture and its verifier asserts that current source still matches it. Those two things collide the moment a later task changes something the baseline recorded, and rewriting the capture to make the check pass would destroy the "before" picture Phase 18 compares against. So the capture stays frozen and drift is DECLARED in documentedDrift: each entry names the section, the task that moved it, the value as captured, and the value now. Undeclared drift still fails the verifier.

Drift is listed as prose rather than a table on purpose: every numeric table row in this record's narrative must cite the capture's own command id and source checkout, and drift is by definition a fact from a later checkout.

• modelRegistry, tasks 3.2 and 15.1 — the embedding leg moved from unbound to perplexity/pplx-embed-v1-0.6b, priced at four thousandths of a dollar per million input tokens; then the vocabulary widened from four legs to nine, so the capture's unbound status no longer exists and the chat legs carry re-measured prices. Reproduce the current state with the same command the capture used, C7.

Task 3.2 priced the whole enumerated field from billed calls (docs/audits/EVE_SOTA_EMBEDDING_PRICING_2026-09.md). The binding rests on price and data posture; retrieval relevance is still unmeasured and is owned by task 3.5, which may re-bind.

Task 15.1 gave every leg a capability contract, including the legs Eve does not have (docs/audits/EVE_SOTA_MODEL_LEG_CONTRACT_2026-09.md): two operator-configured speech legs and three not-admitted ones joined the four bound legs. Re-pricing the chat legs from the endpoint feed under unchanged pins also measured real auction drift since capture — the turn leg's input price rose 11.2% and the escalation ladder's output price rose 36.6% while its input price fell 9.0%, the floor moving from GMICloud to Novita. Those are the auction moving, not a re-bind.

Reproduction command catalog#

Run all commands from the repository root at checkout 778b9c874163fe9f958cfb44b54b23ab2b444eff. Historical evidence commits are named beside the affected tables above. Live commands observe mutable local state and therefore reproduce the query, not necessarily the captured output.

C0 — checkout identity#

git rev-parse HEAD

C1 — tool grouping and prompt bytes#

jq '{promptBytesHash,promptBytesComponents}' docs/audits/eve-smx-ratchet.json
jq -r '.entries[] | select(.kind=="tool-definition") | .source.file' \
  docs/audits/EVE_BUILDER_PROMPT_INVENTORY_2026-08.json \
  | sort | uniq -c | sort -nr

C2 — family and Wilson floors#

jq '{familyFloors,familyFloorsSource,builderWilsonFloors,builderWilsonFloorsComment,telemetryFloors}' \
  docs/audits/eve-smx-ratchet.json

Baseline family runs are reproducibly derived as cases * 3, where the ratchet's familyFloorsSource names k=3; summing the derived runs gives the recorded deck total.

C3 — cost, cache, and latency evidence#

jq '{generatedAt,baseCommit,liveModelVerification,weeklyEvidenceLifecycle}' \
  docs/audits/eve-phase-11/2026-08-30-run.json
jq '{capturedDate,sourceCommit,costReport,sameDayRouteCalibration,remedy,decision}' \
  docs/audits/eve-smx-weekly/2026-08-23-follow-up.json

C4 — workbench views and agent MCP#

jq '{generatedAt,baseCommit,registry,phase7StrictBrowserBattery,phase8}' \
  docs/audits/eve-phase-7-8/2026-08-30-run.json

C5 — invocation inventory#

jq '{schemaVersion,contentHash,siteCount,pointCount:(.points|length),bySurface:(.points|group_by(.id|split(".")[0])|map({surface:.[0].id|split(".")[0],points:length,sites:map(.files|length)|add}))}' \
  apps/oshun/bff/src/assistant/generated/assistant-invocation-inventory.json

C6 — tour catalog#

pnpm --dir libs/oshun/shell-assistant exec tsx -e \
  "import {CURATED_ASSISTANT_TOURS as t} from './src/tour-plans.ts'; console.log(JSON.stringify({total:t.length,byAudience:Object.fromEntries(['customer','studio','admin'].map(a=>[a,t.filter(x=>x.audience===a).length])),steps:t.reduce((n,x)=>n+x.steps.length,0)}))"

C7 — model registry#

pnpm --dir apps/oshun/bff exec tsx -e \
  "import {ASSISTANT_MODEL_REGISTRY as r} from './src/assistant/model-registry.ts'; console.log(JSON.stringify({total:r.length,bound:r.filter(x=>x.status==='bound').length,unbound:r.filter(x=>x.status==='unbound').length,entries:r}))"

C8 — memory posture#

rg -n 'OPERATOR_MEMORY_CATEGORIES|SLOT_CEILINGS|resolveOperatorMemoryEnabled|operatorMemoryEnabled|assistant_operator_memory' \
  apps/oshun/bff/src/assistant/operator-memory.ts

C9 — voice and selection source surfaces#

rg -l 'data-admin-selection-scope=' apps/oshun/admin/src/components --glob '*.tsx'
rg -n 'data-admin-selection-scope=|Start voice input|Enable voice output|Dictate a message|Play this reply aloud' \
  apps/oshun/web/src/components/assistant/AssistantPanel.tsx \
  apps/oshun/admin/src/components

L1 — local queue, leases, memory, and pgvector#

docker exec oshun-dev-postgres psql -U oshun -d oshun_dev -X -v ON_ERROR_STOP=1 -Atc \
  "WITH s AS (SELECT status, count(*)::int n FROM work_item GROUP BY status) SELECT jsonb_build_object('workItems', COALESCE(sum(n),0), 'byStatus', COALESCE(jsonb_object_agg(status,n), '{}'::jsonb), 'queueEligible', COALESCE(sum(n) FILTER (WHERE status='ready'),0), 'leasedOrInProgress', COALESCE(sum(n) FILTER (WHERE status IN ('leased','in-progress')),0)) FROM s;"
docker exec oshun-dev-postgres psql -U oshun -d oshun_dev -X -v ON_ERROR_STOP=1 -Atc \
  "SELECT jsonb_build_object('leaseRows', count(*), 'live', count(*) FILTER (WHERE (lease->>'expiresAt')::timestamptz > clock_timestamp()), 'expired', count(*) FILTER (WHERE (lease->>'expiresAt')::timestamptz <= clock_timestamp()), 'holders', count(DISTINCT lease->>'agentId')) FROM work_item WHERE lease IS NOT NULL AND status IN ('leased','in-progress');"
docker exec oshun-dev-postgres psql -U oshun -d oshun_dev -X -v ON_ERROR_STOP=1 -Atc \
  "SELECT jsonb_build_object('memoryRows', (SELECT count(*) FROM assistant_operator_memory), 'operators', (SELECT count(DISTINCT operator_sub) FROM assistant_operator_memory), 'categories', (SELECT count(DISTINCT category) FROM assistant_operator_memory), 'vectorVersion', (SELECT extversion FROM pg_extension WHERE extname='vector'));"
for eve_runtime_name in OSHUN_ADMIN_DATABASE_URL OSHUN_V1_DATABASE_URL OSHUN_ASSISTANT_OPERATOR_MEMORY; do
  if printenv "$eve_runtime_name" >/dev/null; then
    echo "$eve_runtime_name=set"
  else
    echo "$eve_runtime_name=unset"
  fi
done

L2 — local services#

docker ps --format '{{.Names}}|{{.Image}}|{{.Status}}|{{.Ports}}'
for eve_probe_port in 6379 9000 9001 6333; do
  if timeout 1 bash -c "</dev/tcp/127.0.0.1/$eve_probe_port" 2>/dev/null; then
    echo "tcp:$eve_probe_port=reachable"
  else
    echo "tcp:$eve_probe_port=unreachable"
  fi
done
docker exec oshun-dev-redis redis-cli ping

L3 — local executables and harnesses#

node --version
pnpm --version
blender --version
adb --version
adb devices -l
maestro --version
codex --version
claude --version
pnpm --dir apps/oshun/web exec playwright --version
pnpm --dir apps/oshun/web exec node --input-type=module -e \
  "import {chromium,firefox,webkit} from 'playwright'; import {existsSync} from 'node:fs'; console.log(JSON.stringify(Object.fromEntries(Object.entries({chromium,firefox,webkit}).map(([k,v])=>[k,{path:v.executablePath(),exists:existsSync(v.executablePath())}]))));"

L4 — required Unreal path#

test -x /root/workspace/UnrealEngine-5.5/Engine/Binaries/Linux/UnrealEditor-Cmd
runuser -u ueagent -- \
  /root/workspace/UnrealEngine-5.5/Engine/Binaries/Linux/UnrealEditor-Cmd -Version

The second command is expected to report the engine version and then exit red on this host's missing Vulkan/display device; availability and runnable rendering are deliberately separate facts.

L5 — name-only local provider configuration#

stat -c '%a %n' /root/Desktop/workspace/drawer/env-master.env
(
  set -a
  . /root/Desktop/workspace/drawer/env-master.env
  set +a
  for eve_key_name in OPENROUTER_API_KEY DEEPGRAM_API_KEY OPENAI_API_KEY ELEVENLABS_API_KEY CARTESIA_API_KEY; do
    if [ -n "${!eve_key_name:-}" ]; then
      echo "$eve_key_name=set"
    else
      echo "$eve_key_name=unset"
    fi
  done
)

This command prints variable names and set/unset state only. It must never be changed to print lengths, prefixes, hashes, or values into committed evidence.