Eve SOTA dense docs index — eve.docs-dense-index.v1

• Task: 3.3
• Evaluated: 2026-09-05
• Bound model: perplexity/pplx-embed-v1-0.6b at perplexity/int8, protocol plain, 1024 dims
• Full index: 57,008 rows × 1024 dims (222.7 MB), corpus cc21e105c22d, chunker 79148b760b1f
• Member index: 0 rows — empty by construction, loader returns null
• Record digest: 0ca5c36ac6688e69151ca17942d94508a3ae646b7ac9a3a0c4bac5c3cbc52f8b

Why a file beside the corpus#

pgvector on this host cannot compute an L2 or cosine distance even over a three-dimensional literal, and exact cosine over the real corpus size in-process costs tens of milliseconds per query — a fraction of the model turn it feeds. The index therefore lives beside the lexical corpus with the same loader discipline and no runtime service.

pgvector 0.8.2: L1 works = true; L2 failure = stack-depth-limit-exceeded; cosine failure = stack-depth-limit-exceeded. Observed by docs/audits/eve-sota-environment-capabilities/2026-09-01.json (vector.pgvector-operators) and a direct psql reproduction on 2026-09-05: '[1,2,3]'::vector <-> '[1,2,4]'::vector and <=> both raise "stack depth limit exceeded" at max_stack_depth 2MB; <+> returns 1.

Trigger for an approximate structure: exact search is O(rows × dims) per query; an order-of-magnitude larger corpus, or a query latency budget below the measured median, is the point at which an approximate structure earns its complexity (task 3.8 owns the operational half).

The exact lexical corpus boundary#

Row i of a scope's vector matrix is chunk i of that scope's corpus file. The chunker version is derived by the corpus builder from its own chunking source and constants, so a rechunk shows up as a version change rather than as rows that silently stopped lining up.

Disclosure before retrieval#

rows are exactly the chunks of this scope's lexical corpus, which the corpus builder filters by its positive member opt-in (MEMBER_AUDIENCE_PREFIXES); a member index is additionally refused if any href is not audience=member in docs-graph-inventory.json

• Member rows: 0; member violations: 0
• Member loader returns null: true
• Audience filtered before retrieval on every scope: true
• The loader never overlays one scope onto another: true (proved by the spec of that name)

Build, tombstones, freshness#

Re-check at generation#

• member: fresh; loader null
• full: fresh; loader loaded; self-query top-1 is its own row: true; exact search 42.52 ms median / 61.68 ms max over 7 queries

pnpm --dir apps/oshun/bff exec tsx scripts/build-docs-dense-index.ts --check exits 0 fresh, 1 stale, 2 corrupt. Stale means the corpus or the registry binding moved; corrupt means the files no longer match their manifest. The loader refuses both; the lexical path is the degraded mode.

Persistence#

Manifests are committed; vectors and chunk rows are local-only under these ignore rules:

• apps/oshun/bff/src/assistant/generated/docs-dense-index.*.vectors.f32
• apps/oshun/bff/src/assistant/generated/docs-dense-index.*.chunks.jsonl

Honest limits#

• The full-scope vectors and chunk rows are local-only (gitignored, ~223 MB). A checkout without them records the re-check as not performed, with the reason, instead of implying it passed; the committed manifest still binds the corpus hash, the file hashes and the embedding identity, so a rebuild elsewhere either reproduces the same files or fails --check.
• Nothing serves this index yet. search_docs remains lexical BM25F; task 3.4 decides hybrid fusion and the fail-loud modes for embedding, index, or provider loss. This record proves the index exists, binds what it claims, and searches — not that dense retrieval helps.
• Retrieval quality is unmeasured. The self-query proves a row retrieves itself; task 3.5 owns Recall@k and nDCG over the relevance set against the recorded shortlist, and task 3.7 owns promotion.
• Exact search is a documented approximation of a vector database, not a replacement for one: O(rows × dims) per query, all rows resident in memory, no incremental update. Tombstones are computed only when the previous build is present on the same machine, and the record says when they were not.
• The member index is empty by construction — no page has opted into the member audience — so the disclosure boundary is proved on the empty case and on the loader refusing to overlay scopes, not on a populated member index. If a page ever opts in, the member index must be rebuilt and the audience re-check re-run.
• The bound model's snapshot basis is a vendor version, not a dated slug or a published checkpoint. A re-pointed version invalidates every stored vector; --check catches a registry change but cannot see a silent re-point behind the same slug.
• Latency is the in-process scan on this host over unit-vector queries drawn from the index itself; it excludes the query embedding call, which is a network round trip the pricing record measured at ~200 ms median.