Task 3.4 now has one query-time retrieval facade behind search_docs. It
combines the existing BM25F arm and the exact dense-index arm with equal-weight
reciprocal-rank fusion (RRF, k=60) over 50 candidates per arm. Ranks are
combined instead of adding incomparable BM25 and cosine scores, exact chunk
links are deduplicated, ties are deterministic, and no page may occupy more than
two returned slots.
The implementation follows the original SIGIR RRF paper as an algorithmic reference. The paper does not establish product readiness for this corpus.
Runtime modes#
lexical-onlyis the shipping default and never resolves an embedding credential or dense index.hybrid-fallbackuses the dense arm only when the reviewed embedding route and a fresh, scope-matched index are available. Credential, admission, index, or provider failure returns lexical results.hybrid-requiredfails loudly on those same conditions.
The full lexical loader can fall back to a narrower corpus. Dense retrieval follows the corpus that actually loaded and never widens back to the requested scope. Dense rows are materialized through the lexical corpus itself, and a row/link disagreement fails instead of citing the wrong chunk.
Measurement and decision#
The provider-free mechanics battery contains agreement, disjoint relevant arms,
and empty-input controls. Both individual arms measure Recall@3 1.0 and MRR
0.75; RRF measures Recall@3 1.0 and MRR 1.0 because evidence ranked second
by both arms moves first. This is a controlled invariant, not a relevance win.
The retained Task 3.2 probe remains the authority for the chosen embedding route, its two identity ranks, price, and the decision not to adopt a reranker. The retained Task 3.3 record remains the authority for the fresh 57,008 × 1,024 full index. A new live query attempt used only the reviewed US-regional route; the provider returned HTTP 403 because regional routing is not enabled for the current account. The attempt stopped there. It did not fall through to the global endpoint and retained no member content or credential.
Claim boundary#
Task 3.1 still awaits attributed human labels in four classes. Task 3.5 owns real-corpus Recall/nDCG/MRR, citations, answer support, abstention, latency, and cost; Task 3.6 owns retrieval attacks; Task 3.7 owns promotion. Accordingly, hybrid retrieval is implemented and measurable but not promoted, lexical-only remains the default, and no reranker is admitted.