Date: 2025-09-20
Status: Accepted
Deciders: AI Platform Team
Context#
Lilith AI teachers need to provide accurate, cited responses grounded in authoritative sources. Pure LLM responses risk hallucination and lack verifiability. We need a system that:
- Grounds responses in source material
- Provides citations for claims
- Handles multi-tradition knowledge
- Maintains low latency
Decision#
Implement a Retrieval-Augmented Generation (RAG) pipeline with:
- Hybrid Search: BM25 (lexical) + Dense embeddings (semantic)
- Reciprocal Rank Fusion: Combine retrieval results
- Cross-Encoder Re-ranking: Precision optimization
- Citation Extraction: Map claims to sources
Architecture#
text
Query → Expansion → Hybrid Search → RRF → Re-rank → Top-K → LLM → Citations
Rationale#
Why Hybrid Search#
- Lexical: Exact term matching (important for spiritual terms)
- Semantic: Meaning-based matching
- Combination outperforms either alone
Why Reciprocal Rank Fusion#
- Score-agnostic combination
- Robust to different ranking scales
- Simple and effective
Why Cross-Encoder Re-ranking#
- More accurate than bi-encoders
- Applied only to top candidates (manageable latency)
Alternatives Considered#
- Dense-only retrieval: Missed exact term matches
- Keyword search only: Missed semantic similarity
- LLM-only (no RAG): Hallucination risk too high
Consequences#
Positive#
- Grounded, accurate responses
- Verifiable citations
- Reduced hallucination
Negative#
- Additional latency (~150ms)
- Infrastructure complexity
- Index maintenance overhead
Performance#
- Recall@10: 0.92
- MRR: 0.85
- Latency (p50): 150ms
Related#
- Algorithms - RAG Pipeline
- Sophia Search API:
apps/sophia/search-api/