A hybrid retrieval architecture that combines semantic search, lexical retrieval, score fusion and topology-aware document expansion for enterprise knowledge systems.
Traditional RAG systems often treat documents as disconnected chunks. This can destroy hierarchy, hyperlinks, references and the contextual relationships between documents.
Queries and relevant documents may use different terminology, reducing lexical retrieval effectiveness.
Pure semantic similarity can retrieve conceptually related content without preserving the document's structural context.
Hyperlinks and document relationships contain useful retrieval signals that chunk-level systems often ignore.
The system combines multiple independent retrieval signals before deterministically expanding the result set through document topology.
The implementation exposes several independent retrieval strategies that can be normalized and fused into a unified ranking.
Lexical relevance using term frequency, inverse document frequency and document length.
Semantic similarity using sentence-transformer embeddings and optional HNSW acceleration.
Sparse vector retrieval providing an independent lexical signal.
Metadata-level retrieval based on document title and structural information.
After primary retrieval, the expansion controller follows explicit document relationships. Linked documents are evaluated using the semantic similarity of their anchor context.
The repository includes benchmark modes for synthetic retrieval, BEIR SciFact and HotpotQA.
| Benchmark | Purpose | Command |
|---|---|---|
| Toy | Validate retrieval pathways and cycle protection |
--mode toy
|
| BEIR SciFact | Dense + lexical retrieval evaluation |
--mode beir-scifact
|
| HotpotQA | Multi-hop link-aware retrieval |
--mode hotpotqa-small
|
| All | Execute the full benchmark suite |
--mode all
|
Hybrid sum/max fusion can outperform pure sparse retrieval on NDCG@10.
Multi-pass fusion can improve Recall@100.
Reciprocal Rank Fusion combines bounded dense scores with unbounded lexical scores.
Deterministic link traversal can recover multi-hop supporting context without agentic query decomposition.
Explicit visited-state tracking prevents duplicate document traversal and parsing loops during topology expansion.
Reproduce the experiments, inspect the retrieval algorithms, and evaluate topology-aware retrieval on your own datasets.