Advanced Hybrid RAG - Biomedical Research Assistant
ProductionA Retrieval-Augmented Generation system designed for biomedical literature search and question answering. The project combines hybrid retrieval (Pinecone + BM25), LLM-powered query expansion, Cohere cross-encoder reranking, and persistent conversational memory to improve answer quality over traditional vector-search-only pipelines.
Problem
Standard RAG systems using only vector search struggle with biomedical queries due to domain-specific terminology, long scientific passages, and the need for high factual precision. Simple chunking breaks scientific context, and single-retrieval strategies miss relevant passages. The goal was to build a system that researchers could actually trust for literature review.
Architecture
The system uses a query router to classify incoming requests (chitchat, general, biomedical, malicious) before processing. Biomedical queries are expanded via LLM, then fed into both dense (Pinecone) and sparse (BM25) retrieval paths. Results are fused into a hybrid candidate pool, reranked by Cohere's cross-encoder, and passed to Llama 3.3 70B for generation. Conversational history is persisted in SQLite with a rolling context window.
Challenges
The biggest challenge was balancing retrieval recall with precision. Dense retrieval alone missed keyword-specific biomedical terms, while BM25 alone lacked semantic understanding. The hybrid approach required careful fusion weighting. Another challenge was sentence-aware chunking - preserving scientific context while keeping chunks small enough for the embedding model's context window. I also had to design a query router that could reliably distinguish biomedical questions from general ones without adding too much latency.
Lessons Learned
Retrieval quality matters more than model size. Switching from a simple vector-only pipeline to hybrid retrieval + reranking improved faithfulness by 52% without changing the LLM. Sentence-aware chunking was the single biggest win for answer faithfulness. I also learned that evaluation must be built early - having metrics from day one let me iterate quickly and defend design decisions with data rather than intuition.