RustyRAG lowest-latency open-source RAG on GitHub

RustyRAG is an open-source retrieval-augmented generation application built in Rust that achieves sub-200ms response times locally and sub-600ms responses across continents without GPU hardware. The system consolidates the entire RAG pipeline into a single binary, using Groq and Cerebras for low-latency language model inference, local Jina embeddings for vectorization, and Milvus for vector search. Key features include contextual retrieval with LLM-powered context prefixes for better search accuracy, semantic chunking of PDFs with page attribution, and real-time streaming responses, making it significantly faster than traditional Python-based RAG implementations.