Pruning RAG context down to what the answer actually needs

Kapa, which builds AI assistants for technical documentation and product knowledge bases, added a pruning step between its retrieval and answer-generation stages to cut costs without sacrificing accuracy. A small, cheap AI model reviews all retrieved document chunks alongside the question, discards irrelevant ones, and passes only the useful chunks to the more expensive model. This drops about 68% of retrieved context, preserves 96% recall, and reduces per-query costs by roughly 34%.