Llmbuffer – Python library for cache-optimized LLM conversation history

llmbuffer is a Python library that optimizes LLM prompt caching by ordering messages so stable content (system prompt, committed history) forms a byte-stable prefix, while volatile content (RAG results, timestamps) stays at the end. This prevents cache invalidation on dynamic context changes, cutting input costs by ~43% compared to naive concatenation in benchmarks — from $0.028 to $0.016 per 15-turn conversation on Anthropic pricing.