vllm-project/vllm: A high-throughput and memory-efficient inference and serving engine for LLMs

vLLM, an open-source library for fast LLM inference and serving, has released its alpha version V1 with significant performance improvements. The update includes a 1.7x speedup, optimized execution, enhanced multimodal support, and zero-overhead prefix caching, aiming to provide easy, fast, and cheap LLM serving for everyone.