auto-round: A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU

AutoRound is an advanced toolkit that compresses large AI language models down to 2–4 bits of precision while keeping accuracy high — making powerful models faster and cheaper to run. It works across major hardware platforms and integrates with tools like vLLM, SGLang, and Transformers. A 200GB DeepSeek-R1 model compressed with AutoRound retained 97.9% accuracy, and 7B models can be quantized in roughly 10 minutes on a single GPU.