How much VRAM do you need to run an LLM locally?

Running a large language model (LLM) locally requires enough GPU memory (VRAM), determined by three factors: model size, quantization, and context length. A useful formula multiplies parameter count by bits-per-weight divided by 8, then adds KV cache memory (roughly 75 MB per 1,000 tokens per billion parameters) and 10% overhead. Most users should start with Q4 quantization, which shrinks models fourfold with minimal quality loss, making 7–8B models manageable on 12 GB GPUs.