GPU VRAM Calculator for Local LLMs
Estimate how much GPU memory you need to run a model locally. Pick the model size, quantization, and context length, and see which common GPUs can hold it.
0
Model weights (GB)
0
KV cache (GB)
0
Overhead ~20% (GB)
0
Total VRAM needed (GB)
Will it fit? Common GPUs and Macs
| Device | Memory | Usable for model | Verdict |
|---|
The math, step by step:
1) Weights = parameters x bytes per parameter. A 8B model at Q4 (about 4.5 effective bits, 0.56 bytes/param in GGUF K-quants) needs 8 x 0.56 = 4.5 GB just for weights.
2) KV cache stores attention keys and values for every token in context. Exact size needs the model's layer count and hidden dimension, so this tool approximates both from the parameter count with a standard heuristic: transformer params are roughly 12 x layers x dim², and dim is roughly 128 x layers for typical architectures. Solving gives layers = cube root of (params / 196,608) and dim = 128 x layers. Sanity check: 7B gives 33 layers x dim 4224 (real Llama 7B: 32 x 4096). KV bytes = 2 (K and V) x layers x dim x 2 bytes (FP16) x context x batch, multiplied by the GQA factor (grouped-query attention shares KV heads, cutting the cache to 1/4 or 1/8 on most modern models).
3) Overhead: runtimes (llama.cpp, vLLM, etc.) need extra memory for activations, CUDA/Metal buffers, and fragmentation. We add 20% on top of weights + KV.
4) For Apple Silicon Macs, macOS reserves part of unified memory, so we count about 75% of total RAM as usable for the model.
These are planning estimates within roughly 10-20% of real usage; actual numbers vary by architecture and runtime. Estimating API costs instead of local hosting? See the LLM Price Calculator.
1) Weights = parameters x bytes per parameter. A 8B model at Q4 (about 4.5 effective bits, 0.56 bytes/param in GGUF K-quants) needs 8 x 0.56 = 4.5 GB just for weights.
2) KV cache stores attention keys and values for every token in context. Exact size needs the model's layer count and hidden dimension, so this tool approximates both from the parameter count with a standard heuristic: transformer params are roughly 12 x layers x dim², and dim is roughly 128 x layers for typical architectures. Solving gives layers = cube root of (params / 196,608) and dim = 128 x layers. Sanity check: 7B gives 33 layers x dim 4224 (real Llama 7B: 32 x 4096). KV bytes = 2 (K and V) x layers x dim x 2 bytes (FP16) x context x batch, multiplied by the GQA factor (grouped-query attention shares KV heads, cutting the cache to 1/4 or 1/8 on most modern models).
3) Overhead: runtimes (llama.cpp, vLLM, etc.) need extra memory for activations, CUDA/Metal buffers, and fragmentation. We add 20% on top of weights + KV.
4) For Apple Silicon Macs, macOS reserves part of unified memory, so we count about 75% of total RAM as usable for the model.
These are planning estimates within roughly 10-20% of real usage; actual numbers vary by architecture and runtime. Estimating API costs instead of local hosting? See the LLM Price Calculator.