KV cache calculator
Updated 12 November 2026 · LLM Hardware team
This calculator estimates how much memory the KV cache needs at a given context length. It uses the site formula kv_bytes_per_token = 2 x layers x kv_heads x head_dim x bytes_per_element, then multiplies by the token count (decimal GB).
Best for: checking whether a 32K or 128K window still fits after weights, before you raise Ollama or llama.cpp context. The VRAM calculator covers weights at a short context (about 4K); add this cache estimate on top for longer windows.
KV cache
8.6 GB
Bytes per token
262,144
2 × 64 × 8 × 128 × 2 × 32768 ÷ 1e9 = 8.6 GB
How the KV cache is calculated
The leading 2 covers separate keys and values.
Worked example: Qwen3-32B at f16 (2 bytes per element) and 32,768 tokens is 2 x 64 x 8 x 128 x 2 x 32,768 / 1e9 = 8.6 GB. q8_0 (1 byte) is about half (4.3 GB). q4_0 (0.5 bytes) is about a quarter (2.1 GB). Those cache-type options match the calculator and Ollama's OLLAMA_KV_CACHE_TYPE values (f16, q8_0, q4_0), which apply when Flash Attention is enabled.
Limits: this assumes full attention on every layer. Sliding-window or hybrid-attention models (for example Gemma) and MLA models (for example DeepSeek) use less and are not modelled. Cite this: LLM Hardware team, KV cache calculator, https://llmhardware.io/kv-cache-calculator, formula as of 12 November 2026.
Preset results
A dash in the 128K column for Qwen3 rows means the model's config.json max_position_embeddings is 40,960, so 128K is above the listed native context. Qwen's cards reach 131,072 tokens with YaRN. The KV cache quantization guide publishes the 128K cache figures for Qwen3-32B (34.4 GB at f16) and Qwen3-30B-A3B (12.9 GB at f16).
| Model | Layers | KV heads | Head dim | max_position_embeddings | KV f16 @4K | KV f16 @32K | KV f16 @128K | Source |
|---|---|---|---|---|---|---|---|---|
| Qwen3-8B | 36 | 8 | 128 | 40,960 | 0.6 GB | 4.8 GB | — | config.json checked 2026-10-30 |
| Llama 3.1 8B | 32 | 8 | 128 | 131,072 | 0.5 GB | 4.3 GB | 17.2 GB | config.json checked 2026-10-30 |
| Qwen3-30B-A3B | 48 | 4 | 128 | 40,960 | 0.4 GB | 3.2 GB | — | config.json checked 2026-10-30 |
| Qwen3-32B | 64 | 8 | 128 | 40,960 | 1.1 GB | 8.6 GB | — | config.json checked 2026-10-30 |
| Llama 3.3 70B | 80 | 8 | 128 | 131,072 | 1.3 GB | 10.7 GB | 42.9 GB | config.json checked 2026-10-30 |
VRAM calculator · methodology page
Frequently asked questions
How much KV cache does a 32K context use?
It depends on the model. For Qwen3-32B at f16, 32K is 8.6 GB of KV cache. The same window is about 4.3 GB at q8_0 and 2.1 GB at q4_0. Pick the preset and context above to see the number for another model.
Does the VRAM calculator already include this cache?
The VRAM calculator's 1.2 factor plus 2 GB covers a short context (about 4K). For 32K or 128K, add the KV cache from this page on top of that weight estimate.
What are f16, q8_0, and q4_0?
They are the KV cache types this calculator, Ollama (OLLAMA_KV_CACHE_TYPE), and llama.cpp use. f16 is the default high-precision cache (2 bytes per element). q8_0 uses about half the cache memory of f16. q4_0 uses about a quarter. In Ollama, quantized cache types are used when Flash Attention is enabled.
Why is 128K a dash for Qwen3 presets?
The preset table follows each model's config.json max_position_embeddings. For the Qwen3 presets that value is 40,960, so 128K is shown as a dash. Qwen documents a 131,072-token window with YaRN; those 128K cache figures are in the KV cache quantization guide.
Can I use this for Gemma or DeepSeek?
Not as a complete estimate. The formula assumes full attention on every layer. Sliding-window or hybrid-attention models such as Gemma, and MLA models such as DeepSeek, use less cache than this page will show.