KV cache calculator

Updated 12 November 2026 · LLM Hardware team

This calculator estimates how much memory the KV cache needs at a given context length. It uses the site formula kv_bytes_per_token = 2 x layers x kv_heads x head_dim x bytes_per_element, then multiplies by the token count (decimal GB).

Best for: checking whether a 32K or 128K window still fits after weights, before you raise Ollama or llama.cpp context. The VRAM calculator covers weights at a short context (about 4K); add this cache estimate on top for longer windows.

KV cache

8.6 GB

Bytes per token

262,144

2 × 64 × 8 × 128 × 2 × 32768 ÷ 1e9 = 8.6 GB

How the KV cache is calculated

kv_bytes_per_token = 2 × layers × kv_heads × head_dim × bytes_per_element

The leading 2 covers separate keys and values.

Worked example: Qwen3-32B at f16 (2 bytes per element) and 32,768 tokens is 2 x 64 x 8 x 128 x 2 x 32,768 / 1e9 = 8.6 GB. q8_0 (1 byte) is about half (4.3 GB). q4_0 (0.5 bytes) is about a quarter (2.1 GB). Those cache-type options match the calculator and Ollama's OLLAMA_KV_CACHE_TYPE values (f16, q8_0, q4_0), which apply when Flash Attention is enabled.

Limits: this assumes full attention on every layer. Sliding-window or hybrid-attention models (for example Gemma) and MLA models (for example DeepSeek) use less and are not modelled. Cite this: LLM Hardware team, KV cache calculator, https://llmhardware.io/kv-cache-calculator, formula as of 12 November 2026.

Preset results

A dash in the 128K column for Qwen3 rows means the model's config.json max_position_embeddings is 40,960, so 128K is above the listed native context. Qwen's cards reach 131,072 tokens with YaRN. The KV cache quantization guide publishes the 128K cache figures for Qwen3-32B (34.4 GB at f16) and Qwen3-30B-A3B (12.9 GB at f16).

ModelLayersKV headsHead dimmax_position_embeddingsKV f16 @4KKV f16 @32KKV f16 @128KSource
Qwen3-8B 36 8 128 40,960 0.6 GB4.8 GB— config.json checked 2026-10-30
Llama 3.1 8B 32 8 128 131,072 0.5 GB4.3 GB17.2 GB config.json checked 2026-10-30
Qwen3-30B-A3B 48 4 128 40,960 0.4 GB3.2 GB— config.json checked 2026-10-30
Qwen3-32B 64 8 128 40,960 1.1 GB8.6 GB— config.json checked 2026-10-30
Llama 3.3 70B 80 8 128 131,072 1.3 GB10.7 GB42.9 GB config.json checked 2026-10-30

VRAM calculator · methodology page

Frequently asked questions

How much KV cache does a 32K context use?

It depends on the model. For Qwen3-32B at f16, 32K is 8.6 GB of KV cache. The same window is about 4.3 GB at q8_0 and 2.1 GB at q4_0. Pick the preset and context above to see the number for another model.

Does the VRAM calculator already include this cache?

The VRAM calculator's 1.2 factor plus 2 GB covers a short context (about 4K). For 32K or 128K, add the KV cache from this page on top of that weight estimate.

What are f16, q8_0, and q4_0?

They are the KV cache types this calculator, Ollama (OLLAMA_KV_CACHE_TYPE), and llama.cpp use. f16 is the default high-precision cache (2 bytes per element). q8_0 uses about half the cache memory of f16. q4_0 uses about a quarter. In Ollama, quantized cache types are used when Flash Attention is enabled.

Why is 128K a dash for Qwen3 presets?

The preset table follows each model's config.json max_position_embeddings. For the Qwen3 presets that value is 40,960, so 128K is shown as a dash. Qwen documents a 131,072-token window with YaRN; those 128K cache figures are in the KV cache quantization guide.

Can I use this for Gemma or DeepSeek?

Not as a complete estimate. The formula assumes full attention on every layer. Sliding-window or hybrid-attention models such as Gemma, and MLA models such as DeepSeek, use less cache than this page will show.