Key fact: A 32 GB GPU fits a dense model up to about 50B at Q4; a 70B at Q4 needs 44 GB.
How we got it: VRAM formula: params x 0.5 x 1.2 + 2.
Checked:
Best LLMs for 32 GB VRAM: What an RTX 5090 Fits
By LLM Hardware team · Methodology ·
Key facts
- Ceiling
- About 50B dense at Q4; Bolt Q4 (46.4 GB) does not fit
- Sweet spot
- Qwen3-32B at Q5 (26 GB) or Q4 with a 32K context
- Checked
- 2026-10-15
Checked 2026-10-15. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.
Citeable facts
Key fact: The RTX 5090 has 32 GB GDDR7 and 1,792 GB/s memory bandwidth.
How we got it: NVIDIA RTX 5090 spec page, checked 14 October 2026.
Checked:
Key fact: Qwen3-32B at f16 KV cache uses about 8.6 GB for a 32K context.
How we got it: 2 x 64 layers x 8 KV heads x 128 x 2 bytes x 32,768 tokens, from the Qwen3-32B config.
Checked:
Key fact: Nemotron 3.5 Bolt at Q4 is 46.4 GB and does not fit 32 GB. The file NVIDIA shipped is 2-bit and is aimed at 24 GB.
How we got it: 74 x 0.5 x 1.2 + 2. NVIDIA model card, 15 October 2026.
Checked:
Methodology — parameter math, quantization bytes, and the source list.
A 32 GB GPU holds a dense model up to about 50B parameters at Q4 by our VRAM formula (params x bytes x 1.2 + 2): 50 x 0.6 + 2 = 32 GB. In practice the useful step over 24 GB is not a bigger model class but more room: a 32B model at Q5 or Q6 instead of Q4, or a 32B at Q4 with a long context. A 70B at Q4 needs 44 GB and still does not fit. The consumer card in this tier is the RTX 5090: 32 GB GDDR7 and 1,792 GB/s of memory bandwidth per NVIDIA.
What fits in 32 GB
| Model | Quant | VRAM formula | Fits 32 GB? |
|---|---|---|---|
| Qwen3-32B | Q4 | 21.2 GB | Yes, with about 10 GB for context |
| Qwen3-32B | Q5 | 26.0 GB | Yes |
| Qwen3-32B | Q6 | 30.8 GB | Tight, short context only |
| Qwen3-32B | Q8 | 40.4 GB | No |
| Gemma 3 27B | Q6 | 26.3 GB | Yes |
| Gemma 3 27B | Q8 | 34.4 GB | No |
| Qwen3-30B-A3B (MoE) | Q4 | 20.3 GB | Yes; sized on 30.5B total |
| gpt-oss-20b (MoE) | native MXFP4 | OpenAI: runs within 16 GB | Yes, with lots of headroom |
| Nemotron 3.5 Bolt (74B total) | NVIDIA 2-bit | Card: under 24 GB at 128K | Yes, that file, with context room |
| Nemotron 3.5 Bolt (74B total) | Q4 | 46.4 GB | No |
| Muse Spark Open (816B total) | Q4 | 491.6 GB | No |
| Llama 3.3 70B | Q4 | 44 GB | No |
MoE models are sized on total parameters, not active ones, because every expert has to sit in memory. Nemotron 3.5 Bolt’s 9B active count does not make the Q4 fit: 46.4 GB still misses 32 GB. The 2-bit checkpoint is the one that fits, and the write-up is the Nemotron 3.5 Bolt hardware page. Check your own context length in the VRAM calculator; the formula above assumes a short context. A 24 GB card’s options are on the 24 GB GPU guide.
Where 32 GB earns its keep: long context
The KV cache grows with context. Qwen3-32B (64 layers, 8 KV heads, head size 128) stores about 256 KiB per token at f16, so a 32K context adds about 8.6 GB. 21.2 GB of Q4 weights plus 8.6 GB of cache is about 30 GB: it fits 32 GB and does not fit 24 GB. Quantizing the cache to q8_0 halves that; see the KV-cache quantization guide.
When another option is better
- A used 24 GB card (RTX 3090 or 4090) runs the same 27-32B Q4 models. If you stay at Q4 and short context, 32 GB buys little. See the 24 GB GPU guide.
- 70B models need 44 GB at Q4. Two 24 GB cards, a 64 GB or larger Mac, or a 128 GB Strix Halo or DGX Spark box fit it; a single 32 GB card does not. See the 64 GB to 128 GB guide.
- Power: NVIDIA lists 575 W for the RTX 5090 and recommends a 1,000 W PSU. A unified-memory box draws far less at the wall.
Hardware for this tier
Check current price on Amazon. Speed figures are not published here; once the model fits, memory bandwidth decides tokens per second.
ASIN B0DS2X13PH. Amazon Associates link. The live price is on that page, not locked in here.
ASIN B0BG94PS2F. Amazon Associates link. The live price is on that page, not locked in here.
Sources
- NVIDIA, GeForce RTX 5090 specs (checked 14 Oct 2026)
- Qwen3-32B model card
- Qwen3-30B-A3B model card (30.5B total, 3.3B active)
- Gemma 3 27B model card
- gpt-oss-20b model card (21B total, 3.6B active)
- Nemotron 3.5 Bolt 74B-A9B model card (15 Oct 2026)
- Llama 3.3 70B Instruct model card
- LLMHardware methodology: the VRAM formula
Questions
What is the biggest model for 32 GB of VRAM?
About 50B dense at Q4 by our VRAM formula. In practice, Qwen3-32B at Q5 or Q6, or Q4 with a long context. A 70B at Q4 needs 44 GB.
Is 32 GB worth it over 24 GB?
Only if you want higher quants or long context on 27-32B models. At Q4 and short context, a 24 GB card runs the same models.
A correction or a hardware question goes to the contact page.