Key fact: Nemotron 3.5 Bolt is 74B total and 9B active. A Q4 estimate of the full model is 46.4 GB, so it does not fit a 24 GB or 32 GB GPU.
How we got it: NVIDIA model card, 15 October 2026. Formula: 74 × 0.5 × 1.2 + 2.
Checked:
Nemotron 3.5 Bolt Hardware: 24 GB Runs the 2-bit File, Not Q4
By LLM Hardware team · Methodology ·
Key facts
- Q4 formula
- 46.4 GB; NVIDIA's 2-bit file targets 24 GB
- Ollama tag
- nemotron-3.5-bolt:latest listed 25GB on 17 Oct 2026
- Checked
- 2026-10-17
Checked 2026-10-17. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.
Citeable facts
Key fact: The weights NVIDIA shipped for local use are a 2-bit checkpoint aimed at 24 GB, not a Q4 file.
How we got it: Same card: IQ2-XXS experts, architecture search targeting under 24 GB at batch size 1 and 128K context.
Checked:
Key fact: The parent Super VL model is 120B total and 12B active. Q4 is 74 GB. Ollama’s q4_K_M file was listed at 90GB on 17 October 2026.
How we got it: NVIDIA blog, 15 October 2026. Formula: 120 × 0.5 × 1.2 + 2. Ollama library tag 120b-a12b-q4_K_M on 17 October 2026. Q8 was listed at 133GB and does not fit 128 GB.
Checked:
Key fact: Ollama listed nemotron-3.5-bolt:latest at 25GB on 17 October 2026. NVIDIA’s runtime budget for the 2-bit file is under 24 GB.
How we got it: Ollama library: nemotron-3.5-bolt:latest, :74b-a9b, and :74b-a9b-iq2 shared one hash and a 25GB label. NVIDIA card: under 24 GB at batch 1 and 128K.
Checked:
Methodology — parameter math, quantization bytes, and the source list.
Nemotron 3.5 Bolt is for a 24 GB GPU only if you load the 2-bit checkpoint NVIDIA published for that budget. The model is 74 billion parameters total and 9 billion active. Our Q4 formula on the total, 74 × 0.5 × 1.2 + 2, is 46.4 GB, which misses every 24 GB and 32 GB card. Best for someone who already has 24 GB and wants NVIDIA’s local agent checkpoint, not a higher-quality quant.
What fits
| Checkpoint | Memory | 24 GB | 32 GB | 64–128 GB |
|---|---|---|---|---|
| Bolt 2-bit (the shipped local file) | NVIDIA: under 24 GB at batch 1 and 128K | Yes, that is the design point | Yes, with more context room | Yes |
| Bolt Q4 formula (74B total) | 46.4 GB | No | No | Yes on 64 GB, tight after the OS |
| Bolt Q8 formula | 90.8 GB | No | No | 128 GB yes, 64 GB no |
| Super VL Q4 (120B total) | 74 GB formula. Ollama q4_K_M listed 90GB | No | No | 128 GB fits the 90GB file. 96 GB is the edge. 64 GB no. Q8 file 133GB misses 128 GB |
| Muse Spark Open Q4 (816B total) | 491.6 GB | No | No | No. Needs a 512 GB-class machine |
MoE memory follows total parameters, because every expert sits in memory. Active parameters (9B on Bolt, 12B on Super VL, about 42B on Muse Spark Open) describe compute per token, not the download. Check a context length in the VRAM calculator. The table is weights at a short context, plus NVIDIA’s own 128K budget for the 2-bit Bolt file.
Which file to load
Weights: nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB. The card, read 15 October 2026, says the experts are stored at IQ2-XXS (about 2.06 bits) so the network fits a 24 GB budget, and that the architecture search targeted under 24 GB at batch size 1 and a 128K context. On 17 October 2026 Ollama listed nemotron-3.5-bolt:latest, :74b-a9b, and :74b-a9b-iq2 as one 25GB download. A 25GB file is the edge of a 24 GB card and may spill. The command is ollama run nemotron-3.5-bolt. Bolt E4B is still not in this table: NVIDIA said 23 October 2026, and this pass did not find those weights.
Super VL weights start at the BF16 repo. 120 × 0.6 + 2 = 74 GB at Q4. Ollama’s nemotron-3.5-super:120b-a12b-q4_K_M was listed at 90GB on 17 October, and 120b-a12b-q8_0 at 133GB. The command is ollama run nemotron-3.5-super. A 64 GB Mac or a 64 GB DGX Spark does not fit the 90GB file. 128 GB does. 96 GB is the edge after macOS. 133GB does not fit 128 GB. See best LLMs for 64 GB to 128 GB and the 24 GB GPU guide.
Muse Spark Open is not a 24 GB model
Muse Spark Open landed the same day: about 816 billion parameters, about 42 billion active, up to 1 million tokens of context. It is a different model from Muse Glimmer 30B, which does fit a 24 GB card. Spark Open at Q4 is 816 × 0.6 + 2 = 491.6 GB. The Hugging Face llama.cpp post says the BF16 checkpoint is about 1.5 TB, and that an imatrix 2-bit quant used 304 GB and ran at 18 tokens per second on an M3 Ultra with 512 GB. That speed is theirs, not a figure from this site. Unsloth’s run guide says the dynamic 2-bit file is the one to start with on 256 GB of RAM or unified memory. A 128 GB Strix Halo box, a 128 GB DGX Spark, and a 32 GB GPU do not run it. Do not buy one of those to “run Muse Spark Open.”
When another option is better
- You want Q4 quality on 24 GB. Load a model whose Q4 actually fits: Qwen3-32B is 21.2 GB, Qwen3.8-27B is 18.2 GB. See the 24 GB GPU guide.
- You want a higher quant of a 70B. That is a 64 GB to 128 GB machine, not Bolt. Q4 of Llama 3.3 70B is 44 GB.
- You only have 16 GB. Bolt does not fit. A 14B at Q4 is 10.4 GB and does. See what you can run.
Sources
- NVIDIA Nemotron 3.5 Bolt 74B-A9B-24GB model card (read 15 Oct 2026)
- NVIDIA blog, Super VL and Bolt (15 Oct 2026)
- Hugging Face, Muse Spark Open (15 Oct 2026), including the 304 GB / 18 tok/s note on a 512 GB M3 Ultra
- Unsloth, Muse Spark Open run guide (2-bit on 256 GB)
- LLMHardware methodology
Questions
Does Nemotron 3.5 Bolt fit in 24 GB?
The published 2-bit checkpoint is what NVIDIA sized for 24 GB. A normal Q4 of all 74 billion parameters is 46.4 GB and does not fit. Do not load a Q4 or Q8 build onto a 24 GB card.
Will a 16 GB GPU run it?
No. NVIDIA’s own budget for the 2-bit checkpoint is 24 GB, including context. A 16 GB card is below that.
Does an RTX 5090 run the Q4?
No. 46.4 GB is more than 32 GB. The 5090 can hold the 2-bit checkpoint with more room for context than a 24 GB card. It cannot hold a Q4 of the full 74B.
What about Nemotron 3.5 Super VL and Muse Spark Open?
The Q4 formula on Super VL is 74 GB. On 17 October 2026 Ollama listed nemotron-3.5-super:120b-a12b-q4_K_M at 90GB. 128 GB fits that file. 96 GB is the edge after macOS. The q8_0 tag was 133GB and misses 128 GB. Muse Spark Open is 816B total. Q4 is 491.6 GB, so it does not fit 128 GB. A 2-bit quant is a 256 GB-class download, not a gaming GPU.
A correction or a hardware question goes to the contact page.