AI pulled the Mistral / Mixtral size table. The MoE VRAM nuance and the per-size hardware picks were edited against the Mistral docs and the site’s VRAM formula.
Key facts
7B class
Fits 8 GB at Q4
Mixtral-class MoE
All experts must reside; active params only change speed
Checked
2026-10-13
Author
LLM Hardware team
Checked 2026-10-13. Author: LLM Hardware team. Amazon prices are not stored or shown on this site.
Citeable facts
Key fact: Mistral 7B at Q4 needs 6.2 GB and fits 8 GB. Nemo 12B at Q4 needs 9.2 GB and misses 8 GB. Small 22B at Q4 needs 15.2 GB and fits 16 GB. The 24B tag needs 16.4 GB and misses 16 GB.
How we got it: 7 × 0.6 + 2 = 6.2. 12 × 0.6 + 2 = 9.2. 22 × 0.6 + 2 = 15.2. 24 × 0.6 + 2 = 16.4. Ollama library on 11 October 2026 listed mistral, mistral-nemo, and mistral-small at 22B and 24B.
Checked:
Key fact: Mistral Large 4 was not a downloadable weight on 11 October 2026. This page does not size a GPU or add a buy link for it.
How we got it: Hugging Face mistralai/Mistral-Large-4.0-1T05-A52B upcoming page listed weights for 31 October 2026.
Checked:
Methodology
explains the VRAM formula, the bytes per weight at each quantization, and the full source list.
Updated 9 November 2026 · Mistral 7B to 22B · VRAM requirements · Consumer GPU guide · Ollama setup
Mistral 7B at Q4 is 7 × 0.6 + 2 = 6.2 GB, so it fits an 8 GB GPU and does not fit a 6 GB GPU. Mistral Nemo 12B at Q4 is 9.2 GB and needs 12 GB. Mistral Small 22B at Q4 is 15.2 GB and needs 16 GB. The commands are ollama run mistral, ollama run mistral-nemo, and ollama run mistral-small. In LM Studio, pick the Q4_K_M GGUF of the same size.
What is Mistral AI?
Mistral AI is a French AI company founded in 2023 that releases open-weight models known for punching above their weight class. Mistral 7B was the first model to significantly challenge Llama 2 at the same parameter count.
Mistral 7B v0.37B params, 6.2 GB at Q4. Fits an 8 GB GPU. Does not fit a 6 GB GPU. Q8 is 10.4 GB and misses 8 GB.
Mistral Nemo 12BThe best small Mistral in 2026. 12B params, 128K context window, strong code and multilingual performance. Released in collaboration with NVIDIA.
Mistral Small 22B22B parameters — a significant quality jump. Needs a 16 GB GPU at Q4. Best Mistral for complex reasoning tasks on consumer hardware.
Ollama tagsmistral, mistral-nemo, mistral-small — all available via ollama run <tag>
The Gigabyte RTX 4070 12 GB (B0BZHCQ6PF) fits mistral at Q4 (6.2 GB) and mistral-nemo at Q4 (9.2 GB). It does not fit mistral-small (15.2 GB).
VRAM is estimated using params × bytes_per_param × 1.2 + 2. Q4 is params_B × 0.6 + 2. Q8 is params_B × 1.2 + 2. FP16 is params_B × 2.4 + 2. Add extra headroom for KV cache with long context windows.
Model
Params
Q4_K_M VRAM
Q8 VRAM
FP16 VRAM
Min GPU
Mistral 7B v0.3
7B
6.2 GB
10.4 GB
18.8 GB
8 GB GPU
Mistral Nemo 12B
12B
9.2 GB
16.4 GB
30.8 GB
12 GB GPU
Mistral Small 22B
22B
15.2 GB
28.4 GB
54.8 GB
16 GB GPU
Mistral Large 2
123B
75.8 GB
149.6 GB
Does not fit 128 GB
128 GB unified
Mistral 7B at Q4 is 6.2 GB, so it fits an 8 GB GPU and does not fit a 6 GB GPU. Use the VRAM Calculator for context-length-adjusted estimates.
Which Quantization Should You Use?
Quantization trades a small amount of output quality for a large reduction in VRAM usage. For most users, Q4_K_M is the right default.
Q4_K_M — recommended
Weights are about a quarter the size of FP16 weights, and total VRAM falls by less: Mistral 7B is 6.2 GB at Q4 vs 18.8 GB at FP16. Quality loss is usually small. The Ollama default for all Mistral models. Start here.
Q8_0 — best quality
Approximately doubles the weight size vs Q4 but preserves near-FP16 output quality. Mistral 7B at Q8 is 7 × 1.2 + 2 = 10.4 GB, so it misses an 8 GB GPU such as the RTX 4060 and needs a 12 GB card. Use Q8 when you have extra VRAM and want maximum output fidelity.
FP16 — reference only
Mistral 7B at FP16 is 7 × 2.4 + 2 = 18.8 GB, so it needs 24 GB. It does not fit a 16 GB GPU. Daily use should stay on the Q4 file.
Q2/Q3 — constrained devices
Aggressive quantizations for very limited hardware. Quality degrades noticeably — particularly on instruction following and code. Only consider if your device cannot fit Q4. Mistral 7B at Q2 can run on 2–3 GB but output quality suffers.
What Mistral Models Can You Run on Your GPU?
Find your GPU or Mac below. Each card shows which Mistral models fit, and what does not.
RTX 4060 8GB
Runs:
+Mistral 7B Q4 (6.2 GB)
Does not fit:
-Mistral 7B Q8 (10.4 GB)
-Mistral Nemo Q4 (9.2 GB)
-Mistral Small 22B Q4 (15.2 GB)
An 8 GB card fits mistral at Q4. It does not fit mistral-nemo or mistral-small.
Intel Arc B580 12GB
Runs:
+Mistral 7B Q4 (6.2 GB) and Q8 (10.4 GB)
+Mistral Nemo Q4 (9.2 GB)
Does not fit:
-Mistral Nemo Q8 (16.4 GB)
-Mistral Small 22B Q4 (15.2 GB)
12 GB is the Nemo Q4 tier. It does not run mistral-small.
RTX 4060 Ti 16GB
Runs:
+Mistral Nemo Q4 (9.2 GB)
+Mistral Small 22B Q4 (15.2 GB)
Does not fit:
-Mistral Nemo Q8 (16.4 GB)
-Mistral Small 22B Q8 (28.4 GB)
-mistral-small:24b (16.4 GB)
16 GB fits the 22B Q4 estimate and does not fit the 24B tag or Nemo at Q8.
RTX 4070 12GB
Runs:
+Mistral 7B Q8 (10.4 GB)
+Mistral Nemo Q4 (9.2 GB)
Does not fit:
-Mistral Small 22B Q4 (15.2 GB)
-Mistral Nemo Q8 (16.4 GB)
12 GB fits mistral and mistral-nemo at Q4. It does not fit mistral-small (15.2 GB).
RTX 4070 Ti Super 16GB
Runs:
+Mistral Small 22B Q4 (15.2 GB)
Does not fit:
-Mistral Small 22B Q8 (28.4 GB)
-mistral-small:24b (16.4 GB)
-Mistral Large 2 (75.8 GB)
Same 16 GB VRAM ceiling as the 4060 Ti 16GB, with higher memory bandwidth: 672 GB/s vs 288 GB/s in the speed-ceiling table on this page. Bandwidth sets an upper bound on generation speed; these are spec figures, not measured tokens/sec. Pick it over the 4060 Ti if you want the higher bandwidth ceiling for Mistral Small 22B.
RTX 4090 24GB
Runs:
+Mistral Small 22B Q4 (15.2 GB)
+Mistral Nemo Q8 (16.4 GB)
Does not fit:
-Mistral Small 22B Q8 (28.4 GB)
-Mistral Large 2 Q4 (75.8 GB)
24 GB fits Nemo at Q8 and the 22B model at Q4. It does not fit Small at Q8.
RTX 5090 32GB
Runs:
+Mistral Small 22B Q8 (28.4 GB)
Does not fit:
-Mistral Large 2 Q4 (75.8 GB)
32 GB fits the 22B model at Q8 and does not fit Mistral Large 2.
Mac mini M4 16GB
Runs:
+Mistral 7B Q4 (6.2 GB)
+Mistral Nemo Q4 (9.2 GB), tight after a 4 GB macOS reserve
Does not fit:
-Mistral Small 22B Q4 (15.2 GB)
-Mistral Nemo Q8 (16.4 GB)
Treat usable memory as about 12 GB. mistral-nemo is the ceiling and is tight. mistral-small does not fit.
Mac mini M4 24GB
Runs:
+Mistral Small 22B Q4 (15.2 GB)
+Mistral Nemo Q8 (16.4 GB), tight after the macOS reserve
Does not fit:
-Mistral Small 22B Q8 (28.4 GB)
-mistral-small:24b is tight on 24 GB unified
A 24 GB Mac holds the 22B Q4 estimate. The 24B tag at 16.4 GB is the tight case.
Mac mini M4 Pro 48GB
Runs:
+Mistral Small 22B Q8 (28.4 GB)
Does not fit:
-Mistral Large 2 Q4 (75.8 GB)
48 GB gives Mistral Small 22B Q8 generous headroom. Excellent for high-quality Mistral Small inference. Still cannot run Mistral Large 2 at any usable quantization.
Mac Studio M4 Max 128GB
Runs:
+Mistral Large 2 Q4 (75.8 GB) after a 4 GB macOS reserve
Does not fit:
-Mistral Large 2 Q8 (149.6 GB)
128 GB unified holds the 75.8 GB Q4 estimate. Q8 does not. No new dollar was verified for a 128 GB Studio listing on 11 October, and this guide does not invent an ASIN.
Inference Speed by Hardware
Token generation speed is bottlenecked by memory bandwidth. The table below shows Q4_K_M bandwidth ceilings at low batch size, not measured speeds. Real-world results are lower and vary by driver version, context length, and system load.
Hardware
Bandwidth
7B Q4 tok/s
Nemo 12B tok/s
Small 22B tok/s
RTX 5090 32GB
1,792 GB/s
≤290 t/s
≤195 t/s
≤118 t/s
RTX 4090 24GB
1,008 GB/s
≤163 t/s
≤110 t/s
≤67 t/s
RTX 4070 Ti Super 16GB
672 GB/s
≤109 t/s
≤74 t/s
≤45 t/s
RTX 4080 16GB
716 GB/s
≤116 t/s
≤78 t/s
≤48 t/s
RTX 4060 Ti 16GB
288 GB/s
≤47 t/s
≤32 t/s
≤19 t/s
Intel Arc B580 12GB
456 GB/s
≤74 t/s
≤50 t/s
—
RTX 4060 8GB
272 GB/s
≤44 t/s
—
—
Mac Studio M4 Max 128GB
See Apple specs
n/a
n/a
n/a
Mac mini M4 Pro 48GB
~273 GB/s
≤45 t/s
≤30 t/s
≤18 t/s
Mac mini M4 24GB
~120 GB/s
≤20 t/s
≤14 t/s
≤8 t/s
Speed ceilings: tokens/sec ≤ bandwidth (GB/s) / model size in memory (GB), rounded up. These are upper bounds, not measured speeds. Mac bandwidth figures are approximate. The Mac Studio M4 Max row shows n/a because its bandwidth depends on the chip configuration; check Apple's specs for the exact chip. A dash means the model does not fit at that VRAM tier.
How to Run Mistral Locally
Ollama
ollama run mistral-nemo
Easiest option. One command installs and runs the model. Tags: mistral (7B), mistral-nemo (12B), mistral-small (22B). GPU is auto-detected on NVIDIA, AMD (ROCm), and Apple Silicon. Defaults to Q4_K_M quantization.
LM Studio
Search "Mistral" in Discover
GUI-based model browser and chat interface. Download GGUF quantizations directly within the app. Best for non-technical users. Runs on Windows, Mac, and Linux. Lets you pick Q4, Q8, or other quants from a dropdown.
Hugging Face + llama.cpp
mistralai/Mistral-Nemo-Instruct-2407-GGUF
Download GGUF files from the official mistralai org or community repos (bartowski, unsloth) on Hugging Face. Run with llama.cpp for maximum control over quantization, context length, and GPU layer offloading.
+You want the best reasoning at the 8B parameter count
Which Hardware Should You Buy for Mistral?
Entry level
RTX 4060 8GB
Runs Mistral 7B at Q4 (6.2 GB). It does not fit 7B at Q8 (10.4 GB) or Nemo 12B at Q4 (9.2 GB); both need a 12 GB card. Budget entry for Mistral 7B at Q4 only.
Best value
Intel Arc B580 12GB
Fits mistral-nemo at Q4 (9.2 GB) and does not fit mistral-small (15.2 GB). Verify the Ollama oneAPI path before you buy Arc.
Sweet spot
RTX 4060 Ti 16GB
Fits all three consumer Mistral sizes at Q4: 7B (6.2 GB), Nemo 12B (9.2 GB) and Small 22B (15.2 GB). Small 22B at Q4 leaves about 0.8 GB of the 16 GB, so it is tight and needs a short context. The 24B tag (16.4 GB) does not fit. If Mistral Small 22B at Q4 is your target, 16 GB is the smallest card size that holds it.
Mid-range
AMD RX 7900 XTX 24GB
Runs Mistral Small 22B at Q4 (15.2 GB) with about 8.8 GB to spare. Q8 is 28.4 GB and does not fit 24 GB; it needs a 32 GB card such as the RTX 5090. Works with Ollama via ROCm on Linux.
High end
RTX 4090 24GB
Best single consumer GPU for Mistral 7B–22B. Mistral Small at Q4 runs fast with headroom. 1,008 GB/s bandwidth means fast generation across all sizes. Cannot run Mistral Large 2.
Mac ecosystem
Mac mini M4 24GB
Holds Mistral Small 22B at Q4 (15.2 GB): 24 GB leaves about 8.8 GB before the macOS reserve and about 4.8 GB after a 4 GB reserve. Mistral 7B fits at Q4 and Q8. Nemo 12B fits at Q4, and Q8 (16.4 GB) is tight. For Mistral Large 2 (75.8 GB at Q4), you need a 128 GB Mac. Silent and energy-efficient.
Mistral 7B vs Llama 3.1 8B — which is better for local use?
Both fit an 8 GB GPU at Q4. Mistral 7B is 6.2 GB. Llama 3.1 8B is 6.8 GB. Neither fits a 6 GB card. Mistral 7B at Q8 is 10.4 GB and misses 8 GB. Llama 3.1 8B has the longer context window.
Can an 8GB GPU run Mistral Nemo 12B?
No. 12 × 0.6 + 2 = 9.2 GB, so an 8 GB RTX 4060 does not fit mistral-nemo. Use a 12 GB card, or stay on ollama run mistral. Q8 of Nemo is 16.4 GB and misses 16 GB.
Can I run Mistral on a Mac?
The small tags do. After a 4 GB macOS reserve, a 16 GB Mac has about 12 GB left: mistral at Q4 (6.2 GB) fits, and mistral-nemo at Q4 (9.2 GB) is tight. mistral-small at Q4 is 15.2 GB and needs the 24 GB Mac, which leaves about 8.8 GB before the macOS reserve and about 4.8 GB after it. The 24B tag is 16.4 GB and does not fit 16 GB of unified memory after that reserve. Mistral Large 2 at 123B is 75.8 GB at Q4 and needs a 128 GB Mac. Mistral Large 4 weights were not released on 11 October 2026; the Hugging Face upcoming page said 31 October 2026, so this guide does not size a GPU for it.
What is the best GPU for Mistral 7B?
Mistral 7B at Q4 is 6.2 GB. An 8 GB GPU fits it. A 6 GB GPU does not. Q8 is 10.4 GB and misses 8 GB. The RTX 4060 is the Q4 card for this model.
Mistral Nemo vs Phi-4 — which should I run locally?
Mistral Nemo 12B and Phi-4 14B target similar hardware. Phi-4 has stronger STEM and reasoning benchmarks in its size class. Mistral Nemo has better multilingual support and a longer context window (128K tokens). For coding, math, and science tasks in English, Phi-4 is the better choice. For multilingual tasks or long-context use, Mistral Nemo is the pick. Both run well on the Intel Arc B580 12GB at Q4.
How do I run Mistral with Ollama?
Run ollama run mistral (6.2 GB at Q4), ollama run mistral-nemo (9.2 GB, needs 12 GB), or ollama run mistral-small (22B default, 15.2 GB, fits 16 GB). The 24B tag is 16.4 GB and does not fit 16 GB. An 8 GB GPU should stay on mistral. Those tags were on the Ollama library on 11 October 2026.
Check VRAM requirements for Mistral models, or compare hardware options.
Mistral model sizes on this page are site formula estimates read against the model cards. Formula details and the sitewide source list are on the methodology page. For this guide we used:
Spot a number that does not match the linked source? Contact us and we will update the guide.
We use cookies and personalised ads
This site uses cookies and shows personalised ads via Google AdSense. We and our partners store and access information on your device to serve relevant ads and improve your experience.
You can accept all cookies, decline (non-personalised ads only), or
manage preferences.
See our Privacy Policy.
We use Google Analytics
This site would like to use Google Analytics to understand how visitors use the site. If you accept, Google Analytics will load. If you decline, or if you make no choice, Google Analytics will not load. No ads are shown on this visit.
See our Privacy Policy and Cookie Policy.
Cookie preferences
Choose which cookies you allow. Strictly necessary cookies are always active.
Strictly necessary
Session state, security, and performance. Cannot be disabled.