What Hardware Do You Need to Run Mistral Locally?

By LLM Hardware team · Methodology ·

AI pulled the Mistral / Mixtral size table. The MoE VRAM nuance and the per-size hardware picks were edited against the Mistral docs and the site’s VRAM formula.

Key facts

7B class
Fits 8 GB at Q4
Mixtral-class MoE
All experts must reside; active params only change speed
Checked
2026-10-13
Author
LLM Hardware team

Checked 2026-10-13. Author: LLM Hardware team. Amazon prices are not stored or shown on this site.

Citeable facts

Key fact: Mistral 7B at Q4 needs 6.2 GB and fits 8 GB. Nemo 12B at Q4 needs 9.2 GB and misses 8 GB. Small 22B at Q4 needs 15.2 GB and fits 16 GB. The 24B tag needs 16.4 GB and misses 16 GB.

How we got it: 7 × 0.6 + 2 = 6.2. 12 × 0.6 + 2 = 9.2. 22 × 0.6 + 2 = 15.2. 24 × 0.6 + 2 = 16.4. Ollama library on 11 October 2026 listed mistral, mistral-nemo, and mistral-small at 22B and 24B.

Checked:

Key fact: Mistral Large 4 was not a downloadable weight on 11 October 2026. This page does not size a GPU or add a buy link for it.

How we got it: Hugging Face mistralai/Mistral-Large-4.0-1T05-A52B upcoming page listed weights for 31 October 2026.

Checked:

Methodology explains the VRAM formula, the bytes per weight at each quantization, and the full source list.

Updated 9 November 2026 · Mistral 7B to 22B · VRAM requirements · Consumer GPU guide · Ollama setup

Mistral 7B at Q4 is 7 × 0.6 + 2 = 6.2 GB, so it fits an 8 GB GPU and does not fit a 6 GB GPU. Mistral Nemo 12B at Q4 is 9.2 GB and needs 12 GB. Mistral Small 22B at Q4 is 15.2 GB and needs 16 GB. The commands are ollama run mistral, ollama run mistral-nemo, and ollama run mistral-small. In LM Studio, pick the Q4_K_M GGUF of the same size.

What is Mistral AI?

Mistral AI is a French AI company founded in 2023 that releases open-weight models known for punching above their weight class. Mistral 7B was the first model to significantly challenge Llama 2 at the same parameter count.

The Gigabyte RTX 4070 12 GB (B0BZHCQ6PF) fits mistral at Q4 (6.2 GB) and mistral-nemo at Q4 (9.2 GB). It does not fit mistral-small (15.2 GB).

Gigabyte RTX 4070 12 GB on Amazon

Mistral VRAM Requirements by Model Size

VRAM is estimated using params × bytes_per_param × 1.2 + 2. Q4 is params_B × 0.6 + 2. Q8 is params_B × 1.2 + 2. FP16 is params_B × 2.4 + 2. Add extra headroom for KV cache with long context windows.

ModelParamsQ4_K_M VRAMQ8 VRAMFP16 VRAMMin GPU
Mistral 7B v0.3 7B 6.2 GB 10.4 GB 18.8 GB 8 GB GPU
Mistral Nemo 12B 12B 9.2 GB 16.4 GB 30.8 GB 12 GB GPU
Mistral Small 22B 22B 15.2 GB 28.4 GB 54.8 GB 16 GB GPU
Mistral Large 2 123B 75.8 GB 149.6 GB Does not fit 128 GB 128 GB unified

Mistral 7B at Q4 is 6.2 GB, so it fits an 8 GB GPU and does not fit a 6 GB GPU. Use the VRAM Calculator for context-length-adjusted estimates.

Which Quantization Should You Use?

Quantization trades a small amount of output quality for a large reduction in VRAM usage. For most users, Q4_K_M is the right default.

Q4_K_M — recommended

Weights are about a quarter the size of FP16 weights, and total VRAM falls by less: Mistral 7B is 6.2 GB at Q4 vs 18.8 GB at FP16. Quality loss is usually small. The Ollama default for all Mistral models. Start here.

Q8_0 — best quality

Approximately doubles the weight size vs Q4 but preserves near-FP16 output quality. Mistral 7B at Q8 is 7 × 1.2 + 2 = 10.4 GB, so it misses an 8 GB GPU such as the RTX 4060 and needs a 12 GB card. Use Q8 when you have extra VRAM and want maximum output fidelity.

FP16 — reference only

Mistral 7B at FP16 is 7 × 2.4 + 2 = 18.8 GB, so it needs 24 GB. It does not fit a 16 GB GPU. Daily use should stay on the Q4 file.

Q2/Q3 — constrained devices

Aggressive quantizations for very limited hardware. Quality degrades noticeably — particularly on instruction following and code. Only consider if your device cannot fit Q4. Mistral 7B at Q2 can run on 2–3 GB but output quality suffers.

What Mistral Models Can You Run on Your GPU?

Find your GPU or Mac below. Each card shows which Mistral models fit, and what does not.

RTX 4060 8GB

Runs:

  • +Mistral 7B Q4 (6.2 GB)

Does not fit:

  • -Mistral 7B Q8 (10.4 GB)
  • -Mistral Nemo Q4 (9.2 GB)
  • -Mistral Small 22B Q4 (15.2 GB)

An 8 GB card fits mistral at Q4. It does not fit mistral-nemo or mistral-small.

Intel Arc B580 12GB

Runs:

  • +Mistral 7B Q4 (6.2 GB) and Q8 (10.4 GB)
  • +Mistral Nemo Q4 (9.2 GB)

Does not fit:

  • -Mistral Nemo Q8 (16.4 GB)
  • -Mistral Small 22B Q4 (15.2 GB)

12 GB is the Nemo Q4 tier. It does not run mistral-small.

RTX 4060 Ti 16GB

Runs:

  • +Mistral Nemo Q4 (9.2 GB)
  • +Mistral Small 22B Q4 (15.2 GB)

Does not fit:

  • -Mistral Nemo Q8 (16.4 GB)
  • -Mistral Small 22B Q8 (28.4 GB)
  • -mistral-small:24b (16.4 GB)

16 GB fits the 22B Q4 estimate and does not fit the 24B tag or Nemo at Q8.

RTX 4070 12GB

Runs:

  • +Mistral 7B Q8 (10.4 GB)
  • +Mistral Nemo Q4 (9.2 GB)

Does not fit:

  • -Mistral Small 22B Q4 (15.2 GB)
  • -Mistral Nemo Q8 (16.4 GB)

12 GB fits mistral and mistral-nemo at Q4. It does not fit mistral-small (15.2 GB).

RTX 4070 Ti Super 16GB

Runs:

  • +Mistral Small 22B Q4 (15.2 GB)

Does not fit:

  • -Mistral Small 22B Q8 (28.4 GB)
  • -mistral-small:24b (16.4 GB)
  • -Mistral Large 2 (75.8 GB)

Same 16 GB VRAM ceiling as the 4060 Ti 16GB, with higher memory bandwidth: 672 GB/s vs 288 GB/s in the speed-ceiling table on this page. Bandwidth sets an upper bound on generation speed; these are spec figures, not measured tokens/sec. Pick it over the 4060 Ti if you want the higher bandwidth ceiling for Mistral Small 22B.

RTX 4090 24GB

Runs:

  • +Mistral Small 22B Q4 (15.2 GB)
  • +Mistral Nemo Q8 (16.4 GB)

Does not fit:

  • -Mistral Small 22B Q8 (28.4 GB)
  • -Mistral Large 2 Q4 (75.8 GB)

24 GB fits Nemo at Q8 and the 22B model at Q4. It does not fit Small at Q8.

RTX 5090 32GB

Runs:

  • +Mistral Small 22B Q8 (28.4 GB)

Does not fit:

  • -Mistral Large 2 Q4 (75.8 GB)

32 GB fits the 22B model at Q8 and does not fit Mistral Large 2.

Mac mini M4 16GB

Runs:

  • +Mistral 7B Q4 (6.2 GB)
  • +Mistral Nemo Q4 (9.2 GB), tight after a 4 GB macOS reserve

Does not fit:

  • -Mistral Small 22B Q4 (15.2 GB)
  • -Mistral Nemo Q8 (16.4 GB)

Treat usable memory as about 12 GB. mistral-nemo is the ceiling and is tight. mistral-small does not fit.

Mac mini M4 24GB

Runs:

  • +Mistral Small 22B Q4 (15.2 GB)
  • +Mistral Nemo Q8 (16.4 GB), tight after the macOS reserve

Does not fit:

  • -Mistral Small 22B Q8 (28.4 GB)
  • -mistral-small:24b is tight on 24 GB unified

A 24 GB Mac holds the 22B Q4 estimate. The 24B tag at 16.4 GB is the tight case.

Mac mini M4 Pro 48GB

Runs:

  • +Mistral Small 22B Q8 (28.4 GB)

Does not fit:

  • -Mistral Large 2 Q4 (75.8 GB)

48 GB gives Mistral Small 22B Q8 generous headroom. Excellent for high-quality Mistral Small inference. Still cannot run Mistral Large 2 at any usable quantization.

Mac Studio M4 Max 128GB

Runs:

  • +Mistral Large 2 Q4 (75.8 GB) after a 4 GB macOS reserve

Does not fit:

  • -Mistral Large 2 Q8 (149.6 GB)

128 GB unified holds the 75.8 GB Q4 estimate. Q8 does not. No new dollar was verified for a 128 GB Studio listing on 11 October, and this guide does not invent an ASIN.

Inference Speed by Hardware

Token generation speed is bottlenecked by memory bandwidth. The table below shows Q4_K_M bandwidth ceilings at low batch size, not measured speeds. Real-world results are lower and vary by driver version, context length, and system load.

HardwareBandwidth7B Q4 tok/sNemo 12B tok/sSmall 22B tok/s
RTX 5090 32GB 1,792 GB/s ≤290 t/s ≤195 t/s ≤118 t/s
RTX 4090 24GB 1,008 GB/s ≤163 t/s ≤110 t/s ≤67 t/s
RTX 4070 Ti Super 16GB 672 GB/s ≤109 t/s ≤74 t/s ≤45 t/s
RTX 4080 16GB 716 GB/s ≤116 t/s ≤78 t/s ≤48 t/s
RTX 4060 Ti 16GB 288 GB/s ≤47 t/s ≤32 t/s ≤19 t/s
Intel Arc B580 12GB 456 GB/s ≤74 t/s ≤50 t/s —
RTX 4060 8GB 272 GB/s ≤44 t/s — —
Mac Studio M4 Max 128GB See Apple specs n/a n/a n/a
Mac mini M4 Pro 48GB ~273 GB/s ≤45 t/s ≤30 t/s ≤18 t/s
Mac mini M4 24GB ~120 GB/s ≤20 t/s ≤14 t/s ≤8 t/s

Speed ceilings: tokens/sec ≤ bandwidth (GB/s) / model size in memory (GB), rounded up. These are upper bounds, not measured speeds. Mac bandwidth figures are approximate. The Mac Studio M4 Max row shows n/a because its bandwidth depends on the chip configuration; check Apple's specs for the exact chip. A dash means the model does not fit at that VRAM tier.

How to Run Mistral Locally

Ollama

ollama run mistral-nemo

Easiest option. One command installs and runs the model. Tags: mistral (7B), mistral-nemo (12B), mistral-small (22B). GPU is auto-detected on NVIDIA, AMD (ROCm), and Apple Silicon. Defaults to Q4_K_M quantization.

LM Studio

Search "Mistral" in Discover

GUI-based model browser and chat interface. Download GGUF quantizations directly within the app. Best for non-technical users. Runs on Windows, Mac, and Linux. Lets you pick Q4, Q8, or other quants from a dropdown.

Hugging Face + llama.cpp

mistralai/Mistral-Nemo-Instruct-2407-GGUF

Download GGUF files from the official mistralai org or community repos (bartowski, unsloth) on Hugging Face. Run with llama.cpp for maximum control over quantization, context length, and GPU layer offloading.

For step-by-step installation instructions, see the how to run LLMs locally guide. For a comparison of Ollama vs LM Studio, see the Ollama vs LM Studio guide.

Mistral 7B vs Llama 3.1 8B vs Qwen3 8B

All three are strong open-weight 7–8B models that run on an 8 GB GPU. Here is how they compare for local use:

Mistral 7BLlama 3.1 8BQwen3 8B
VRAM (Q4) 6.2 GB 6.8 GB 6.8 GB
Coding Good Good Excellent
Multilingual Excellent Good Excellent
Speed (RTX 4060) n/a n/a n/a
Context window 32K 128K 32K
Best use case Multilingual, fast general use Long context, general Coding, reasoning

Choose Mistral 7B if...

  • +You need multilingual support (French, Spanish, German, etc.)
  • +You want the 7B Q4 file on an 8 GB GPU
  • +You do not have a 6 GB card: 6.2 GB does not fit there
  • +You want a widely used open model

Choose Llama 3.1 8B if...

  • +You work with long documents (128K context)
  • +You want the strongest overall benchmarks at 8B
  • +You want the broadest fine-tune and tool ecosystem
  • +You need Meta's official weights with commercial-friendly license

Choose Qwen3 8B if...

  • +You primarily use the model for coding tasks
  • +You want built-in chain-of-thought reasoning (thinking mode)
  • +You work in Chinese alongside English
  • +You want the best reasoning at the 8B parameter count

Which Hardware Should You Buy for Mistral?

Entry level

RTX 4060 8GB

Runs Mistral 7B at Q4 (6.2 GB). It does not fit 7B at Q8 (10.4 GB) or Nemo 12B at Q4 (9.2 GB); both need a 12 GB card. Budget entry for Mistral 7B at Q4 only.

Best value

Intel Arc B580 12GB

Fits mistral-nemo at Q4 (9.2 GB) and does not fit mistral-small (15.2 GB). Verify the Ollama oneAPI path before you buy Arc.

Sweet spot

RTX 4060 Ti 16GB

Fits all three consumer Mistral sizes at Q4: 7B (6.2 GB), Nemo 12B (9.2 GB) and Small 22B (15.2 GB). Small 22B at Q4 leaves about 0.8 GB of the 16 GB, so it is tight and needs a short context. The 24B tag (16.4 GB) does not fit. If Mistral Small 22B at Q4 is your target, 16 GB is the smallest card size that holds it.

Mid-range

AMD RX 7900 XTX 24GB

Runs Mistral Small 22B at Q4 (15.2 GB) with about 8.8 GB to spare. Q8 is 28.4 GB and does not fit 24 GB; it needs a 32 GB card such as the RTX 5090. Works with Ollama via ROCm on Linux.

High end

RTX 4090 24GB

Best single consumer GPU for Mistral 7B–22B. Mistral Small at Q4 runs fast with headroom. 1,008 GB/s bandwidth means fast generation across all sizes. Cannot run Mistral Large 2.

Mac ecosystem

Mac mini M4 24GB

Holds Mistral Small 22B at Q4 (15.2 GB): 24 GB leaves about 8.8 GB before the macOS reserve and about 4.8 GB after a 4 GB reserve. Mistral 7B fits at Q4 and Q8. Nemo 12B fits at Q4, and Q8 (16.4 GB) is tight. For Mistral Large 2 (75.8 GB at Q4), you need a 128 GB Mac. Silent and energy-efficient.

For a full cross-budget GPU comparison, see the best GPU for LLMs guide.

Frequently Asked Questions

Mistral 7B vs Llama 3.1 8B — which is better for local use?

Both fit an 8 GB GPU at Q4. Mistral 7B is 6.2 GB. Llama 3.1 8B is 6.8 GB. Neither fits a 6 GB card. Mistral 7B at Q8 is 10.4 GB and misses 8 GB. Llama 3.1 8B has the longer context window.

Can an 8GB GPU run Mistral Nemo 12B?

No. 12 × 0.6 + 2 = 9.2 GB, so an 8 GB RTX 4060 does not fit mistral-nemo. Use a 12 GB card, or stay on ollama run mistral. Q8 of Nemo is 16.4 GB and misses 16 GB.

Can I run Mistral on a Mac?

The small tags do. After a 4 GB macOS reserve, a 16 GB Mac has about 12 GB left: mistral at Q4 (6.2 GB) fits, and mistral-nemo at Q4 (9.2 GB) is tight. mistral-small at Q4 is 15.2 GB and needs the 24 GB Mac, which leaves about 8.8 GB before the macOS reserve and about 4.8 GB after it. The 24B tag is 16.4 GB and does not fit 16 GB of unified memory after that reserve. Mistral Large 2 at 123B is 75.8 GB at Q4 and needs a 128 GB Mac. Mistral Large 4 weights were not released on 11 October 2026; the Hugging Face upcoming page said 31 October 2026, so this guide does not size a GPU for it.

What is the best GPU for Mistral 7B?

Mistral 7B at Q4 is 6.2 GB. An 8 GB GPU fits it. A 6 GB GPU does not. Q8 is 10.4 GB and misses 8 GB. The RTX 4060 is the Q4 card for this model.

Mistral Nemo vs Phi-4 — which should I run locally?

Mistral Nemo 12B and Phi-4 14B target similar hardware. Phi-4 has stronger STEM and reasoning benchmarks in its size class. Mistral Nemo has better multilingual support and a longer context window (128K tokens). For coding, math, and science tasks in English, Phi-4 is the better choice. For multilingual tasks or long-context use, Mistral Nemo is the pick. Both run well on the Intel Arc B580 12GB at Q4.

How do I run Mistral with Ollama?

Run ollama run mistral (6.2 GB at Q4), ollama run mistral-nemo (9.2 GB, needs 12 GB), or ollama run mistral-small (22B default, 15.2 GB, fits 16 GB). The 24B tag is 16.4 GB and does not fit 16 GB. An 8 GB GPU should stay on mistral. Those tags were on the Ollama library on 11 October 2026.

Check VRAM requirements for Mistral models, or compare hardware options.

Sources & methodology

Mistral model sizes on this page are site formula estimates read against the model cards. Formula details and the sitewide source list are on the methodology page. For this guide we used:

Spot a number that does not match the linked source? Contact us and we will update the guide.