Best Hardware for Local LLMs by VRAM Tier

Buy the cheapest device whose memory holds the model you want to run: at Q4, 8 GB runs 8B models, 12 GB runs 14B, 16 GB runs up to about 20B, 24 GB runs 27B to 32B, and 64 GB of unified memory runs 70B. Memory bandwidth then sets the speed, so two 16 GB cards that fit the same models can differ by more than 2x: NVIDIA lists 448 GB/s for the RTX 5060 Ti and 960 GB/s for the RTX 5080.

All 31 GPUs, mini PCs, and Apple Silicon devices tracked for local AI inference. Use the VRAM calculator to see which models fit each device, or open the compare table. Kept guides such as the RTX 4090 LLM guide stay on their own URLs.

Which VRAM tier do you need?

Each row joins two numbers that most buying guides keep apart: what fits (our VRAM formula at Q4_K_M, params × 0.5 × 1.2 + 2 GB) and how fast the memory is (vendor spec). On Macs we subtract a 4 GB macOS reserve first. Every device listed here has a card further down this page.

Memory What fits Devices on this page (bandwidth) Best for
8 GB 8B (6.8 GB). Not 14B (10.4 GB). RTX 4060 Trying 7B to 8B chat and coding models. Budget GPU guide
12 GB 14B (10.4 GB). Not 20B (14.0 GB). RTX 5070 (672 GB/s), RTX 4070, Intel Arc B580 A 14B model such as qwen3:14b at short context. 12 GB picks
16 GB 20B (14.0 GB). Not 27B (18.2 GB). RTX 5060 Ti (448 GB/s), RTX 5070 Ti (896 GB/s), RTX 5080 (960 GB/s), RTX 4060 Ti 16 GB, RTX 4070 Ti Super, RTX 4080 14B with room for a long context, or 20B at short context. GPU buying guide
24 GB 27B (18.2 GB) and 32B (21.2 GB). Not 70B (44 GB). RTX 4090 (1,008 GB/s), RTX 3090 (936 GB/s), RX 7900 XTX (960 GB/s). Mac mini M5 Pro 24 GB has about 20 GB usable, so 27B is tight. 27B to 32B models, including Qwen3.8-27B. 24 GB GPU guide
32 GB 32B with headroom for context. Not 70B. RTX 5090 (1,792 GB/s) The fastest single card here for 27B to 32B. RTX 5090 vs 4090
36 to 64 GB unified After the 4 GB reserve: 27B at Q8 on 48 GB (34.4 GB), 70B on 64 GB (44 GB). Mac Studio M5 Max 36 GB (460 GB/s), 48 GB and 64 GB (614 GB/s) One quiet box that runs 70B at Q4. Apple Silicon guide
96 to 128 GB unified 70B at Q8 (86 GB): tight on 96 GB, fits on 128 GB. On 128 GB, the 93.68 GB Qwen3.8-Flash-Next IQ4 file fits. Mac Studio M5 Ultra 96 GB (1.2 TB/s), Mac Studio M5 Max 128 GB (614 GB/s), NVIDIA DGX Spark (273 GB/s), GMKtec EVO-X2 (256 GB/s) Models too big for any single consumer card. DGX Spark vs Mac Studio vs Strix Halo

Worked example: generating each token reads the whole model from memory once, so bandwidth divided by file size is a ceiling on tokens per second (without speculative decoding). The Ollama library lists qwen3:14b at 9.3 GB. On an RTX 5060 Ti that ceiling is 448 / 9.3 = 48 tokens per second; on an RTX 5080 it is 960 / 9.3 = 103. Real runs land below the ceiling, and we have not benchmarked either card. The ceiling still shows why the RTX 5080 can be up to about twice as fast on a model that fits both. Bandwidth for the cards listed without a number is on each card below.

Not sure which model you want? Start with what you can run on your current machine or put the exact model and context length in the VRAM calculator.

NVIDIA GPUs

NVIDIA GeForce RTX 4060 (8GB)
8 GB VRAM ~272 GB/s 115W See the listing

Buy New, only 1 left (2026-10-17).

NVIDIA GeForce RTX 4070 (12GB)
12 GB VRAM ~504 GB/s 200W See the listing

In stock, only 9 left (2026-10-17).

NVIDIA GeForce RTX 4090 (24GB)
24 GB VRAM ~1008 GB/s 450W See the listing

Currently unavailable (2026-10-17).

NVIDIA GeForce RTX 3090 (24GB)
24 GB VRAM ~936 GB/s 350W See the listing

24 GB fits Llama 3.1 8B Q4 (6.8 GB) and Qwen3.6-27B Q4 (18.2 GB). 70B Q4 is 44 GB and does not fit.

Buy New, only 1 left (2026-10-17).

NVIDIA GeForce RTX 5090 (32GB)
32 GB VRAM ~1792 GB/s 575W See the listing

Currently unavailable. See All Buying Options (2026-10-17).

NVIDIA RTX 4080 (16GB)
16 GB VRAM ~716 GB/s 320W See the listing

Only 1 left (2026-10-17).

NVIDIA RTX 4060 Ti (16GB)
16 GB VRAM ~288 GB/s 165W See the listing

Currently unavailable (2026-10-17).

NVIDIA RTX 5080 (16GB)
16 GB VRAM ~960 GB/s 360W See the listing
NVIDIA RTX 5070 (12GB)
12 GB VRAM ~672 GB/s 250W See the listing

In stock, sold by Amazon.com (2026-10-17).

NVIDIA RTX 5070 Ti (16GB)
16 GB VRAM ~896 GB/s 300W See the listing

Currently unavailable (2026-11-04). We don't know when or if this item will be back in stock.

NVIDIA RTX 4070 Ti Super (16GB)
16 GB VRAM ~672 GB/s 285W See the listing

Buy Used from Hcir Equipment, 8 left (2026-10-17).

NVIDIA DGX Spark
128 GB VRAM ~273 GB/s 170W See the listing

Currently unavailable (2026-10-17).

PNY GeForce RTX 5060 Ti (16GB)
16 GB VRAM ~448 GB/s 180W See the listing

Currently unavailable (2026-10-17). No new price.

AMD GPUs

AMD Radeon RX 7900 XTX (24GB)
24 GB VRAM ~960 GB/s 355W See the listing

Currently unavailable (2026-10-17).

Intel GPUs

Intel Arc B580 (12GB)
12 GB VRAM ~456 GB/s 190W See the listing

Stock not confirmed on Amazon (2026-10-17).

Apple Silicon

Apple Mac mini (M6, 16GB)
16 GB unified ~153 GB/s See the listing

After a 4 GB macOS reserve, about 12 GB is usable. 7B Q4 is 6.2 GB and fits. 8B Q4 is 6.8 GB and fits. Gemma 3 12B Q4 is 9.2 GB and fits with little room. 14B Q4 is 10.4 GB and is tight. 27B Q4 is 18.2 GB and does not fit.

In stock, sold by Amazon.com (2026-10-18).

Apple Mac mini (M6, 24GB)
24 GB unified ~170 GB/s See the listing

After a 4 GB macOS reserve, about 20 GB is usable. 14B Q4 is 10.4 GB and fits. 27B Q4 is 18.2 GB and is tight. 32B Q4 is 21.2 GB and does not fit.

Sold by Amazon.com. Usually ships within 3 to 5 weeks (2026-10-18).

Apple Mac Studio (M5 Max, 36GB)
36 GB unified ~460 GB/s See the listing

After a 4 GB macOS reserve, about 32 GB is usable. Qwen3.6-27B Q4 is 18.2 GB and fits. LFM2.5-24B-A2B Q4 is 16.4 GB and fits. Q8 is 30.8 GB and is the edge. 70B Q4 is 44 GB and does not fit.

In stock, Amazon.com (2026-10-17).

Apple Mac mini (M5 Pro, 24GB)
24 GB unified ~307 GB/s See the listing

After a 4 GB macOS reserve, about 20 GB is usable. 27B Q4 is 18.2 GB and is tight. LFM2.5-24B-A2B Q4 is 16.4 GB and is tight. Q8 is 30.8 GB and does not fit.

In stock, Amazon.com (2026-10-17).

Apple Mac mini (M4, 16GB)
16 GB unified ~120 GB/s 25W See the listing

After a 4 GB macOS reserve, about 12 GB is usable. 7B Q4 is 6.2 GB and fits. 14B Q4 is 10.4 GB and is tight. 27B Q4 is 18.2 GB and does not fit.

No live price shown; Amazon showed a High price badge (2026-10-01)

Apple Mac mini (M4, 24GB)
24 GB unified ~120 GB/s 25W See the listing

After a 4 GB macOS reserve, about 20 GB is usable. 14B Q4 is 10.4 GB and fits. 27B Q4 is 18.2 GB and is tight. 32B Q4 is 21.2 GB and does not fit.

No live price shown; Amazon showed a High price badge (2026-10-01)

Apple Mac mini (M4 Pro, 24GB)
24 GB unified ~273 GB/s 35W See the listing

After a 4 GB macOS reserve, about 20 GB is usable. 27B Q4 is 18.2 GB and is tight. LFM2.5-24B-A2B Q4 is 16.4 GB and is tight. Q8 is 30.8 GB and does not fit.

New stock not confirmed on Amazon (2026-10-11).

Apple Mac mini (M4 Pro, 48GB)
48 GB unified ~273 GB/s 40W See the listing

After a 4 GB macOS reserve, about 44 GB is usable. 27B Q8 is 34.4 GB and fits. 70B Q4 is 44 GB and is the edge. 70B Q8 is 86 GB and does not fit.

Currently unavailable (2026-10-01)

Apple Mac Studio (M4 Max, 64GB)
64 GB unified ~546 GB/s 85W See the listing

After a 4 GB macOS reserve, about 60 GB is usable. 70B Q4 is 44 GB and fits. 27B FP16 is 66.8 GB and does not fit. 70B Q8 is 86 GB and does not fit.

Currently unavailable (2026-10-01)

Apple Mac Studio (M4 Max, 128GB)
128 GB unified ~546 GB/s 85W See the listing

After a 4 GB macOS reserve, about 124 GB is usable. 70B Q8 is 86 GB and fits. Qwen3.8-Flash-Next UD-Q4_K_XL is 111.33 GB and is tight.

Price not rechecked since 2026-10-01. Configure 64GB+ at Apple.

Apple Mac Studio (M5 Max, 48GB)
48 GB unified ~614 GB/s See the listing

After a 4 GB macOS reserve, about 44 GB is usable. 27B Q8 is 34.4 GB and fits. 70B Q4 is 44 GB and is the edge. 70B Q8 is 86 GB and does not fit.

Apple Mac Studio (M5 Max, 64GB)
64 GB unified ~614 GB/s See the listing

After a 4 GB macOS reserve, about 60 GB is usable. 70B Q4 is 44 GB and fits. 27B FP16 is 66.8 GB and does not fit. 70B Q8 is 86 GB and does not fit.

Apple Mac Studio (M5 Max, 128GB)
128 GB unified ~614 GB/s See the listing

After a 4 GB macOS reserve, about 124 GB is usable. 70B Q8 is 86 GB and fits. Qwen3.8-Flash-Next UD-Q4_K_XL is 111.33 GB and is tight.

Apple Mac Studio (M5 Ultra, 96GB)
96 GB unified ~1200 GB/s See the listing

After a 4 GB macOS reserve, about 92 GB is usable. 70B Q4 is 44 GB and fits. 70B Q8 is 86 GB and is tight. Nemotron 3.5 Bolt at Q4 is 46.4 GB and fits. Qwen3.8-Flash-Next UD-Q4_K_XL is 111.33 GB and does not fit.

Mini PCs

GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB)
128 GB unified ~256 GB/s 140W See the listing

128 GB at ~256 GB/s. Dense 70B Q4 (44 GB) fits and is slow. Qwen3.8-Flash-Next UD-IQ4_XS is 93.68 GB and fits. UD-Q4_K_XL is 111.33 GB and is tight. DeepSeek-V4-Flash only at an aggressive quant (~155 GB at a normal 4-bit). Not CUDA.

In stock (2026-10-17).

Beelink GTR9 Pro (128GB, sold out)
128 GB unified ~256 GB/s See the listing

128 GB at ~256 GB/s when the listing is this GTR9 Pro. UD-IQ4_XS (93.68 GB) fits. UD-Q4_K_XL (111.33 GB) is tight. Sold out on 2026-10-11. Not CUDA.

Browse All Models All Guides Compare GPUs VRAM Calculator

Sources & last reviewed

Last reviewed by the LLM Hardware team. The date changes only when the numbers or sources on this page change.

Vendor spec pages (VRAM / unified memory)

Memory bandwidth used in the tier table

Model cards used for the fit tiers

VRAM estimates use params × bytes per parameter × 1.2 + 2 GB (Q4_K_M ≈ 0.5 bytes), explained on the methodology page. Found a mismatch? Tell us on the contact page.