How Recommendations Work

Last updated: November 14, 2026 by the LLM Hardware team

Data sources

The site’s model list was imported from the Hugging Face Hub API: the top models by download count across the major pipeline types (text generation, code generation, multimodal, image generation, video generation), plus the latest models from the most active open-source providers at the time. The automated daily refresh has been off since 2 May 2026, and the list was last changed on 29 May 2026, so models released after that date are not in it. Guides cover newer models from their official model cards.

Hardware specs are taken from manufacturer spec sheets and updated in the weekly maintenance run. Hardware pages show no price. Manufacturer launch list-price (MSRP) tables appear only on five guides, each with a cited source and date: Best GPU for Running LLMs Locally, Best Budget GPU for LLMs, Best 24GB VRAM GPU for LLMs, Best Local LLM for Coding, and the RTX 4090 LLM guide. They do not show an Amazon price. Each product link sends you to Amazon for the live price.

VRAM calculation

Required VRAM is estimated from the model's parameter count, the quantization format, and a runtime overhead factor. The formula we use is:

VRAM (GB) = parameters_billions × bytes_per_param × overhead_factor + os_reserve

Where:

  • parameters_billions: total parameter count in billions (e.g. 7 for a 7B model).
  • bytes_per_param: bytes per weight at the chosen quantization (table below).
  • overhead_factor: 1.2 to cover KV cache, activations, and framework overhead at modest (4K to 8K) context lengths.
  • os_reserve: 2 GB held back for the OS and the inference engine itself (llama.cpp, Ollama, vLLM, LM Studio).

bytes_per_param by quantization:

Quantization Bytes/param Notes
Q4_K_M 0.5 4-bit GGUF, the common quality/size sweet spot
Q5_K_M 0.625 5-bit GGUF, modestly better quality than Q4
Q8 1.0 8-bit, near-lossless against FP16
FP16 2.0 Full 16-bit precision

Worked example: Llama 3 8B at Q4_K_M

VRAM = 8 × 0.5 × 1.2 + 2
     = 4.8 + 2
     = 6.8 GB

That matches what an 8B Q4_K_M GGUF actually consumes on a single GPU in llama.cpp with a 4K context window, cross-checked against the runs in the XiongjieDai repo cited below.

KV cache scales with context length

The 1.2 overhead factor is only honest at short contexts. The KV cache itself scales linearly with context length:

kv_bytes_per_token = 2 × layers × kv_heads × head_dim × bytes_per_element

The leading 2 covers separate keys and values. bytes_per_element is 2 for FP16/BF16 KV cache, 1 for INT8 KV-cache quantization. For a Llama 3 8B model (32 layers, 8 KV heads, head_dim 128, FP16 KV cache) that works out to roughly 131 KB per token, so a 32K context adds about 4 GB on top of the weights. For long-context workloads we use that figure directly instead of the overhead factor.

Estimate versus published download size

This check compares two planning numbers from the formula above with the file size Ollama lists for a named tag. It is an arithmetic check against a published download, checked 14 November 2026. It is not a test we ran on a GPU, and it is not a speed or quality result.

What it can show: the weights term (parameters x bytes per parameter) sits in the same neighborhood as the published Q4_K_M, Q8, and FP16 downloads in this table. On these Q4_K_M rows the listed file is about 22% to 37% larger than the 0.5-bytes-per-parameter term (the 0.6B row is further off, as a small file plus format overhead). Q8 and FP16 rows are closer. The full VRAM figure is larger still, because the 1.2 factor and the 2 GB reserve are runtime allowances, not part of the file. For several mid-size Q4_K_M tags the full VRAM figure lands near the listed download plus a little headroom, which is a side observation, not proof that 1.2 was designed to match files.

What it does not show: runtime memory once the model is loaded, KV cache growth, tokens per second, or quality. Mixture-of-experts rows are sized on total parameters (Qwen3-30B-A3B is 30.5B total, 3.3B activated, per the model card), not on the active count. Tag sizes can change; the access date is the check date.

Ollama tag Quant Params Weights term VRAM formula Published size Diff vs weights
qwen3:0.6b-q4_K_MQ4_K_M0.6B0.30 GB2.36 GB523 MB+74.3%
qwen3:4b-q4_K_MQ4_K_M4B2.00 GB4.40 GB2.6 GB+30.0%
qwen3:8b-q4_K_MQ4_K_M8B4.00 GB6.80 GB5.2 GB+30.0%
qwen3:14b-q4_K_MQ4_K_M14B7.00 GB10.40 GB9.3 GB+32.9%
qwen3:32b-q4_K_MQ4_K_M32B16.00 GB21.20 GB20 GB+25.0%
qwen3:8b-q8_0Q8_08B8.00 GB11.60 GB8.9 GB+11.3%
qwen3:32b-q8_0Q8_032B32.00 GB40.40 GB35 GB+9.4%
qwen3:30b-a3b-q4_K_MQ4_K_M30.5B total15.25 GB20.30 GB19 GB+24.6%
llama3.1:8b-instruct-q4_K_MQ4_K_M8B4.00 GB6.80 GB4.9 GB+22.5%
llama3.1:8b-instruct-q8_0Q8_08B8.00 GB11.60 GB8.5 GB+6.2%
llama3.1:70b-instruct-q4_K_MQ4_K_M70B35.00 GB44.00 GB43 GB+22.9%
llama3.1:70b-instruct-fp16FP1670B140.00 GB170.00 GB141 GB+0.7%
gemma3:12b-it-q4_K_MQ4_K_M12B (tag name)6.00 GB9.20 GB8.2 GB+36.7%
gemma3:27b-it-q4_K_MQ4_K_M27B (tag name)13.50 GB18.20 GB17 GB+25.9%

Weights term = params x bytes/param. VRAM formula = params x bytes/param x 1.2 + 2. Diff = (published decimal GB - weights term) / weights term. Ollama tag pages checked 14 November 2026. The two Gemma rows use the 12B and 27B figures from the tag names; their Hugging Face README URLs returned HTTP 401 on that date, so a config.json parameter count was not confirmed.

Hardware tiers

Each model gets three hardware recommendations:

Minimum

The smallest GPU or chip that can load the model in Q4_K_M quantization at a 4K context window. Expect slower inference and limited context.

Comfortable

Fits the model in Q5_K_M or Q8 quantization with an 8K to 16K context. Suitable for everyday use and longer conversations.

Headroom

Enough VRAM for FP16 or Q8 at 32K+ context, or for the next generation of models as they grow in parameter count.

Hardware is matched to tiers by comparing the estimated VRAM requirement at each quantization level against the GPU or unified-memory capacity. For each tier, the matcher selects the hardware with the smallest qualifying VRAM (the lowest VRAM that still meets that tier's threshold). For GGUF-tagged models, it prefers an Apple Silicon row when one qualifies.

Where our numbers come from

This site does not measure tokens-per-second or wattage. Some pages, including the tokens-per-second guide and Can I Run, show a computed bandwidth ceiling (memory bandwidth divided by the Q4_K_M weight size). That ceiling is not a measurement. Where a page quotes a published run, it should come from one of the open community benchmarks listed here. "Fits on this card" claims come from the site VRAM formula.

llama.cpp llama-bench discussion (Apple Silicon results)

Apple Silicon tokens-per-second baselines for M-series chips, published by the llama.cpp project itself.

XiongjieDai/GPU-Benchmarks-on-LLM-Inference

Community llama-bench results across NVIDIA consumer + datacenter cards and Apple Silicon, with model, quant, and context length recorded per run.

Home GPU LLM Leaderboard

VRAM-tier-organised open leaderboard with tokens/sec per GPU; used to cross-check our GPU tier ranking.

Hardware-Corner GPU ranking for local LLM

Consumer + pro GPU tokens/sec at multiple context lengths; used to spot-check our context-length sensitivity claims.

Modal — How much VRAM do I need for LLM inference?

Independent VRAM-formula breakdown (weights + KV cache + activation + framework overhead); used to anchor our formula and overhead factor.

Hugging Face Hub API

Model parameter counts, tags, and downloads, bulk-imported for the site model list. The automated refresh has been off since 2 May 2026.

Parameter detection

Parameter counts are extracted from the model ID and tags using regex patterns that match common naming conventions (for example 7B, 70b, 0.5B). When a model does not encode its size in its name or tags, no VRAM estimate is shown.

What we do NOT claim

  • We do not run a benchmark rig, and this site does not measure hardware. A speed figure should name its public run (model, quant, hardware and publisher) or be labelled as a bandwidth ceiling. Some older guides still show approximate speed figures with no named run; treat those as rough and unverified. Memory-fit guidance uses the site VRAM formula.
  • We do not publish measured tokens-per-second numbers. A speed figure on this site is either a named public run or a labelled bandwidth ceiling. Pages are written with AI assistance and reviewed before publishing.
  • We do not run controlled studies on model output quality. For evals we link to the community benchmarks that do.
  • We are not a hardware vendor and we do not receive sample hardware. Affiliate links on this site are disclosed; they do not change which hardware gets recommended.
  • If a number on this site looks wrong, please use the contact page with the page URL.

Cross-checks and corrections

Memory-fit numbers come from the site VRAM formula, and not every speed figure is cited yet. Reader-flagged errors are logged on the corrections page and the original guide is updated with a dated note. Send corrections through the contact page with the page URL and the source you would prefer we cite.

Limitations

  • Estimates assume a single GPU or unified memory pool. Multi-GPU setups, CPU offloading, and model parallelism are not modeled.
  • Mixture-of-Experts (MoE) models (DeepSeek, Mixtral) activate only a fraction of parameters per token, but all experts stay resident in memory, so our formula uses the total parameter count. Offloading experts to system RAM is not modeled.
  • Apple Silicon unified memory is shared between CPU and GPU. Available VRAM depends on system RAM allocation and other running applications.
  • Context length significantly affects KV cache size. The 1.2 overhead factor is calibrated for 4K to 8K context; 32K+ context needs the explicit KV-cache formula above.
  • Quantization quality varies by implementation. GGUF Q4_K_M and GPTQ Q4 are not equivalent even at the same bit width.

Disclaimer

All recommendations are for informational purposes only. Verify requirements before purchasing. See our full disclaimer.