Find Hardware That
Runs Any LLM Locally

New to Local AI? Start Here

Run AI like ChatGPT on your own PC — free, private, offline. Ollama in one click, first model running in 10 minutes. Complete beginner guide.

LFM2.5-8B-A1B Hardware Requirements

Liquid AI 8.3B MoE with 1.5B active params and 128K context. Q4 fits in ~6 GB; runs on an RTX 4060 8GB and any 12 GB+ card comfortably. Day-one MLX support.

DeepSeek Hardware Requirements

What GPU to run DeepSeek R1 7B, 14B, 32B, and 70B locally — per-GPU compatibility table.

Qwen3 Hardware Requirements

Run Qwen3 4B–32B locally on any GPU. VRAM requirements, quantization tips, and model picks.

Qwen3.5 Hardware Requirements

Qwen3.5-27B needs 15 GB at Q4 (RTX 4090). 35B-A3B MoE needs 20 GB. Thinking mode overhead and Ollama setup.

Qwen3.6 Hardware Requirements

Qwen3.6 improves on 3.5 with identical VRAM needs. 27B on RTX 4090, 35B-A3B on Mac Studio M4 Max 64 GB.

Kimi K2 Hardware Requirements

Kimi K2 is a 1T MoE model — it cannot run on consumer hardware. What CAN you run: Kimi-VL-A3B, distilled 7B–32B variants.

CPU Offloading for LLMs

Run models too large for your VRAM using -ngl layer splitting. Speed expectations, Ollama setup, and when to skip it.

Mistral Large Hardware Requirements

Mistral Large 2 (123B) needs 70 GB VRAM. Runs on Mac Studio M4 Max 128 GB. Best consumer alternative: Mistral Small 3.1 on RTX 4090.

Why Is My LLM Slow?

6 causes with specific fixes: memory bandwidth, CPU offloading penalty, KV cache bloat, wrong quantization, thermal throttling.

GPU VRAM Per Dollar Tier List

Every GPU ranked by VRAM per dollar. Intel Arc B580 leads on cost per GB. RTX 3090 used is the best all-around value.

How Much VRAM Do I Need?

Exact VRAM for every model size: 7B needs 5 GB, 13B needs 9 GB, 70B needs 38 GB at Q4. Includes KV cache overhead tables.

LLM Inference Speed: Tokens Per Second by GPU

Benchmark tok/s for RTX 4060 through RTX 5090. Why memory bandwidth matters more than CUDA cores.

Qwen3-30B-A3B Hardware Requirements

MoE misconception guide: 30B-A3B needs 17 GB VRAM, NOT 3 GB. GPU table, quant options, and 16 GB workaround.

Ollama vs LM Studio vs llama.cpp vs vLLM

Complete 4-way comparison. 10-row decision matrix. All three local runners use the same inference engine — speed is identical.

Llama 4 Scout Hardware Requirements

109B MoE model needs 57 GB VRAM at Q4. No single consumer GPU fits it — multi-GPU or Mac Studio M4 Max required.

Qwen3-235B Hardware Requirements

Largest open-weight model (235B MoE) needs 119 GB VRAM. Only Mac Studio M4 Ultra fits it. Consumer alternatives listed.

Local LLM Function Calling and Tool Use

Set up agents that call APIs and run code with Ollama. Best models: Qwen3-14B, Llama 3.1 8B. Python examples included.

Codestral Hardware Requirements

Mistral's coding model (22B) needs 13 GB VRAM. Any 16 GB GPU works. FIM support for Continue.dev and Cursor auto-complete.

Best GPU for Coding LLMs

GPU picks matched to the best coding models by VRAM tier. RTX 4060 Ti 16GB for Codestral; RTX 4090 for Qwen2.5-Coder-32B.

LLMs on Integrated Graphics: Intel & AMD iGPU

AMD Radeon 890M gets 12-18 tok/s on 7B models. Best models for no-GPU laptops and how to enable GPU acceleration.

Gemma 4 Hardware Requirements

Gemma 4 27B MoE runs in 8 GB VRAM at Q4 — RTX 4060 gets ~28 tok/s. Best model for 8 GB GPUs in 2026.

What Can I Run on My GPU?

VRAM tier guide: 8GB through 128GB — exact model names, quantization, and speeds.

Best GPU for LLMs

RTX 4060 to RTX 5090 vs Mac Studio — budget tiers compared.

CPU-Only LLM Inference

Run LLMs without a GPU. RAM requirements, speed benchmarks, and best models for CPU inference.

RTX 4090 for LLMs

Best single consumer GPU: 24 GB GDDR6X, 1008 GB/s, up to 110 tok/s. Full guide with model table.

Best LLMs to Run Locally

Top model picks by VRAM tier — Qwen3 8B, DeepSeek-R1-Distill, Gemma 3, Phi-4, Llama 3.3 70B.

Gemma 3 Hardware Requirements

Google's Gemma 3 (4B to 27B). The 27B fits in 16 GB at Q4. GPU picks and Ollama setup.

RTX 5070 for LLMs

Blackwell 12 GB GDDR7. Runs Qwen3 14B at Q4 — 33% more bandwidth than RTX 4070 at a lower price.

Best LLM for Coding Locally

Qwen3 14B, Codestral 22B, Phi-4 14B — ranked by GPU tier. Ollama commands + Continue.dev, Aider, Cursor setup.

Qwen3 14B vs Phi-4 14B

Head-to-head at 12 GB VRAM. Both fit in Q4_K_M — which wins for coding, math, and everyday use?

LLMs on Windows: Complete Setup

Run Ollama and LM Studio on Windows 10/11. NVIDIA, AMD, Intel Arc GPU driver setup. Fix CUDA detection, CPU fallback, port conflicts.

Apple Silicon for Local LLMs

How M3 and M4 unified memory actually performs vs discrete GPUs. The Mac Studio M4 Max 64GB fits 70B at Q4_K_M — no $-equivalent NVIDIA setup does.

Best Budget GPU for LLMs

Sub-$500 picks that actually run modern models. RTX 3060 12GB still leads on $/GB VRAM in 2026.

Llama 4 Hardware Requirements

Llama 4 Scout (109B MoE) runs on 12-16 GB VRAM at Q4. Maverick needs 200+ GB. Which hardware to buy for Meta's latest model.

LM Studio vs Ollama vs Jan

LM Studio for beginners, Ollama for developers and servers, Jan for privacy. Which local AI app should you install first?

Gemma 4 Hardware Requirements

Gemma 4 27B MoE (4B active) fits in 8 GB VRAM at Q4 — RTX 4060 runs it at 28 tok/s. 31B dense needs 24 GB.

RTX 5060 for LLMs: 8 GB Only

Only 8 GB VRAM — runs 7-8B models well, cannot run 14B. The RTX 5060 Ti 16 GB is the smarter AI buy.

Local AI Coding Assistant Setup

Free GitHub Copilot: VS Code + Continue + Ollama + Qwen3-Coder 14B. Works offline, 100% private, needs 16 GB VRAM.

AMD vs NVIDIA for Local LLMs

AMD wins on VRAM per dollar; NVIDIA wins on Windows ease. ROCm 6.3 on Linux is finally production-ready. Full comparison for 2026.

AMD RX 9060 XT LLM Guide

16 GB GDDR6 — double the VRAM of RTX 5060 for a small premium. Runs Qwen3 14B Q4 at 38 tok/s with ROCm 6.3.

AMD Strix Halo Mini PC Guide

128 GB unified memory runs Llama 70B at 8-10 tok/s — about half the price of Mac Studio M4 Max 128 GB.

Run Claude Locally? Here's What Works

Claude weights are not public. Best local alternatives: Qwen3 14B matches Claude Haiku (16 GB), Qwen3 72B matches Sonnet (48 GB).

Best LLMs for 48 GB VRAM

Llama 3.3 70B Q4 fits at 42 GB, runs 20 tok/s on Mac M4 Pro. Qwen3 72B also fits. The full 70B tier breakdown.

Best LLMs for 24 GB VRAM

Qwen3 32B Q4 (19 GB) runs at 28 tok/s on RTX 4090. The jump from 16 GB that unlocks real 32B quality.

Open WebUI Setup Guide

ChatGPT-style interface for your local Ollama — RAG, voice, web search, multi-user. Docker install in one command.

Best Budget GPU for LLMs

Entry tier: RTX 3060 12 GB. Budget tier: Intel Arc B580. Mid-budget: AMD RX 9060 XT 16 GB. Best value overall: RTX 5070 Ti.

LLM Fine-Tuning Hardware

QLoRA on 7B needs 10 GB VRAM; 14B needs 24 GB. RTX 4090 is the best consumer GPU. AMD lacks tool support — NVIDIA only.

Dual GPU Setup for 70B Models

Two RTX 3090s give 48 GB combined VRAM — runs Llama 70B at 8-10 tok/s. NVLink vs PCIe, PSU sizing, Ollama setup.

Local Voice AI Setup

Whisper + Ollama + Kokoro TTS: fully offline voice assistant in 2026. 8 GB VRAM, 2-5s latency, works air-gapped. Setup guide.

RTX 4070 Ti Super vs RTX 4080

Both 16 GB, same models, 7% speed difference, a noticeable price gap. In 2026 the RTX 5070 Ti beats both by 33% for similar money.

RTX 3090 vs RTX 4070 Ti Super

Used 24 GB vs new 16 GB. 70B models need 24 GB — which is the better buy for local LLMs?

RTX 4070 vs 4080 vs 4090

12 GB vs 16 GB vs 24 GB VRAM. 32B models need 24 GB — which GPU should you buy?

AI on Your Gaming PC

Your gaming GPU already runs AI. RTX 4060 to RTX 4090 tier table — see exactly what you can run with your GPU right now.

Best LLMs for 8 GB VRAM

RTX 3060 / 4060 users: Qwen3 7B Q8 runs at 35 t/s, Phi-4 14B fits at Q4. Full model fit table and best picks.

Private Offline AI Setup

Run AI with zero data leaving your PC. Ollama + Qwen3 14B, no internet after setup, works on air-gapped machines.

Run ChatGPT Locally — Free

Open-source models now match GPT-4 quality. Free, private, no subscription. Ollama + Open WebUI in under 10 minutes.

Best LLMs for 16 GB VRAM

RTX 4060 Ti, 4080, 4070 Ti Super users: Qwen3 14B Q8 fits at 14.8 GB and runs at 30+ t/s. Gemma 3 27B Q4 also fits.

RTX 5070 vs RTX 4070 for LLMs

Same 12 GB VRAM, same models. 5070 is 33% faster for less money. Upgrade verdict: buying new — yes; already own 4070 — no.

RTX 5080 vs RTX 4090 for LLMs

RTX 5080 is 16 GB, RTX 4090 is 24 GB. Both fast for 7-14B — only the 4090 fits 32B at Q4. Buy the 5080 unless you need 32B+.

RTX 4060 vs RTX 4070 for LLMs

8 GB vs 12 GB: the 4070 is nearly 2x faster AND runs 14B models the 4060 cannot. Worth the premium if you run anything above 7B.

Best LLMs for 24 GB VRAM

RTX 4090/3090 users: Qwen3 32B Q4 runs at 38 t/s. DeepSeek R1 32B fits. Full model fit table and top picks for the 32B sweet spot.

Run LLMs on Mac: Ollama Setup

M1 through M4 all run AI locally. Metal GPU acceleration is automatic. One command install, then ollama run qwen3:8b — done in 5 minutes.

M4 Mac Mini vs M4 Pro for LLMs

M4 24 GB vs M4 Pro 48 GB. M4 Pro is 2.3x faster and the only Mac Mini that runs 70B models. Clear buy recommendation inside.

Run LLMs on Linux: Ollama Setup

One curl command installs Ollama on Ubuntu, Fedora, or Arch. NVIDIA works instantly. AMD needs ROCm — step-by-step included.

How to Run DeepSeek R1 Locally

Step-by-step Ollama setup for DeepSeek R1 distills (8B–70B). Thinking mode explained, common issues fixed.

How to Run Qwen3 Locally

Ollama setup for Qwen3 0.6B–32B and MoE. Thinking mode per query, LM Studio alternative, Windows/Mac/Linux.

Open WebUI + Ollama Setup

Get a ChatGPT-like browser interface for your local LLMs in 5 minutes. Free, private, no API key required.

How to Run Llama 3 Locally

Run Meta Llama 3.1 8B or Llama 3.3 70B via Ollama. Step-by-step for Windows, Mac, Linux. Hardware picks included.

How to Run Mistral Locally

Run Mistral 7B (4.5 GB), Nemo 12B, or Small 22B via Ollama. Works on any 8 GB+ GPU. Windows, Mac, Linux.

How to Run Gemma 3 Locally

Google's Gemma 3 27B fits in 16 GB VRAM. Multimodal — process images locally. Ollama setup guide.

How to Run Phi-4 Locally

Microsoft's Phi-4 14B needs only 9-10 GB VRAM and beats Llama 3.1 8B on reasoning. Ollama setup guide.

How to Run Llama 4 Scout Locally

Llama 4 Scout needs 58 GB VRAM due to MoE — explains why, dual-GPU llama.cpp setup, and alternatives for 8-24 GB GPUs.

Local AI Coding Assistant Setup

VS Code + Ollama + Continue.dev — free GitHub Copilot alternative. Chat model + FIM autocomplete configured in under 10 minutes.

RTX 5070 Ti vs RTX 5080 for LLMs

Both have 16 GB GDDR7 and run identical models. 5080 is 7% faster. Buy the 5070 Ti unless budget is no concern.

LM Studio: Complete Setup Guide

Desktop GUI for running LLMs offline. Download, load a model, and chat in minutes. No terminal needed. GPU acceleration, model browser, OpenAI-compatible API.

Best LLMs for 32 GB VRAM (RTX 5090)

RTX 5090 users: Qwen3 32B Q8 runs at 45+ t/s. QwQ-32B reasoning model fits. Full model fit table for the 32 GB tier.

Best LLMs for 12 GB VRAM (RTX 4070 / 5070)

Qwen3 14B Q4 runs at 30 t/s on RTX 4070. Phi-4 14B fits. Gemma 3 12B is fastest. The 12 GB sweet spot explained.

Ollama Commands Cheat Sheet

Every Ollama CLI command in one place: run, pull, list, API endpoints, Modelfile guide, environment variables, one-liners.

AMD RX 9070 XT for Local LLMs

16 GB GDDR6, 896 GB/s — matches RTX 5080 LLM throughput. RDNA 4 ROCm 6.2 setup guide and model fit table.

DeepSeek V3: Can You Run It Locally?

DeepSeek V3 (685B) needs 390 GB VRAM — no consumer GPU can run it. Honest analysis and the best consumer alternatives.

RTX 5060 Ti for LLMs: 8 GB vs 16 GB

Always buy the 16 GB variant. Qwen3 14B Q4 at 35+ t/s. The RTX 5070 is 2x faster — worth considering.

LLM System Requirements: CPU, RAM, PSU

GPU VRAM is the bottleneck but you also need 32 GB RAM, NVMe SSD, and a right-sized PSU. Complete spec tables per tier.

Run Qwen3 30B MoE Locally

30B total params, only 3B active — runs at 30 t/s on RTX 4090 while fitting in 20 GB. Thinking mode guide and performance vs dense comparison.

Ollama Python API Guide

Use Ollama from Python: ollama library, OpenAI-compatible endpoint, streaming, embeddings, async. Working code examples for all patterns.

How to Run 70B Models Locally

Llama 3.3 70B needs 42 GB VRAM. RTX 4090 alone is not enough. Mac M4 Pro 48 GB is the best consumer option. Dual GPU setup also covered.

RTX 5090 vs RTX 4090 for LLMs

5090 is 78% faster and has 32 GB vs 24 GB. Enables Qwen3 32B at Q6. Worth the premium? Speed table, model fit, buy verdict.

Mac Mini vs Mac Studio for LLMs

M4 Pro 48 GB runs 70B at 12 t/s. M4 Max 64 GB runs it at 20 t/s. Full comparison and buy recommendation.

Best LLMs for 6 GB VRAM

RTX 3060 6GB and GTX 1660 Super: Qwen3 7B Q4 at 30 t/s. Limited to 7-8B but surprisingly capable. Upgrade analysis vs 8 GB included.

llama.cpp Guide: Run Without Ollama

Direct GGUF inference with full control. CUDA/Metal/ROCm build, GPU layer flags, server mode, performance tuning. For power users.

RTX 5070 vs RTX 5070 Ti for LLMs

The 5070 is faster than the 5070 Ti on shared models — the 5070 Ti just has more VRAM. The price step up buys 14B Q8 capability, not speed.

RAG: Chat With Your Own Documents

Add document search to your local LLM: Open WebUI (no code), AnythingLLM (one app), or Python LangChain. Hardware requirements and embedding model picks.

Best LLMs for Writing Locally

Llama 3.3 70B for fiction, Qwen3 14B Q8 for content, Mistral Small 22B for creative. Temperature guide and system prompts for each use case.

Is 8 GB VRAM Enough for AI?

Yes — Qwen3 7B at 35 t/s fits comfortably, but 14B models at Q4 do not. Upgrade analysis, Stable Diffusion verdict, best 8 GB GPUs.

Vision/Multimodal LLMs Locally

Gemma 3 4B runs image + text in 4 GB VRAM. Describe images, extract text from screenshots, analyze charts — all private with Ollama.

Home Server LLM Setup Guide

Run Ollama 24/7 on a home server. Network access, Tailscale VPN, Open WebUI frontend. Power cost: Mac Mini M4 Pro is 10x cheaper to run than a gaming PC.

Run Ollama in Docker (GPU Support)

NVIDIA GPU passthrough in 1 extra flag. Docker Compose file with Open WebUI included. 3 commands from zero to running model.

How to Speed Up Ollama

Flash attention, optimized context size, Q4_K_M, KEEP_ALIVE=-1. Full environment variables reference and benchmark commands.