Editorial: AI structured the three-way comparison. The break-point logic ("when each tier wins") follows the VRAM formula and each card’s listed memory and bandwidth.
Key facts
4070
12 GB at 504 GB/s. 14B Q4 is 10.4 GB; Qwen3-14B gets about 15,800 tokens of f16 context.
4080
16 GB at 716 GB/s. Qwen3-14B Q4 with its full 32K context is 14.8 GB.
4090
24 GB at 1,008 GB/s. 32B Q4 is 21.2 GB and fits; 70B Q4 is 44 GB and does not.
Checked
2026-10-13
Checked 2026-10-13. Author: LLM Hardware team. Amazon prices are not stored or shown on this site.
Citeable facts
Key fact: Qwen3-14B at Q4_K_M fits a 12 GB card with about 15,800 tokens of f16 KV cache, and a 16 GB card with its full 32,768-token window.
How we got it: Weights 14.8 (model card) x 0.5 = 7.4 GB, plus the 2 GB reserve, plus KV at 2 x 40 x 8 x 128 x 2 = 163,840 bytes per token (config.json), per the methodology's long-context rule. 12 GB: 2.6 GB / 163,840 bytes. 16 GB at 32,768 tokens: 7.4 + 2 + 5.37 = 14.8 GB.
Checked:
Key fact: Q4_K_M, site formula, before long-context KV cache: 14B is 10.4 GB, 32B is 21.2 GB and 70B is 44 GB.
How we got it: params x 0.5 x 1.2 + 2 with params 14, 32 and 70.
Checked:
Key fact: Bandwidth ceilings for qwen3:14b (9.3GB): about 54 tok/s on the RTX 4070, 76 on the RTX 4080 and 108 on the RTX 4090. Upper bounds, not test results.
How we got it: Catalog bandwidth 504, 716 and 1,008 GB/s divided by Ollama's listed download size, treating every weight as read once per token.
Checked:
Methodology
explains the VRAM formula, the bytes per weight at each quantization, and the full source list.
Updated November 2026 · RTX 4070 12 GB vs RTX 4080 16 GB vs RTX 4090 24 GB · VRAM comparison · Bandwidth ceilings · 32B model compatibility
Three NVIDIA consumer GPUs for local AI, each with a different VRAM tier. The RTX 4070 has 12 GB, the RTX 4080 has 16 GB, and the RTX 4090 has 24 GB. That VRAM difference determines which models you can run, and the bandwidth difference determines how fast they generate tokens. This guide breaks down exactly what you get at each price point so you can pick the right card for your workload.
Quick Verdict
RTX 4070 12GB — Budget
Best budget pick
Fits 8B at Q4 (6.8 GB) and 14B at Q4 (10.4 GB) at short context
504 GB/s bandwidth (NVIDIA)
Low 200W power draw
Cannot run 32B or 70B models
Best for casual and budget users
RTX 4080 16GB: Middle Tier
16 GB at 716 GB/s
Fits 14B at Q4 with a 32K context and 20B at Q4 (14.0 GB)
716 GB/s bandwidth (catalog figure)
Handles 20-22B models at Q4
Cannot run 32B in VRAM
For 14B with long context
RTX 4090 24GB — Max Performance
Best for power users
Runs Qwen3 32B and DeepSeek 32B at Q4
1008 GB/s; 127.7 tok/s on Llama 3 8B Q4_K_M (XiongjieDai)
Fastest token generation of any single GPU
Still cannot run 70B models
Best for power users and researchers
None of these three cards run 70B models
Llama 3.3 70B at Q4_K_M is 44 GB by the site formula (70 x 0.5 x 1.2 + 2, before long-context KV cache), almost twice the RTX 4090's 24 GB. A 48 GB Mac has about 44 GB usable after macOS, which is the edge; a 64 GB Mac Studio has about 60 GB usable. If 70B is your goal, compare unified-memory machines rather than these three cards.
Full Comparison: RTX 4070 vs RTX 4080 vs RTX 4090
Metric
RTX 4070 12GB
RTX 4080 16GB
RTX 4090 24GB
VRAM
12 GB GDDR6X
16 GB GDDR6X
24 GB GDDR6X
Memory bandwidth
504 GB/s
716 GB/s
1008 GB/s
8B Q4 VRAM (6.8 GB)
Fits
Fits
Fits (127.7 tok/s Llama 3 8B, XiongjieDai)
14B Q4 VRAM (10.4 GB)
Fits, short context
Fits
Fits
32B Q4 VRAM (21.2 GB)
doesn't fit
doesn't fit
Fits, short context
Llama 3.3 70B Q4 (44 GB)
No
No
No
DeepSeek-R1-14B Q4 (10.4 GB)
Yes, short context
Yes
Yes
Can run Qwen3 32B Q4
No (21.2 GB needed)
No
Yes (21.2 GB)
Total graphics power (NVIDIA)
200 W
320 W
450 W
Best for
7B-14B models, budget efficiency
14B with 32K context, 20B at Q4
Everything up to 32B at high speed
The one speed figure in this table is a third-party llama.cpp run listed in the sources below; this site does not run speed tests. VRAM rows are Q4_K_M, site formula, before long-context KV cache. Prices are on each Amazon listing, not on this page.
VRAM Comparison: The 12 GB to 16 GB to 24 GB Cliff
VRAM capacity is the single most important specification for local LLMs. It decides which models run entirely in VRAM. A model that does not fit spills layers to system RAM, which is much slower; this site has not tested by how much on these cards. Each jump in VRAM tier unlocks a meaningful new capability tier.
12 GB (RTX 4070): 7B-14B at Q4, nothing larger
By the site formula at Q4_K_M, a 14B model is 10.4 GB, which leaves 1.6 GB for context; the Qwen3-14B example below works out how many tokens that buys. An 8B at Q8 is 11.6 GB and fits with 0.4 GB spare. What does not fit: 14B at Q8 (18.8 GB), a 20B at Q4 (14.0 GB), and 32B or 70B at any quantization.
16 GB (RTX 4080): 14B with a 32K context, 20-22B at Q4
At 16 GB a 14B at Q4 has room for a 32K context (see the Qwen3-14B example below). A 20B at Q4 is 14.0 GB and a 22B is 15.2 GB, the second with little room for context. What still does not fit: 14B at Q8 (18.8 GB by the formula for the 14B class, 19.8 GB for Qwen3-14B's 14.8B parameters; Ollama lists qwen3:14b-q8_0 at 16GB), Gemma 3 27B at Q4 (18.2 GB), Qwen3 32B at Q4 (21.2 GB) and any 70B.
24 GB (RTX 4090): 32B at Q4, everything up to the 70B wall
At 24 GB the 32B class fits at a short context: Qwen3 32B and DeepSeek-R1-Distill-32B at Q4 are 21.2 GB by the site formula (32 x 0.5 x 1.2 + 2), leaving 2.8 GB, and Gemma 3 27B at Q4 is 18.2 GB. A 14B at Q8 (18.8 GB) fits too. The wall is 70B: Llama 3.3 70B at Q4_K_M is 44 GB, almost twice the card.
How much context fits: Qwen3-14B on 12, 16 and 24 GB
The VRAM rows on this page use the site formula at Q4_K_M, whose 1.2 factor covers about 4K to 8K tokens of context. For longer conversations the methodology sizes the KV cache directly: 2 x layers x KV heads x head dimension x bytes per element, per token. Qwen3-14B's config.json lists 40 layers, 8 key-value heads and a head dimension of 128. Its model card lists 14.8B parameters and a native context of 32,768 tokens. The tables on this page use a round 14 for the 14B class (10.4 GB), while this example uses the card's 14.8B, which is 10.9 GB at a short context.
With an f16 cache (2 bytes per element) each token costs 2 x 40 x 8 x 128 x 2 = 163,840 bytes, about 0.16 MB. The Q4_K_M weights are 14.8 x 0.5 = 7.4 GB, and the 2 GB reserve for the operating system and the inference engine still applies. So the total is 7.4 + 2 + KV cache.
Context (tokens)
f16 KV cache
Total
RTX 4070 (12 GB)
RTX 4080 (16 GB)
RTX 4090 (24 GB)
4,096
0.67 GB
10.1 GB
Fits, 1.9 GB spare
Fits
Fits
8,192
1.34 GB
10.7 GB
Fits, 1.3 GB spare
Fits
Fits
16,384
2.68 GB
12.1 GB
Does not fit (0.1 GB over)
Fits, 3.9 GB spare
Fits
32,768 (native maximum)
5.37 GB
14.8 GB
Does not fit
Fits, 1.2 GB spare
Fits, 9.2 GB spare
On the RTX 4070, Qwen3-14B at Q4 leaves 12 - 9.4 = 2.6 GB for the cache, which is about 15,800 tokens at f16 (2.6 GB / 163,840 bytes). An 8-bit KV cache halves the cost to 81,920 bytes per token and stretches that to about 31,700 tokens, slightly short of the native 32,768. The RTX 4080 holds the full native window at f16. The RTX 4090 has 14.6 GB for cache, about 89,000 tokens at f16, though the model card says context past 32,768 tokens needs YaRN scaling. The Ollama and llama.cpp settings for an 8-bit cache are in our KV cache quantization guide.
Speed Comparison: Bandwidth Drives Token Speed
Token generation on one GPU is limited mostly by memory bandwidth, because each new token reads the model's weights from VRAM. The RTX 4090's 1,008 GB/s is about 41% more than the RTX 4080's 716 GB/s, which is about 42% more than the RTX 4070's 504 GB/s. Dividing bandwidth by the size of the weight file gives an upper bound on tokens per second (table below). Real speed is lower, and this site has not tested these cards.
Llama 3.1 8B Q4_K_M
RTX 4070: not tested here
RTX 4080: not tested here
RTX 4090: 127.7 tok/s (Llama 3 8B, XiongjieDai, May 2024)
Qwen3 14B Q4_K_M
RTX 4070: fits at short context (10.4 GB)
RTX 4080: fits (10.4 GB)
RTX 4090: fits (10.4 GB)
DeepSeek-R1-32B Q4
RTX 4070: cannot run (21.2 GB)
RTX 4080: cannot run (21.2 GB)
RTX 4090: fits at short context
Bandwidth-derived upper bound, not a speed test
Method: memory bandwidth from our hardware catalog divided by the download size Ollama lists for each tag, read on 30 October 2026. It assumes every weight is read once per generated token and leaves out the KV cache, so real speeds are lower, and lower again at long context. This site has not run speed tests on any of these cards.
Ollama tag (listed download)
RTX 4070, 504 GB/s
RTX 4080, 716 GB/s
RTX 4090, 1,008 GB/s
qwen3:8b (5.2GB)
at most about 96 tok/s
at most about 137 tok/s
at most about 193 tok/s
qwen3:14b (9.3GB)
at most about 54 tok/s
at most about 76 tok/s
at most about 108 tok/s
qwen3:32b (20GB)
does not fit
does not fit
at most about 50 tok/s
The ratio between the cards is the same for every model that fits: the RTX 4080's ceiling is about 1.42 times the RTX 4070's, and the RTX 4090's is about 1.41 times the RTX 4080's. Where a model does not fit, the bound does not apply, because the layers left in system RAM run at system-memory speed.
All three cards fit 8B models. Speed follows bandwidth (504, 716 and 1,008 GB/s), but the only measured figure here is the 4090's 127.7 tok/s on Llama 3 8B Q4_K_M from the XiongjieDai llama.cpp runs. At 32B only the 4090 holds the model in VRAM, so the comparison is moot for the other two cards.
Power and Running Cost: 450W vs 200W
The RTX 4090 draws approximately 450W under LLM load. The RTX 4070 draws approximately 200W. That 250W difference adds up over sustained use — and it affects PSU requirements and electricity bills.
Estimated GPU energy at 8 hours/day of full board power (NVIDIA TGP)
RTX 4070 (200W)
~48 kWh/month
~584 kWh/year
RTX 4080 (320W)
~77 kWh/month
~934 kWh/year
RTX 4090 (450W)
~108 kWh/month
~1,314 kWh/year
At 8 hours a day the RTX 4090 uses about 730 kWh a year more than the RTX 4070 at full board power. Multiply by your local electricity rate; real inference draw is often below TGP.
PSU requirements also differ: the RTX 4090 needs at minimum a 850W PSU with a modern CPU. NVIDIA lists 750W required system power for the RTX 4080. The RTX 4070 runs comfortably on 650W. If you are upgrading and your PSU is borderline, factor in upgrade cost for the higher-draw cards.
The 70B Question: None of These Cards Can Do It
If your goal is running 70B-class models such as Llama 3.3 70B, none of the three cards in this comparison holds one in VRAM. Here is why:
Llama 3.3 70B Q4_K_M needs 44 GB by the site formula
70 x 0.5 x 1.2 + 2 = 44 GB at Q4_K_M, before long-context KV cache. The RTX 4090 tops out at 24 GB, 20 GB short. GPU plus CPU offloading runs, but this site has not tested how fast on these cards.
A 72B model needs 45.2 GB
72 x 0.5 x 1.2 + 2 = 45.2 GB at Q4_K_M. None of the three cards comes close, and offloading speed on these cards has not been tested here.
For 70B, you need a Mac or dual-GPU setup
Llama 3.3 70B at Q4 needs 44 GB. A 48 GB Mac holds it with very little room; a 64 GB Mac Studio is more comfortable. Two 24 GB GPUs (48 GB combined) also work but need a board with two suitable slots and careful configuration. No single consumer NVIDIA GPU in this comparison runs it.
What the RTX 4090 does unlock at 24 GB
The jump from 16 GB to 24 GB is what fits 32B models: Qwen3 32B and DeepSeek-R1-Distill-32B at Q4 are 21.2 GB by the site formula, and Gemma 3 27B at Q4 is 18.2 GB. If 32B is your target, the 4090 is the card of these three. If 70B is your target, look at 64 GB-class unified-memory machines instead.
Price-to-Performance Analysis
Raw price-per-token-per-second is one lens, but the model compatibility cliff matters more. A card that runs a model 30% slower but actually runs the model is infinitely better than one that cannot run it at all.
GPU
Price
Bandwidth (catalog)
Unlocks 32B?
RTX 4070 12GB
Check price on Amazon
504 GB/s
No
RTX 4080 16GB
Check price on Amazon
716 GB/s
No
RTX 4090 24GB
Check price on Amazon
1008 GB/s
Yes
We do not publish price-per-token math here because prices move and we have not measured all three cards. The decision framework: pick the RTX 4070 if 8B–14B at Q4 is your ceiling, the RTX 4080 if you want 14B with a 32K context or 20B at Q4 (14.0 GB), and the RTX 4090 if you want 32B at Q4 (21.2 GB).
✓Q4 quality is acceptable — you do not need Q8 near-lossless on 14B
✓You want low power draw (~200W) and a quiet, efficient build
✓You are new to local LLMs and want to try without heavy investment
Avoid if: you want Qwen3 32B, DeepSeek-R1-Distill-32B, or Gemma 3 27B — none fit in 12 GB.
Buy the RTX 4080 16GB if...
✓You want a 14B at Q4 with a 32K context
✓You run 20B to 22B models at Q4 (14.0 to 15.2 GB by the site formula)
✓You want more bandwidth than the 4070 (716 vs 504 GB/s, about 42% more)
✓You want the middle VRAM tier of these three
✓You do not specifically need 32B models
Avoid if: you need Qwen3 32B or DeepSeek-R1-Distill-32B at Q4. They are 21.2 GB by the site formula, 5.2 GB over 16 GB.
Buy the RTX 4090 24GB if...
✓You want to run Qwen3 32B, DeepSeek-R1-Distill-32B, or Gemma 3 27B
✓You want the most bandwidth of these three cards (1008 GB/s)
✓You run multiple users or applications against the same local GPU
✓You do heavy long-context inference where speed compounds over generations
✓Budget is secondary to capability
Avoid if: your only goal is 70B models — the 4090 still cannot run them. Budget goes further on Apple Silicon for 70B.
Running LLMs on Any of These Cards: Ollama Setup
All three cards are fully supported by Ollama and LM Studio. NVIDIA GPU detection is automatic — no special configuration needed.
Install Ollama
Windows and macOS
Download from ollama.com. Installs as a background service, auto-detects NVIDIA GPUs.
Linux
curl -fsSL https://ollama.com/install.sh | sh
RTX 4070 12GB — recommended models
Qwen3 14B Q4_K_M: 10.9 GB by the formula, 9.3GB download, about 15,800 tokens of f16 context
ollama run qwen3:14b
Qwen3 8B Q4_K_M: 6.8 GB by the formula, 5.2GB download, room for long context
ollama run qwen3:8b
Qwen3 8B Q8_0: 11.6 GB by the formula, 8.9GB download, short context only
ollama run qwen3:8b-q8_0
RTX 4080 16GB: models that fit
Qwen3 14B Q4_K_M: room for its full 32K context (14.8 GB by the KV method above; raise the context length in Ollama to use it)
ollama run qwen3:14b
Qwen3 8B Q8_0: 11.6 GB by the formula, 8.9GB download, 4.4 GB spare
ollama run qwen3:8b-q8_0
RTX 4090 24GB — recommended models
Qwen3 32B Q4_K_M: 21.2 GB by the formula, 20GB download, short context
ollama run qwen3:32b
DeepSeek-R1-Distill-32B Q4_K_M
ollama run deepseek-r1:32b
Qwen3 14B Q8_0: 19.8 GB by the formula, 16GB download
ollama run qwen3:14b-q8_0
Verify GPU load after starting a model
ollama ps
GPU% should show 100% for full VRAM inference. If it shows partial %, some layers are on CPU and speed will be significantly slower. Reduce context length or quantization if the model is spilling to CPU.
Frequently Asked Questions
Which NVIDIA GPU is best for running LLMs locally?
The RTX 4090 (24 GB, 1008 GB/s) is the only one of the three that fits 32B at Q4 (21.2 GB). The RTX 4080 16GB (716 GB/s) fits 14B at Q4 (10.4 GB) with a 32K context and 20B at Q4 (14.0 GB). The RTX 4070 12GB (504 GB/s) fits 8B easily and 14B at Q4 with little context room. Pick by the largest model you want to run.
Can the RTX 4070 run Llama 3 70B?
No. Llama 3.3 70B at Q4_K_M is 44 GB by the site formula (70 x 0.5 x 1.2 + 2, Q4_K_M, before long-context KV cache). The RTX 4070 has 12 GB, the RTX 4080 16 GB and the RTX 4090 24 GB, so none of the three holds it in VRAM. A 48 GB Mac has about 44 GB usable after macOS, which is the edge; a 64 GB Mac Studio has about 60 GB usable and holds it. Splitting the model between GPU and system RAM runs, but this site has not tested how fast.
Is the RTX 4090 worth it over the RTX 4080?
Yes, if you want 32B models: the 4090's 24 GB holds Qwen3 32B and DeepSeek-R1-Distill-32B at Q4 (21.2 GB, Q4_K_M, site formula, before long-context KV cache), which 16 GB cannot. It also has about 41% more memory bandwidth than the RTX 4080 (1,008 vs 716 GB/s), which raises the ceiling on token speed; this site has not tested the gap. If you only run 14B models, the 4080 fits them with a 32K context.
Which of these cards runs Qwen3 14B?
All three run it at Q4_K_M. The model card lists 14.8B parameters, which is 10.9 GB by the site formula at a short context, and Ollama lists the qwen3:14b download at 9.3GB. On the 12 GB RTX 4070 that leaves room for about 15,800 tokens of f16 KV cache (worked example above). Q8 is different: the formula gives 19.8 GB and Ollama lists qwen3:14b-q8_0 at 16GB, so Q8 needs the 24 GB RTX 4090, not a 16 GB card.
Is the RTX 4080 faster than the RTX 4090 for 7B and 8B models?
No. The RTX 4090's 1,008 GB/s is about 41% more memory bandwidth than the RTX 4080's 716 GB/s, and token generation is limited mostly by bandwidth, so the 4090 has the higher ceiling on any model both cards hold. For qwen3:8b (a 5.2GB download on Ollama), bandwidth divided by file size gives upper bounds of about 193 tok/s on the 4090 and 137 tok/s on the 4080. Those are ceilings, not test results.
How much context fits with a 14B model on a 12 GB card?
For Qwen3-14B at Q4_K_M, the weights are 14.8 x 0.5 = 7.4 GB and the site keeps 2 GB in reserve, which leaves 2.6 GB of a 12 GB card for the KV cache. At 163,840 bytes per token in f16 (40 layers, 8 KV heads, head dimension 128) that is about 15,800 tokens. An 8-bit cache roughly doubles it to about 31,700 tokens. A 16 GB card holds the model's full 32,768-token window at f16.
Why does this page compare the RTX 4080 and not the RTX 4080 Super?
The RTX 4080 16GB is the card in our hardware catalog, at 716 GB/s. NVIDIA's spec page lists the same 16 GB of GDDR6X, 256-bit memory interface, 320 W total graphics power and 750 W required system power for both cards, so every VRAM fit on this page applies to either one. The bandwidth ceilings use the RTX 4080's 716 GB/s.
Where to find the RTX 4070 and RTX 4080 compared above
Fit notes repeat the site formula estimates from the tables above, at Q4 and a short context unless a context length is given.
NVIDIA GeForce RTX 4070 (12 GB)
12 GB at 504 GB/s. Fits 14B at Q4 (10.4 GB by the site formula) with up to about 15,800 tokens of f16 context for Qwen3-14B, and an 8B at Q8 (11.6 GB) with little room left. Does not fit 20B at Q4 (14.0 GB) or 32B at Q4 (21.2 GB).
16 GB at 716 GB/s. Fits Qwen3-14B at Q4 with its full 32,768-token context (14.8 GB) and a 20B at Q4 (14.0 GB). Does not fit 14B at Q8 (18.8 GB), Gemma 3 27B at Q4 (18.2 GB) or 32B at Q4 (21.2 GB).
Which models fit the RTX 4070, 4080 and 4090 here comes from the site VRAM formula and the model cards listed below. The formula and the sitewide source list are on the methodology page. For this guide we used:
Home GPU LLM Leaderboard. VRAM tier mapping (12/16/24 GB) that drives the 'which one for which model' calls.
Qwen3-14B model card and config.json. Parameter count, layers, KV heads, head dimension and context length for the KV-cache example, read 30 October 2026.
Ollama qwen3 tags. Download sizes for qwen3:8b, qwen3:14b, qwen3:32b and the q8_0 tags, read 30 October 2026.
Spot a number that does not match the linked source? Contact us and we will update the guide.
We use cookies and personalised ads
This site uses cookies and shows personalised ads via Google AdSense. We and our partners store and access information on your device to serve relevant ads and improve your experience.
You can accept all cookies, decline (non-personalised ads only), or
manage preferences.
See our Privacy Policy.
We use Google Analytics
This site would like to use Google Analytics to understand how visitors use the site. If you accept, Google Analytics will load. If you decline, or if you make no choice, Google Analytics will not load. No ads are shown on this visit.
See our Privacy Policy and Cookie Policy.
Cookie preferences
Choose which cookies you allow. Strictly necessary cookies are always active.
Strictly necessary
Session state, security, and performance. Cannot be disabled.