RTX 4070 vs RTX 4080 vs RTX 4090 for Local LLMs: Which Should You Buy? (2026)

By LLM Hardware team · Methodology ·

Editorial: AI structured the three-way comparison. The break-point logic ("when each tier wins") follows the VRAM formula and each card’s listed memory and bandwidth.

Key facts

4070
12 GB at 504 GB/s. 14B Q4 is 10.4 GB; Qwen3-14B gets about 15,800 tokens of f16 context.
4080
16 GB at 716 GB/s. Qwen3-14B Q4 with its full 32K context is 14.8 GB.
4090
24 GB at 1,008 GB/s. 32B Q4 is 21.2 GB and fits; 70B Q4 is 44 GB and does not.
Checked
2026-10-13

Checked 2026-10-13. Author: LLM Hardware team. Amazon prices are not stored or shown on this site.

Citeable facts

Key fact: Qwen3-14B at Q4_K_M fits a 12 GB card with about 15,800 tokens of f16 KV cache, and a 16 GB card with its full 32,768-token window.

How we got it: Weights 14.8 (model card) x 0.5 = 7.4 GB, plus the 2 GB reserve, plus KV at 2 x 40 x 8 x 128 x 2 = 163,840 bytes per token (config.json), per the methodology's long-context rule. 12 GB: 2.6 GB / 163,840 bytes. 16 GB at 32,768 tokens: 7.4 + 2 + 5.37 = 14.8 GB.

Checked:

Key fact: Q4_K_M, site formula, before long-context KV cache: 14B is 10.4 GB, 32B is 21.2 GB and 70B is 44 GB.

How we got it: params x 0.5 x 1.2 + 2 with params 14, 32 and 70.

Checked:

Key fact: Bandwidth ceilings for qwen3:14b (9.3GB): about 54 tok/s on the RTX 4070, 76 on the RTX 4080 and 108 on the RTX 4090. Upper bounds, not test results.

How we got it: Catalog bandwidth 504, 716 and 1,008 GB/s divided by Ollama's listed download size, treating every weight as read once per token.

Checked:

Methodology explains the VRAM formula, the bytes per weight at each quantization, and the full source list.

Updated November 2026 · RTX 4070 12 GB vs RTX 4080 16 GB vs RTX 4090 24 GB · VRAM comparison · Bandwidth ceilings · 32B model compatibility

Three NVIDIA consumer GPUs for local AI, each with a different VRAM tier. The RTX 4070 has 12 GB, the RTX 4080 has 16 GB, and the RTX 4090 has 24 GB. That VRAM difference determines which models you can run, and the bandwidth difference determines how fast they generate tokens. This guide breaks down exactly what you get at each price point so you can pick the right card for your workload.

Quick Verdict

RTX 4070 12GB — Budget

Best budget pick

  • Fits 8B at Q4 (6.8 GB) and 14B at Q4 (10.4 GB) at short context
  • 504 GB/s bandwidth (NVIDIA)
  • Low 200W power draw
  • Cannot run 32B or 70B models
  • Best for casual and budget users

RTX 4080 16GB: Middle Tier

16 GB at 716 GB/s

  • Fits 14B at Q4 with a 32K context and 20B at Q4 (14.0 GB)
  • 716 GB/s bandwidth (catalog figure)
  • Handles 20-22B models at Q4
  • Cannot run 32B in VRAM
  • For 14B with long context

RTX 4090 24GB — Max Performance

Best for power users

  • Runs Qwen3 32B and DeepSeek 32B at Q4
  • 1008 GB/s; 127.7 tok/s on Llama 3 8B Q4_K_M (XiongjieDai)
  • Fastest token generation of any single GPU
  • Still cannot run 70B models
  • Best for power users and researchers

None of these three cards run 70B models

Llama 3.3 70B at Q4_K_M is 44 GB by the site formula (70 x 0.5 x 1.2 + 2, before long-context KV cache), almost twice the RTX 4090's 24 GB. A 48 GB Mac has about 44 GB usable after macOS, which is the edge; a 64 GB Mac Studio has about 60 GB usable. If 70B is your goal, compare unified-memory machines rather than these three cards.

Full Comparison: RTX 4070 vs RTX 4080 vs RTX 4090

Metric RTX 4070 12GB RTX 4080 16GB RTX 4090 24GB
VRAM 12 GB GDDR6X 16 GB GDDR6X 24 GB GDDR6X
Memory bandwidth 504 GB/s 716 GB/s 1008 GB/s
8B Q4 VRAM (6.8 GB) Fits Fits Fits (127.7 tok/s Llama 3 8B, XiongjieDai)
14B Q4 VRAM (10.4 GB) Fits, short context Fits Fits
32B Q4 VRAM (21.2 GB) doesn't fit doesn't fit Fits, short context
Llama 3.3 70B Q4 (44 GB) No No No
DeepSeek-R1-14B Q4 (10.4 GB) Yes, short context Yes Yes
Can run Qwen3 32B Q4 No (21.2 GB needed) No Yes (21.2 GB)
Total graphics power (NVIDIA) 200 W 320 W 450 W
Best for 7B-14B models, budget efficiency 14B with 32K context, 20B at Q4 Everything up to 32B at high speed

The one speed figure in this table is a third-party llama.cpp run listed in the sources below; this site does not run speed tests. VRAM rows are Q4_K_M, site formula, before long-context KV cache. Prices are on each Amazon listing, not on this page.

VRAM Comparison: The 12 GB to 16 GB to 24 GB Cliff

VRAM capacity is the single most important specification for local LLMs. It decides which models run entirely in VRAM. A model that does not fit spills layers to system RAM, which is much slower; this site has not tested by how much on these cards. Each jump in VRAM tier unlocks a meaningful new capability tier.

12 GB (RTX 4070): 7B-14B at Q4, nothing larger

By the site formula at Q4_K_M, a 14B model is 10.4 GB, which leaves 1.6 GB for context; the Qwen3-14B example below works out how many tokens that buys. An 8B at Q8 is 11.6 GB and fits with 0.4 GB spare. What does not fit: 14B at Q8 (18.8 GB), a 20B at Q4 (14.0 GB), and 32B or 70B at any quantization.

16 GB (RTX 4080): 14B with a 32K context, 20-22B at Q4

At 16 GB a 14B at Q4 has room for a 32K context (see the Qwen3-14B example below). A 20B at Q4 is 14.0 GB and a 22B is 15.2 GB, the second with little room for context. What still does not fit: 14B at Q8 (18.8 GB by the formula for the 14B class, 19.8 GB for Qwen3-14B's 14.8B parameters; Ollama lists qwen3:14b-q8_0 at 16GB), Gemma 3 27B at Q4 (18.2 GB), Qwen3 32B at Q4 (21.2 GB) and any 70B.

24 GB (RTX 4090): 32B at Q4, everything up to the 70B wall

At 24 GB the 32B class fits at a short context: Qwen3 32B and DeepSeek-R1-Distill-32B at Q4 are 21.2 GB by the site formula (32 x 0.5 x 1.2 + 2), leaving 2.8 GB, and Gemma 3 27B at Q4 is 18.2 GB. A 14B at Q8 (18.8 GB) fits too. The wall is 70B: Llama 3.3 70B at Q4_K_M is 44 GB, almost twice the card.

How much context fits: Qwen3-14B on 12, 16 and 24 GB

The VRAM rows on this page use the site formula at Q4_K_M, whose 1.2 factor covers about 4K to 8K tokens of context. For longer conversations the methodology sizes the KV cache directly: 2 x layers x KV heads x head dimension x bytes per element, per token. Qwen3-14B's config.json lists 40 layers, 8 key-value heads and a head dimension of 128. Its model card lists 14.8B parameters and a native context of 32,768 tokens. The tables on this page use a round 14 for the 14B class (10.4 GB), while this example uses the card's 14.8B, which is 10.9 GB at a short context.

With an f16 cache (2 bytes per element) each token costs 2 x 40 x 8 x 128 x 2 = 163,840 bytes, about 0.16 MB. The Q4_K_M weights are 14.8 x 0.5 = 7.4 GB, and the 2 GB reserve for the operating system and the inference engine still applies. So the total is 7.4 + 2 + KV cache.

Context (tokens)f16 KV cacheTotalRTX 4070 (12 GB)RTX 4080 (16 GB)RTX 4090 (24 GB)
4,0960.67 GB10.1 GBFits, 1.9 GB spareFitsFits
8,1921.34 GB10.7 GBFits, 1.3 GB spareFitsFits
16,3842.68 GB12.1 GBDoes not fit (0.1 GB over)Fits, 3.9 GB spareFits
32,768 (native maximum)5.37 GB14.8 GBDoes not fitFits, 1.2 GB spareFits, 9.2 GB spare

On the RTX 4070, Qwen3-14B at Q4 leaves 12 - 9.4 = 2.6 GB for the cache, which is about 15,800 tokens at f16 (2.6 GB / 163,840 bytes). An 8-bit KV cache halves the cost to 81,920 bytes per token and stretches that to about 31,700 tokens, slightly short of the native 32,768. The RTX 4080 holds the full native window at f16. The RTX 4090 has 14.6 GB for cache, about 89,000 tokens at f16, though the model card says context past 32,768 tokens needs YaRN scaling. The Ollama and llama.cpp settings for an 8-bit cache are in our KV cache quantization guide.

Speed Comparison: Bandwidth Drives Token Speed

Token generation on one GPU is limited mostly by memory bandwidth, because each new token reads the model's weights from VRAM. The RTX 4090's 1,008 GB/s is about 41% more than the RTX 4080's 716 GB/s, which is about 42% more than the RTX 4070's 504 GB/s. Dividing bandwidth by the size of the weight file gives an upper bound on tokens per second (table below). Real speed is lower, and this site has not tested these cards.

Llama 3.1 8B Q4_K_M

  • RTX 4070: not tested here
  • RTX 4080: not tested here
  • RTX 4090: 127.7 tok/s (Llama 3 8B, XiongjieDai, May 2024)

Qwen3 14B Q4_K_M

  • RTX 4070: fits at short context (10.4 GB)
  • RTX 4080: fits (10.4 GB)
  • RTX 4090: fits (10.4 GB)

DeepSeek-R1-32B Q4

  • RTX 4070: cannot run (21.2 GB)
  • RTX 4080: cannot run (21.2 GB)
  • RTX 4090: fits at short context

Bandwidth-derived upper bound, not a speed test

Method: memory bandwidth from our hardware catalog divided by the download size Ollama lists for each tag, read on 30 October 2026. It assumes every weight is read once per generated token and leaves out the KV cache, so real speeds are lower, and lower again at long context. This site has not run speed tests on any of these cards.

Ollama tag (listed download)RTX 4070, 504 GB/sRTX 4080, 716 GB/sRTX 4090, 1,008 GB/s
qwen3:8b (5.2GB)at most about 96 tok/sat most about 137 tok/sat most about 193 tok/s
qwen3:14b (9.3GB)at most about 54 tok/sat most about 76 tok/sat most about 108 tok/s
qwen3:32b (20GB)does not fitdoes not fitat most about 50 tok/s

The ratio between the cards is the same for every model that fits: the RTX 4080's ceiling is about 1.42 times the RTX 4070's, and the RTX 4090's is about 1.41 times the RTX 4080's. Where a model does not fit, the bound does not apply, because the layers left in system RAM run at system-memory speed.

All three cards fit 8B models. Speed follows bandwidth (504, 716 and 1,008 GB/s), but the only measured figure here is the 4090's 127.7 tok/s on Llama 3 8B Q4_K_M from the XiongjieDai llama.cpp runs. At 32B only the 4090 holds the model in VRAM, so the comparison is moot for the other two cards.

Power and Running Cost: 450W vs 200W

The RTX 4090 draws approximately 450W under LLM load. The RTX 4070 draws approximately 200W. That 250W difference adds up over sustained use — and it affects PSU requirements and electricity bills.

Estimated GPU energy at 8 hours/day of full board power (NVIDIA TGP)

RTX 4070 (200W)

~48 kWh/month

~584 kWh/year

RTX 4080 (320W)

~77 kWh/month

~934 kWh/year

RTX 4090 (450W)

~108 kWh/month

~1,314 kWh/year

At 8 hours a day the RTX 4090 uses about 730 kWh a year more than the RTX 4070 at full board power. Multiply by your local electricity rate; real inference draw is often below TGP.

PSU requirements also differ: the RTX 4090 needs at minimum a 850W PSU with a modern CPU. NVIDIA lists 750W required system power for the RTX 4080. The RTX 4070 runs comfortably on 650W. If you are upgrading and your PSU is borderline, factor in upgrade cost for the higher-draw cards.

The 70B Question: None of These Cards Can Do It

If your goal is running 70B-class models such as Llama 3.3 70B, none of the three cards in this comparison holds one in VRAM. Here is why:

Llama 3.3 70B Q4_K_M needs 44 GB by the site formula

70 x 0.5 x 1.2 + 2 = 44 GB at Q4_K_M, before long-context KV cache. The RTX 4090 tops out at 24 GB, 20 GB short. GPU plus CPU offloading runs, but this site has not tested how fast on these cards.

A 72B model needs 45.2 GB

72 x 0.5 x 1.2 + 2 = 45.2 GB at Q4_K_M. None of the three cards comes close, and offloading speed on these cards has not been tested here.

For 70B, you need a Mac or dual-GPU setup

Llama 3.3 70B at Q4 needs 44 GB. A 48 GB Mac holds it with very little room; a 64 GB Mac Studio is more comfortable. Two 24 GB GPUs (48 GB combined) also work but need a board with two suitable slots and careful configuration. No single consumer NVIDIA GPU in this comparison runs it.

What the RTX 4090 does unlock at 24 GB

The jump from 16 GB to 24 GB is what fits 32B models: Qwen3 32B and DeepSeek-R1-Distill-32B at Q4 are 21.2 GB by the site formula, and Gemma 3 27B at Q4 is 18.2 GB. If 32B is your target, the 4090 is the card of these three. If 70B is your target, look at 64 GB-class unified-memory machines instead.

Price-to-Performance Analysis

Raw price-per-token-per-second is one lens, but the model compatibility cliff matters more. A card that runs a model 30% slower but actually runs the model is infinitely better than one that cannot run it at all.

GPUPriceBandwidth (catalog)Unlocks 32B?
RTX 4070 12GB Check price on Amazon 504 GB/s No
RTX 4080 16GB Check price on Amazon 716 GB/s No
RTX 4090 24GB Check price on Amazon 1008 GB/s Yes

We do not publish price-per-token math here because prices move and we have not measured all three cards. The decision framework: pick the RTX 4070 if 8B–14B at Q4 is your ceiling, the RTX 4080 if you want 14B with a 32K context or 20B at Q4 (14.0 GB), and the RTX 4090 if you want 32B at Q4 (21.2 GB).

Who Should Buy Which GPU

Buy the RTX 4070 12GB if...

  • ✓ You are on a tight budget
  • ✓ You primarily run 7B-14B models (Llama 3.1 8B, Qwen3 14B, Phi-4 14B)
  • ✓ Q4 quality is acceptable — you do not need Q8 near-lossless on 14B
  • ✓ You want low power draw (~200W) and a quiet, efficient build
  • ✓ You are new to local LLMs and want to try without heavy investment

Avoid if: you want Qwen3 32B, DeepSeek-R1-Distill-32B, or Gemma 3 27B — none fit in 12 GB.

Buy the RTX 4080 16GB if...

  • ✓ You want a 14B at Q4 with a 32K context
  • ✓ You run 20B to 22B models at Q4 (14.0 to 15.2 GB by the site formula)
  • ✓ You want more bandwidth than the 4070 (716 vs 504 GB/s, about 42% more)
  • ✓ You want the middle VRAM tier of these three
  • ✓ You do not specifically need 32B models

Avoid if: you need Qwen3 32B or DeepSeek-R1-Distill-32B at Q4. They are 21.2 GB by the site formula, 5.2 GB over 16 GB.

Buy the RTX 4090 24GB if...

  • ✓ You want to run Qwen3 32B, DeepSeek-R1-Distill-32B, or Gemma 3 27B
  • ✓ You want the most bandwidth of these three cards (1008 GB/s)
  • ✓ You run multiple users or applications against the same local GPU
  • ✓ You do heavy long-context inference where speed compounds over generations
  • ✓ Budget is secondary to capability

Avoid if: your only goal is 70B models — the 4090 still cannot run them. Budget goes further on Apple Silicon for 70B.

Running LLMs on Any of These Cards: Ollama Setup

All three cards are fully supported by Ollama and LM Studio. NVIDIA GPU detection is automatic — no special configuration needed.

Install Ollama

Windows and macOS

Download from ollama.com. Installs as a background service, auto-detects NVIDIA GPUs.

Linux

curl -fsSL https://ollama.com/install.sh | sh

RTX 4070 12GB — recommended models

Qwen3 14B Q4_K_M: 10.9 GB by the formula, 9.3GB download, about 15,800 tokens of f16 context

ollama run qwen3:14b

Qwen3 8B Q4_K_M: 6.8 GB by the formula, 5.2GB download, room for long context

ollama run qwen3:8b

Qwen3 8B Q8_0: 11.6 GB by the formula, 8.9GB download, short context only

ollama run qwen3:8b-q8_0

RTX 4080 16GB: models that fit

Qwen3 14B Q4_K_M: room for its full 32K context (14.8 GB by the KV method above; raise the context length in Ollama to use it)

ollama run qwen3:14b

Qwen3 8B Q8_0: 11.6 GB by the formula, 8.9GB download, 4.4 GB spare

ollama run qwen3:8b-q8_0

RTX 4090 24GB — recommended models

Qwen3 32B Q4_K_M: 21.2 GB by the formula, 20GB download, short context

ollama run qwen3:32b

DeepSeek-R1-Distill-32B Q4_K_M

ollama run deepseek-r1:32b

Qwen3 14B Q8_0: 19.8 GB by the formula, 16GB download

ollama run qwen3:14b-q8_0

Verify GPU load after starting a model

ollama ps

GPU% should show 100% for full VRAM inference. If it shows partial %, some layers are on CPU and speed will be significantly slower. Reduce context length or quantization if the model is spilling to CPU.

Frequently Asked Questions

Which NVIDIA GPU is best for running LLMs locally?

The RTX 4090 (24 GB, 1008 GB/s) is the only one of the three that fits 32B at Q4 (21.2 GB). The RTX 4080 16GB (716 GB/s) fits 14B at Q4 (10.4 GB) with a 32K context and 20B at Q4 (14.0 GB). The RTX 4070 12GB (504 GB/s) fits 8B easily and 14B at Q4 with little context room. Pick by the largest model you want to run.

Can the RTX 4070 run Llama 3 70B?

No. Llama 3.3 70B at Q4_K_M is 44 GB by the site formula (70 x 0.5 x 1.2 + 2, Q4_K_M, before long-context KV cache). The RTX 4070 has 12 GB, the RTX 4080 16 GB and the RTX 4090 24 GB, so none of the three holds it in VRAM. A 48 GB Mac has about 44 GB usable after macOS, which is the edge; a 64 GB Mac Studio has about 60 GB usable and holds it. Splitting the model between GPU and system RAM runs, but this site has not tested how fast.

Is the RTX 4090 worth it over the RTX 4080?

Yes, if you want 32B models: the 4090's 24 GB holds Qwen3 32B and DeepSeek-R1-Distill-32B at Q4 (21.2 GB, Q4_K_M, site formula, before long-context KV cache), which 16 GB cannot. It also has about 41% more memory bandwidth than the RTX 4080 (1,008 vs 716 GB/s), which raises the ceiling on token speed; this site has not tested the gap. If you only run 14B models, the 4080 fits them with a 32K context.

Which of these cards runs Qwen3 14B?

All three run it at Q4_K_M. The model card lists 14.8B parameters, which is 10.9 GB by the site formula at a short context, and Ollama lists the qwen3:14b download at 9.3GB. On the 12 GB RTX 4070 that leaves room for about 15,800 tokens of f16 KV cache (worked example above). Q8 is different: the formula gives 19.8 GB and Ollama lists qwen3:14b-q8_0 at 16GB, so Q8 needs the 24 GB RTX 4090, not a 16 GB card.

Is the RTX 4080 faster than the RTX 4090 for 7B and 8B models?

No. The RTX 4090's 1,008 GB/s is about 41% more memory bandwidth than the RTX 4080's 716 GB/s, and token generation is limited mostly by bandwidth, so the 4090 has the higher ceiling on any model both cards hold. For qwen3:8b (a 5.2GB download on Ollama), bandwidth divided by file size gives upper bounds of about 193 tok/s on the 4090 and 137 tok/s on the 4080. Those are ceilings, not test results.

How much context fits with a 14B model on a 12 GB card?

For Qwen3-14B at Q4_K_M, the weights are 14.8 x 0.5 = 7.4 GB and the site keeps 2 GB in reserve, which leaves 2.6 GB of a 12 GB card for the KV cache. At 163,840 bytes per token in f16 (40 layers, 8 KV heads, head dimension 128) that is about 15,800 tokens. An 8-bit cache roughly doubles it to about 31,700 tokens. A 16 GB card holds the model's full 32,768-token window at f16.

Why does this page compare the RTX 4080 and not the RTX 4080 Super?

The RTX 4080 16GB is the card in our hardware catalog, at 716 GB/s. NVIDIA's spec page lists the same 16 GB of GDDR6X, 256-bit memory interface, 320 W total graphics power and 750 W required system power for both cards, so every VRAM fit on this page applies to either one. The bandwidth ceilings use the RTX 4080's 716 GB/s.

Where to find the RTX 4070 and RTX 4080 compared above

Fit notes repeat the site formula estimates from the tables above, at Q4 and a short context unless a context length is given.

NVIDIA GeForce RTX 4070 (12 GB)

12 GB at 504 GB/s. Fits 14B at Q4 (10.4 GB by the site formula) with up to about 15,800 tokens of f16 context for Qwen3-14B, and an 8B at Q8 (11.6 GB) with little room left. Does not fit 20B at Q4 (14.0 GB) or 32B at Q4 (21.2 GB).

GIGABYTE RTX 4070 WINDFORCE OC 12 GB on Amazon

NVIDIA RTX 4080 (16 GB)

16 GB at 716 GB/s. Fits Qwen3-14B at Q4 with its full 32,768-token context (14.8 GB) and a 20B at Q4 (14.0 GB). Does not fit 14B at Q8 (18.8 GB), Gemma 3 27B at Q4 (18.2 GB) or 32B at Q4 (21.2 GB).

MSI RTX 4080 16 GB Gaming X Trio on Amazon

Find the right GPU for your LLM goals and check which models fit your hardware.

Sources & methodology

Which models fit the RTX 4070, 4080 and 4090 here comes from the site VRAM formula and the model cards listed below. The formula and the sitewide source list are on the methodology page. For this guide we used:

Spot a number that does not match the linked source? Contact us and we will update the guide.