Key fact: Llama 3.3 70B at Q4_K_M needs about 44 GB; two RTX 4090s (48 GB) leave 4 GB and a Mac Studio M5 Max 64 GB leaves about 16 GB.
How we got it: 70 x 0.5 x 1.2 + 2 = 44 (methodology). 48 - 44 = 4. Mac: 64 - 4 GB macOS reserve (catalog convention) = 60 usable; 60 - 44 = 16. Ollama lists 70.6B parameters, which gives 44.4 GB. Multi-GPU overhead is not modeled.
Checked:
Key fact: A 32K-token f16 KV cache for Llama 3.3 70B is about 10.7 GB, so 70B Q4 at 32K is about 54.7 GB: it fits a 64 GB Mac Studio, not two 24 GB cards unless the cache is q4_0 (about 46.7 GB).
How we got it: Meta sku_list.py: 80 layers, 8 KV heads, dim 8192 / 64 heads = head size 128. 2 x 80 x 8 x 128 x 2 = 327,680 bytes per token x 32,768 = 10.7 GB. 44 + 10.7 = 54.7. Ollama FAQ: q8_0 about 1/2 (49.4 GB), q4_0 about 1/4 (46.7 GB) of f16.
Checked:
Key fact: The bandwidth ceiling for qwen3:32b (20GB file) is about 90 tok/s on the RTX 5090 and about 50 tok/s on the RTX 4090. These are upper bounds, not runs.
Key fact: 32B at Q5_K_M is 26.0 GB by the formula and fits only the RTX 5090; Qwen's Q5_K_M file is 23.2 GB.
How we got it: 32 x 0.625 x 1.2 + 2 = 26.0; 24 GB < 26.0 < 32 GB. File size from the Qwen/Qwen3-32B-GGUF file list.
Checked:
Methodology
explains the VRAM formula, the bytes per weight at each quantization, and the full source list.
Updated 4 November 2026 · RTX 5090 32 GB GDDR7 vs RTX 4090 24 GB GDDR6X · Listed bandwidth is 1,792 vs 1,008 GB/s.
The RTX 5090 has 32 GB VRAM with 1,792 GB/s memory bandwidth. The RTX 4090 has 24 GB VRAM with 1,008 GB/s. 1,792 / 1,008 − 1 is about 78% more listed bandwidth. A 32B model at Q4 is 21.2 GB and fits both cards. A 32B model at Q6 is 32 × 0.75 × 1.2 + 2 = 30.8 GB and fits the 5090 only. Neither card fits a 70B model at Q4 (44 GB). LFM2.5-24B-A2B Q4 is 16.4 GB and fits both. Q8 is 30.8 GB and fits the 5090 only.
Neither card has a buy link on this page; the Where to buy row under the quick verdict explains why and names a 70B option. The 16 GB cards miss LFM2.5-24B-A2B Q4 (16.4 GB) and miss a 27B Q4 (18.2 GB).
Quick Verdict
RTX 5090 — Maximum Speed
32 GB GDDR7
About 78% more listed bandwidth (not a new bench)
32B at Q6 is 30.8 GB — 5090 only
Fits 32B Q5_K_M (26.0 GB)
575 W TGP; NVIDIA asks for a 1000 W PSU
8 GB more headroom for context
RTX 4090 — Lower cost
24 GB GDDR6X
Cheaper than 5090
Handles 32B at Q4_K_M (21.2 GB)
127.7 tok/s on Llama 3 8B Q4_K_M (XiongjieDai, May 2024)
450 W TGP; NVIDIA asks for an 850 W PSU
Proven workhorse since 2022
Want 70B at Q4?
Neither card alone
70B Q4_K_M = 70 x 0.5 x 1.2 + 2 = 44 GB
Two RTX 4090s: 48 GB, 4 GB left
Mac Studio M5 Max 64 GB: about 60 GB usable, 16 GB left
RTX 5090 = 78% more bandwidth and 32 GB, so 32B at Q5/Q6. RTX 4090 = 32B at Q4 with short context, and costs less. The 5090 is worth it for heavy 32B users. The 4090 is enough if 32B at Q4 meets your needs.
Where to buy
Neither card has a buy link on this page. Our hardware catalog listed both Amazon listings as unavailable when it was last checked on 17 October 2026, so the cards are named here without links. This page does not show prices or stock.
Option
Memory and bandwidth
What it holds, site formula
Buy link
NVIDIA GeForce RTX 5090 (32GB)
32 GB GDDR7, 1,792 GB/s (NVIDIA)
32B at Q5_K_M (26.0 GB) with 6.0 GB left. Not 70B at Q4 (44 GB).
None. Catalog listing (ASUS TUF RTX 5090 32GB OC) unavailable on 17 October 2026.
NVIDIA GeForce RTX 4090 (24GB)
24 GB GDDR6X, 1,008 GB/s (NVIDIA)
32B at Q4_K_M (21.2 GB) with 2.8 GB left. Not 32B at Q5_K_M (26.0 GB).
None. Catalog listing (MSI Gaming X Trio) unavailable on 17 October 2026.
Apple Mac Studio (M5 Max, 64GB)
64 GB unified memory, 614 GB/s (Apple)
70B at Q4_K_M (44 GB) in about 60 GB usable after macOS, 16.0 GB left.
None. Configured at Apple; our catalog has no Amazon listing for this memory size.
Where the extra 8 GB goes: long context on 32B models
Both cards hold a 32B model at Q4, which is about 21.2 GB by our formula. The difference shows up once the context window grows. The KV cache, the memory a model uses to hold the conversation so far, grows with every token. Qwen3-32B has 64 layers, 8 key-value heads and a head size of 128, so a 32K-token window takes about 8.6 GB of cache at the default f16 precision.
That brings the total to about 30 GB, which fits the RTX 5090's 32 GB and misses the RTX 4090's 24 GB. Ollama's FAQ says a q8_0 cache uses about half the memory of f16, which still leaves the total near 25.5 GB. A q4_0 cache uses about a quarter, for about 23.3 GB, and fits 24 GB at the edge with a larger loss in precision.
If you work with long documents or long coding sessions on 32B models, that is the case for the 5090's capacity. For short chats, the 4090 runs the same models. The full math is in the KV cache quantization guide.
Spec Comparison: RTX 5090 vs RTX 4090
Spec
RTX 5090
RTX 4090
VRAM
32 GB GDDR7
24 GB GDDR6X
Memory Bandwidth
1,792 GB/s (NVIDIA)
1,008 GB/s (NVIDIA Ada whitepaper)
Bandwidth Ratio
1.78x more bandwidth
baseline
Total Graphics Power
575 W
450 W
NVIDIA required system power
1000 W
850 W
Architecture
Blackwell
Ada Lovelace
Best 32B quant that fits
Q5_K_M (26.0 GB)
Q4_K_M (21.2 GB)
Largest at Q4_K_M
32B (21.2 GB) with long context headroom
32B (21.2 GB), short context
The 5090's 1,792 GB/s bandwidth is 1.78x the 4090's 1,008 GB/s. At batch size 1, token generation is mostly limited by memory bandwidth, so the 5090's speed ceiling is 1.78 times the 4090's on any model both cards hold. This page has no RTX 5090 speed from a real run to confirm it.
Speed: the bandwidth ceiling
This site does not run hardware, so this page has no tokens-per-second figure of its own for either card, and the community repo cited below has no RTX 5090 run. NVIDIA's specs do give a ceiling. At batch size 1, generating each token means reading the full set of weights from memory once, so tokens per second cannot be higher than memory bandwidth divided by the model file size. Reads of the KV cache, compute time and software overhead all keep real speeds below the ceiling.
Model file (size)
RTX 4090 ceiling at 1,008 GB/s
RTX 5090 ceiling at 1,792 GB/s
qwen3:14b, Q4_K_M (9.3GB)
about 108 tok/s
about 193 tok/s
qwen3:32b, Q4_K_M (20GB)
about 50 tok/s
about 90 tok/s
Qwen3-32B-Q5_K_M.gguf (23.2 GB)
does not fit
about 77 tok/s
Method: bandwidth from NVIDIA divided by file size from the Ollama library and Qwen's GGUF repository, read 30 October 2026. For example, 1,792 / 20 = 89.6. The ratio between the cards is 1.78 on every row, because only bandwidth changes. The one third-party speed on this page, 127.7 tok/s for the RTX 4090 on Llama 3 8B Q4_K_M, comes from the XiongjieDai repo's May 2024 snapshots.
VRAM Advantage: What 32 GB Unlocks vs 24 GB
The 5090's 32 GB does not double the model selection over the 4090's 24 GB — but it does push the 32B quality ceiling higher. The threshold is 32B at Q5_K_M (26.0 GB): the 5090 fits it, the 4090 does not.
What 24 GB covers (RTX 4090)
7–14B models at Q8 (14B Q8 is 18.8 GB)
32B models at Q4_K_M (21.2 GB), short context
27B models at Q4 (18.2 GB) with context room
No 70B: Q4 is 44 GB, Q2_K about 29.7 GB
Long context windows on 14B models
What 32 GB adds (RTX 5090)
32B models at Q5_K_M (26.0 GB) ✓ — key upgrade
32B at Q6 (30.8 GB) ✓ short context only
More headroom for long context on 32B
8 GB more for future model releases
The Q5_K_M threshold: 32B at higher quality
Q5_K_M for a 32B model (Qwen3 32B, DeepSeek R1 Distill 32B) needs 32 × 0.625 × 1.2 + 2 = 26.0 GB. That is 2 GB over the 4090's 24 GB and fits the 5090's 32 GB with room for context. Q6 (30.8 GB) also fits the 5090 at short context. Higher quants lose less quality than Q4; how much that matters depends on the task.
What neither card unlocks: 32B Q8 and 70B Q4
32B Q8_0 needs 40.4 GB — too large for the 5090's 32 GB. Llama 3.3 70B Q4_K_M needs 44 GB — also too large. 70B at Q4 needs two 24 GB cards (48 GB) or a Mac with 64 GB of unified memory; a 48 GB Mac leaves about 44 GB after macOS, exactly the estimate. The 5090 is a 32B card, not a 70B card.
Practical VRAM question
Ask: do I need 32B above Q4, or is Q4 good enough? Q4_K_M on a 32B model is the common default. Higher quants matter more for long-form writing, complex code generation, or benchmarking. If Q4 is enough, the 4090 runs it.
Model fit: both cards and the 70B alternatives
One table for every model size on this page. The site formula column is parameters x bytes per parameter x 1.2 + 2 GB, which covers about 4K to 8K tokens of context. The file column is the download size from the Ollama library or Qwen's GGUF repository, read on 30 October 2026. Files are smaller than the formula because the formula adds runtime overhead and a 2 GB reserve.
Model and quant
Site formula
File
RTX 4090 (24 GB)
RTX 5090 (32 GB)
Two RTX 4090s (48 GB)
Mac Studio M5 Max 64 GB (about 60 GB usable)
Qwen3 14B Q4_K_M
10.4 GB
9.3GB (qwen3:14b)
Fits, 13.6 GB left
Fits, 21.6 GB left
Fits on one card
Fits
Qwen3 14B Q8_0
18.8 GB
16GB (qwen3:14b-q8_0)
Fits, 5.2 GB left
Fits, 13.2 GB left
Fits on one card
Fits
Qwen3 32B Q4_K_M
21.2 GB
20GB (qwen3:32b)
Fits, 2.8 GB left, short context
Fits, 10.8 GB left
Fits on one card
Fits
32B Q5_K_M (Qwen3 32B, DeepSeek R1 Distill 32B)
26.0 GB
23.2 GB (Qwen3-32B-Q5_K_M.gguf)
Does not fit
Fits, 6.0 GB left
Fits across both, 22.0 GB left
Fits
Qwen3 32B Q6_K
30.8 GB
26.9 GB (Qwen3-32B-Q6_K.gguf)
Does not fit
Fits, 1.2 GB left, short context
Fits across both, 17.2 GB left
Fits
Qwen3 32B Q8_0
40.4 GB
35GB (qwen3:32b-q8_0)
Does not fit
Does not fit
Fits across both, 7.6 GB left
Fits, 19.6 GB left
Llama 3.3 70B Q2_K
29.7 GB
26GB (llama3.3:70b-instruct-q2_K)
Does not fit
Fits, 2.3 GB left, short context
Fits across both, 18.3 GB left
Fits
Llama 3.3 70B Q4_K_M
44.0 GB
43GB (llama3.3:70b)
Does not fit
Does not fit
Fits across both, 4.0 GB left
Fits, 16.0 GB left
Llama 3.3 70B Q8_0
86.0 GB
75GB (llama3.3:70b-instruct-q8_0)
Does not fit
Does not fit
Does not fit
Does not fit
How to read it: Q6_K and Q2_K use 0.75 and 0.33 bytes per parameter, the values used elsewhere on this page; they are not in the methodology table. "Short context" marks rows with less than 3 GB left, where a long conversation will not fit. The two-card column adds 24 GB + 24 GB. The methodology does not model multi-GPU setups, so read that column as the most the pair can hold. The Mac column takes 4 GB off for macOS, as in our hardware catalog. A 70B at Q8 needs a 128 GB machine.
Sizing the 70B alternatives
Neither card holds a 70B model at Q4. Llama 3.3 70B at Q4_K_M is 70 x 0.5 x 1.2 + 2 = 44 GB by the site formula, and Ollama's llama3.3:70b download alone is 43GB. Two setups in this comparison clear that line.
Two RTX 4090s (48 GB in total). The pair has 4 GB more than the 44 GB estimate. Ollama's FAQ says a model that does not fit on one GPU is spread across all available GPUs, so the model loads without extra setup. The site formula assumes one memory pool and does not model multi-GPU overhead, so read the 4 GB as the most you have: enough for the 4K to 8K context the formula covers, not for long documents.
Apple Mac Studio with M5 Max and 64 GB. After a 4 GB reserve for macOS, about 60 GB is usable, so 70B at Q4_K_M fits with about 16 GB left. Apple lists 614 GB/s for the M5 Max with an 18-core CPU and 40-core GPU, the only M5 Max configuration offered with 64 GB. It is configured at Apple, and our catalog has no Amazon listing for it. A 48 GB Mac leaves about 44 GB after macOS, exactly the estimate, so this page does not count it as a fit.
Worked example: 70B with a 32K-token context. Meta's model definition for Llama 3.3 70B lists 80 layers and 8 key-value heads, with a head size of 8,192 / 64 = 128. With the methodology's KV formula at f16, that is 2 x 80 x 8 x 128 x 2 = 327,680 bytes per token, or about 10.7 GB for 32,768 tokens. Added to the 44 GB estimate, the total is about 54.7 GB.
Mac Studio M5 Max 64 GB: fits, with about 5.3 GB left of the 60 GB usable.
Two RTX 4090s: 54.7 GB is over 48 GB. A q8_0 cache, which Ollama's FAQ puts at about half of f16, gives about 49.4 GB and is still over. A q4_0 cache, about a quarter, gives about 46.7 GB and fits with 1.3 GB left, at a larger precision loss.
One RTX 5090: no. The 44 GB estimate is over 32 GB before any cache.
Speed ceiling on the Mac. At 614 GB/s, reading a 43GB file once per generated token caps the Mac Studio at about 14 tokens per second on Llama 3.3 70B Q4_K_M (614 / 43 = 14.3). That is an upper bound from bandwidth, not a run, and real speeds are lower. This page gives no ceiling for two cards, because the formula does not model how a split model reads memory.
Price/Performance Analysis
At MSRP, the RTX 5090 is more expensive than the RTX 4090. At retail the gap widens further, and used 4090s are available for less.
RTX 5090 — where it wins
78% more bandwidth (NVIDIA), so a 78% higher speed ceiling on the same model
32B Q5_K_M (26.0 GB) fits only in 32 GB
Better for long sessions where speed compounds
More VRAM headroom for expanding context windows
Newer architecture — longer driver support horizon
RTX 4090 — where it wins
Cheaper depending on where you buy
Runs 32B at Q4_K_M (21.2 GB), short context
Available used at a lower price
Proven, widely supported since 2022
The upgrade math
The 5090 costs more for 78% more bandwidth and 8 GB more memory. If you generate a lot of text daily, the higher ceiling adds up; the speed table above gives the ceilings, and this site has no run on either card to show where real speeds land. For occasional or light use, the 4090 is the rational buy.
Who Should Buy Each Card
Use Case
Buy
Reason
7B–14B models for daily use
Either
Both fit 14B at Q8 (18.8 GB); the 5090 has 78% more bandwidth
32B models at Q4
Either
Both fit Q4_K_M (21.2 GB); 4090 at short context
32B models at Q5/Q6
RTX 5090
Q5_K_M is 26.0 GB, Q6 is 30.8 GB — only the 32 GB card fits
Maximum generation speed
RTX 5090
1,792 vs 1,008 GB/s bandwidth (NVIDIA)
Lower cost
RTX 4090
Cheaper; handles 32B at Q4_K_M (21.2 GB)
70B models at Q4
Two RTX 4090s or a 64 GB Mac Studio
70B Q4 is 44 GB: 4 GB left on 48 GB, about 16 GB on the Mac
When the RTX 5090 is the right call
The 5090 earns its premium when you live at 32B. If your workflow is heavy 32B reasoning or coding where Q5/Q6 quality over Q4_K_M matters, you need the 32 GB ceiling — the 4090 cannot hold 32B Q5_K_M (26.0 GB).
Stick with RTX 4090 if:
Q4_K_M quality on 32B models is enough
You want to save money on hardware
You already own a 4090 — upgrade ROI is poor
Speed is nice but not a blocker
You can find one used at a good price
Running LLMs: Recommended Setup for Both Cards
Both the RTX 5090 and RTX 4090 are fully supported by Ollama and LM Studio. NVIDIA GPU detection is automatic. The only practical difference is which VRAM tier you land in.
Install Ollama
Windows and macOS
Download from ollama.com. Installs as a background service, auto-detects NVIDIA GPUs.
The PROCESSOR column should read 100% GPU, which Ollama's FAQ says means the model is loaded entirely into the GPU. A split such as 48%/52% CPU/GPU means part of the model is in system memory.
Frequently Asked Questions
Is the RTX 5090 worth the upgrade over RTX 4090 for local LLMs?
The RTX 5090 has 32 GB and 1,792 GB/s; the RTX 4090 has 24 GB and 1,008 GB/s (NVIDIA). That is about 78% more bandwidth and 8 GB more memory. The 32 GB lets the 5090 run 32B at Q5_K_M (26.0 GB) or Q6 (30.8 GB, short context); the 4090 tops out at 32B Q4_K_M (21.2 GB). The 5090 is worth it for heavy 32B use. If 32B at Q4 is enough, the 4090 costs less.
Can the RTX 5090 run 70B models?
Not at Q4. Llama 3.3 70B at Q4_K_M is 44 GB by the site formula (70 x 0.5 x 1.2 + 2), and Ollama's llama3.3:70b file alone is 43GB, both over the 5090's 32 GB. A Q2_K 70B (about 29.7 GB) fits the 5090 only at short context; the 4090 cannot hold it. For 70B at Q4, two 24 GB cards give 48 GB, 4 GB over the estimate, and a Mac Studio M5 Max with 64 GB has about 60 GB usable after macOS, about 16 GB over.
Two RTX 4090s or a 64 GB Mac Studio for 70B models?
By the site formula, 70B at Q4_K_M is 44 GB. Two RTX 4090s give 48 GB, and Ollama spreads a model that does not fit one GPU across all available GPUs. The methodology does not model multi-GPU overhead, so 4 GB is the most you have left, enough for a short context. A Mac Studio M5 Max with 64 GB has about 60 GB usable after macOS, which holds the 44 GB model plus a 32K-token cache (about 10.7 GB for Llama 3.3 70B at f16), about 54.7 GB in total. Each 4090 has 1,008 GB/s and the M5 Max has 614 GB/s, but the formula does not model how a split model reads memory, and this page has no speed from a real run for either setup.
What models does the RTX 5090's 32 GB unlock that the 4090 cannot run?
The 5090’s 32 GB fits 32B at Q5_K_M (26.0 GB) and Q6 (30.8 GB, short context), including DeepSeek R1 Distill 32B. The 4090’s 24 GB fits 32B only at Q4_K_M (21.2 GB) with a short context. Neither card fits 32B at Q8 (40.4 GB) or 70B at Q4 (44 GB).
How much faster is the RTX 5090 than RTX 4090 for token generation?
This page has no RTX 5090 speed from a real run; this site does not run hardware. NVIDIA lists 1,792 GB/s for the 5090 and 1,008 GB/s for the 4090, about 78% more. At batch size 1 a token cannot be generated faster than the weights can be read, so the ceiling for qwen3:32b, a 20GB file, is about 90 tok/s on the 5090 and about 50 tok/s on the 4090. Real speeds are lower than these ceilings.
What is the best quantization for 32B models on RTX 5090 vs RTX 4090?
On the RTX 5090 (32 GB): 32B at Q5_K_M (26.0 GB) with room for context, or Q6 (30.8 GB) at short context. On the RTX 4090 (24 GB): 32B at Q4_K_M (21.2 GB); Q5_K_M (26.0 GB) does not fit. The 5090’s practical gain on 32B is one or two quantization levels higher.
The 32 GB and 24 GB fit figures for the RTX 5090 and RTX 4090 use the site VRAM formula and model cards. The formula and the sitewide source list are on the methodology page. For this guide we used:
XiongjieDai GPU-Benchmarks-on-LLM-Inference. Community llama-bench results (May 2024 snapshots); source of the 4090's 127.7 tok/s on Llama 3 8B Q4_K_M. It has no RTX 5090 run.
Ollama FAQ. Multi-GPU loading, KV cache types, context length and ollama ps.
Spot a number that does not match the linked source? Contact us and we will update the guide.
We use cookies and personalised ads
This site uses cookies and shows personalised ads via Google AdSense. We and our partners store and access information on your device to serve relevant ads and improve your experience.
You can accept all cookies, decline (non-personalised ads only), or
manage preferences.
See our Privacy Policy.
We use Google Analytics
This site would like to use Google Analytics to understand how visitors use the site. If you accept, Google Analytics will load. If you decline, or if you make no choice, Google Analytics will not load. No ads are shown on this visit.
See our Privacy Policy and Cookie Policy.
Cookie preferences
Choose which cookies you allow. Strictly necessary cookies are always active.
Strictly necessary
Session state, security, and performance. Cannot be disabled.