By LLM Hardware team · Methodology ·
Drafted with AI assistance. Tokens-per-second figures are cited to the XiongjieDai llama.cpp runs below; we did not test these cards in-house.
Checked 2026-10-13. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.
Best for: buyers who want 24 GB for 32B models at Q4 (21.2 GB) at used-market cost and accept no warranty.
| VRAM | 24 GB GDDR6X (source) |
|---|---|
| Memory bandwidth | 936 GB/s (source) |
| Llama 3 8B Q4_K_M generation | 111.7 tok/s (May 2024 runs) (source) |
| 2× RTX 3090, Llama 3 70B Q4_K_M | 16.3 tok/s (source) |
| Board power / NVIDIA PSU | 350 W / 750 W |
| RTX 3090 | RTX 4090 | |
|---|---|---|
| VRAM | 24 GB | 24 GB |
| Bandwidth | 936 GB/s | 1008 GB/s |
| Llama 3 8B Q4_K_M | 111.7 tok/s | 127.7 tok/s |
| Power / PSU | 350 W / 750 W | 450 W / 850 W |
Pairs with: A 750 W PSU (NVIDIA's figure; 1200 W+ for two cards), 32–64 GB system RAM, and a case for a 3-slot card. PSU, RAM and case sizing for an LLM build · VRAM calculator · Can I run it? · How much VRAM do I need?
Key fact: An RTX 3090 has 24 GB. A 32B model at Q4 is 21.2 GB and fits. A 14B model at Q8 is 18.8 GB and fits. An 8B model at FP16 is 21.2 GB and fits. A 70B model at Q4 is 44 GB and does not fit.
How we got it: VRAM_GB = params × bytes_per_param × 1.2 + 2. Q4 bytes = 0.5, Q8 = 1, FP16 = 2. Command for the 32B fit: ollama pull qwen3:32b.
Checked:
Key fact: NVIDIA lists 350W TDP and 936 GB/s for the RTX 3090, and 450W TDP and 1,008 GB/s for the RTX 4090. The 4090 spec draws more power, not 100W less.
How we got it: Spec rows on this page, matching NVIDIA’s published 3090 and 4090 tables. No new tokens-per-second run was taken on 13 October 2026.
Checked:
Methodology — parameter math, quantization bytes, and the source list.
Updated 24 October 2026 · LLM Hardware team · 24 GB VRAM · Ampere · 350W TDP
The RTX 3090 is a 24 GB card. That is the same capacity class as an RTX 4090, so a 32B model at Q4 (21.2 GB) fits and a 70B model at Q4 (44 GB) does not. A 16 GB card misses the 21.2 GB estimate. Listed bandwidth is 936 GB/s against 1,008 GB/s on the 4090, and listed TDP is 350W against 450W. The value case is a used card. The Amazon button below is a new Founders Edition listing, not that used estimate.
Buy on AmazonA 32B model at Q4 is 21.2 GB and fits this 24 GB card. A 70B model at Q4 is 44 GB and does not. Compare it with the VRAM calculator, the RTX 4090 guide, and how much VRAM you need.
TL;DR Verdict
A used RTX 3090 is the 24 GB capacity play. It fits the same 21.2 GB 32B Q4 estimate as a 4090. The 350W TDP means a strong PSU and airflow, and a used card still needs a mining-wear check. Check current price on Amazon. Buy used if you need 24 GB and the asking price is near that old estimate. Check current price on Amazon.
| Specification | Value |
|---|---|
| VRAM | 24 GB GDDR6X |
| Memory bandwidth | 936 GB/s |
| Memory bus | 384-bit |
| TDP | 350W |
| Architecture | Ampere (GA102) |
| CUDA cores | 10,496 |
| Released | September 2020 |
| CUDA support | Full (CUDA 8.6) |
| PSU requirement | 650W+ recommended |
Full hardware details: RTX 3090 hardware page.
Both cards have 24 GB of GDDR6X. Check current price on Amazon.
VRAM decides which model fits. Listed bandwidth is 936 GB/s on the 3090 and 1,008 GB/s on the 4090, about 8% less bandwidth on the spec sheet, not a measured tokens-per-second gap. This update did not recheck older third-party benches. A 32B model at Q4 is 21.2 GB on either card. A 70B model at Q4 is 44 GB on either card and does not fit.
The RTX 3090 uses the older Ampere architecture (GA102 die) versus the 4090's Ada Lovelace (AD102). Ampere still has excellent CUDA support across all LLM inference tools. There is no software disadvantage.
Key insight: If you need a 32B model at Q4, you need about 21.2 GB, so a 24 GB card fits and a 16 GB card does not. Bandwidth is not a reason to invent a speed percentage. Check current price on Amazon.
The RTX 4090 and RTX 3090 are the consumer NVIDIA cards in this 24 GB class. They fit the same 21.2 GB 32B Q4 estimate. The 4090 lists more bandwidth and a higher TDP (450W versus 350W). This page does not claim a measured speed gap or a current used-price multiple.
| Spec | RTX 3090 | RTX 4090 |
|---|---|---|
| VRAM | 24 GB GDDR6X | 24 GB GDDR6X |
| Bandwidth | 936 GB/s | 1,008 GB/s |
| TDP | 350W | 450W |
| Architecture | Ampere (GA102) | Ada Lovelace (AD102) |
| 32B Q4 fit | 21.2 GB, fits | 21.2 GB, fits |
| Max model at Q4_K_M | 32B | 32B |
| Value | Excellent | Good |
Bottom line: Both cards fit a 32B model at Q4 (21.2 GB) and miss a 70B model at Q4 (44 GB). The 3090 lists 350W and 936 GB/s. The 4090 lists 450W and 1,008 GB/s, so the 4090 spec draws more power, not 100W less. No new tokens-per-second run was taken for this update. Check current price on Amazon.
With 24 GB, a 32B model at Q4 is 21.2 GB and fits, and a 14B model at Q8 is 18.8 GB and fits. A 70B model at Q4 is 44 GB and does not fit. The command for the model that fits is ollama run qwen3:32b.
| Model | Quantization | VRAM Used |
|---|---|---|
| Qwen3 32B | Q4_K_M | 21.2 GB |
| Llama 3.3 70B | Q4_K_M | 44 GB, does not fit |
| Qwen3 14B | Q8 | 18.8 GB |
| DeepSeek-R1-Distill-32B | Q4_K_M | 21.2 GB |
| Llama 3.1 8B | FP16 | 21.2 GB |
Community speed results for this card are in the XiongjieDai community llama-bench runs and the Hardware Corner GPU ranking; this table lists memory fit only. VRAM figures include model weights plus ~1-2 GB for KV cache at moderate context lengths. Use the VRAM Calculator for exact numbers at your context length.
The RTX 4060 Ti 16GB and RTX 4070 Ti Super 16GB are the most common 16 GB alternatives. The RTX 3090 beats both on VRAM — giving 50% more capacity and the ability to run 32B vs 13B models.
| Spec | RTX 3090 (used) | 16 GB Cards (new) |
|---|---|---|
| VRAM | 24 GB | 16 GB |
| Max model at Q4_K_M | 32B (21.2 GB) | 14B (10.4 GB) |
| Max model at Q8 | 14B (18.8 GB) | 8B (11.6 GB) |
| Bandwidth | 936 GB/s | 288-672 GB/s |
| TDP | 350W | 165-285W |
| VRAM advantage | +50% more | baseline |
vs RTX 4060 Ti 16GB
The 3090 has 24 GB and 936 GB/s. The 4060 Ti 16GB has 16 GB and 288 GB/s. A 32B model at Q4 is 21.2 GB and fits the 3090. It misses 16 GB. A 14B model at Q4 is 10.4 GB and fits the 4060 Ti. This page does not claim they cost the same today, and it does not claim a new speed measurement. Check current price on Amazon.
vs RTX 4070 Ti Super 16GB
The 3090 has 24 GB, 936 GB/s, and 350W. The 4070 Ti Super has 16 GB, 672 GB/s, and 285W. A 32B model at Q4 is 21.2 GB and fits only the 3090. Check current price on Amazon.
The RTX 3090 launched in September 2020 and was popular with both gamers and cryptocurrency miners. Mining cards can have high fan hours and thermal stress. Here is what to verify before buying.
Check for mining usage
Download GPU-Z and check "Sensor" tab for fan hours and thermal history. High fan hours (100,000+) indicate heavy use. Ask the seller directly — many will be honest about mining history.
Founders Edition vs AIB cards
The NVIDIA Founders Edition and third-party (AIB) cards from ASUS, MSI, EVGA, and Gigabyte are all fine. AIB cards often have better cooling. There is no meaningful performance difference.
Where to buy
eBay, Craigslist, and Facebook Marketplace are the primary used GPU markets. Be skeptical of deals priced well below the going rate — they often indicate a problematic card. eBay offers buyer protection if the card is dead on arrival.
Power supply requirements
The RTX 3090 has a 350W TDP and uses a 12-pin or 8-pin+8-pin power connector depending on the card. You need a 650W+ PSU with good 12V rail capacity. A 750W PSU is comfortable. Ensure good case airflow — this card runs hot.
The RTX 3090 has full CUDA support (compute capability 8.6, Ampere) and works with every major LLM inference tool. Install the latest NVIDIA drivers and you are ready to go.
Ollama
Easiest setup — one install, GPU auto-detected. Run `ollama pull qwen3:32b` to download a 32B model and `ollama run qwen3:32b` to start chatting. Best for beginners.
LM Studio
GUI-based interface for browsing, downloading, and running GGUF models. Good for exploring models without the terminal. Automatically uses the RTX 3090 for inference.
llama.cpp
Compile with `-DLLAMA_CUDA=ON` for direct GPU inference. Most flexible for power users — supports all quantization formats and full control over layer offloading.
vLLM / text-generation-webui
For server setups and OpenAI-compatible API endpoints. Requires more configuration but enables multi-user access, batching, and integration with other tools.
For a step-by-step walkthrough, see the how to run LLMs locally guide. For context on VRAM and quantization, read the quantization explained guide.
Buy it if you...
Skip it if you...
VRAM calculator
32B Q4 is 21.2 GB. 70B Q4 is 44 GB and does not fit one 3090.
Best GPU for LLMs — Full Guide
Every budget tier covered
RTX 4070 Ti Super LLM Guide
16 GB — faster per watt, but less VRAM
RTX 4070 vs 4080 vs 4090
12 GB, 16 GB, and 24 GB compared with the same formula
LLM Quantization Explained
Q4 vs Q8 vs FP16 — when quality tradeoffs matter
What Can I Run? VRAM Guide
Models organized by VRAM tier — 8GB through 64GB
VRAM Calculator
Check if your model fits at your context length
Yes. 24 GB fits a 32B model at Q4, which is 32 × 0.5 × 1.2 + 2 = 21.2 GB, the same size class as an RTX 4090. A 70B model at Q4 is 44 GB and does not fit. Listed bandwidth is 936 GB/s. Check current price on Amazon.
The RTX 3090 has 24 GB of GDDR6X on a 384-bit bus and 936 GB/s of bandwidth. The RTX 4090 also has 24 GB and lists 1,008 GB/s. A 32B model at Q4 is 21.2 GB and fits. A 14B model at Q8 is 14 × 1.0 × 1.2 + 2 = 18.8 GB and fits. An 8B model at FP16 is 8 × 2.0 × 1.2 + 2 = 21.2 GB and fits. A 70B model at Q4 is 44 GB and does not fit.
Both cards have 24 GB and fit the same 21.2 GB estimate for a 32B model at Q4. NVIDIA lists 1,008 GB/s and 450W TDP for the RTX 4090, and 936 GB/s and 350W TDP for the RTX 3090. The 4090 spec draws more power, not 100W less. This page does not publish a new tokens-per-second measurement. Check current price on Amazon.
Yes. Ollama, LM Studio, and llama.cpp use the card through CUDA. Install the current NVIDIA driver, then run ollama pull qwen3:32b and ollama run qwen3:32b. Check the GPU column with ollama ps. Ollama v0.40.3, published 11 October 2026, did not change those commands. The card is Ampere, compute capability 8.6. A 32B model at Q4 is 21.2 GB. Do not pull llama3.3:70b for this card: that Q4 estimate is 44 GB.
When buying a used RTX 3090, check for mining usage: download GPU-Z and look for high fan hours or wear indicators. Shop eBay, Craigslist, and Facebook Marketplace — be wary of deals priced well below the going rate, which may indicate a damaged card. Both Founders Edition and AIB (third-party) cards are fine. Ensure your power supply can deliver at least 650W, as the RTX 3090 has a 350W TDP.
Check VRAM requirements for any model, or compare the RTX 3090 against other hardware.
VRAM figures on this page come from the site’s VRAM formula and model cards. The sitewide formula and the full source list are on the methodology page. For this guide we used:
Spot a number that does not match the linked source? Contact us and we will update the guide.