Key fact: llama.cpp has --cpu-moe and --n-cpu-moe N to keep MoE expert weights on the CPU.
How we got it: llama-server README, checked 14 October 2026.
Checked:
MoE Hybrid GPU and CPU Offloading with llama.cpp
By LLM Hardware team · Methodology ·
Key facts
- Flags
- --cpu-moe, --n-cpu-moe N in llama-server
- Sizing
- MoE memory follows total params; 30B-A3B Q4 is 20.3 GB
- Checked
- 2026-10-13
Checked 2026-10-13. Author: LLM Hardware team. This site does not store or show Amazon prices. Check current price on Amazon.
Citeable facts
Key fact: Qwen3-30B-A3B has 30.5B total and 3.3B active parameters; Qwen3-235B-A22B has 235B and 22B.
How we got it: Qwen model cards.
Checked:
Key fact: MoE memory follows total parameters: Qwen3-30B-A3B at Q4 is about 20.3 GB.
How we got it: VRAM formula: 30.5 x 0.6 + 2.
Checked:
Methodology — parameter math, quantization bytes, and the source list.
A mixture-of-experts (MoE) model uses only a few experts per token, but all of them have to sit somewhere in memory. llama.cpp can keep the expert weights in system RAM and the rest of the model on the GPU, so a model too big for your VRAM still runs, more slowly.
The llama.cpp flags
The llama-server README lists --cpu-moe (keep all MoE weights on the CPU) and --n-cpu-moe N (keep the experts of the first N layers on the CPU). Start with all layers on the GPU (-ngl 99), then raise N until it loads.
llama-server -m Qwen3-30B-A3B-Q4_K_M.gguf -ngl 99 --n-cpu-moe 20 -c 16384
Memory by model
| Model | Total / active | Q4 by formula | Example split |
|---|---|---|---|
| Qwen3-30B-A3B | 30.5B / 3.3B | 20.3 GB | 12-16 GB GPU plus 32 GB RAM |
| gpt-oss-120b | 117B / 5.1B | OpenAI: one 80 GB GPU | 24 GB GPU plus 64-96 GB RAM |
| Qwen3-235B-A22B | 235B / 22B | 143 GB | 24-32 GB GPU plus 128 GB or more RAM |
VRAM plus system RAM together must hold the weights plus the cache. Splits above are rough planning sizes from our VRAM formula, not measured results. A dense model gains much less from offload, because every weight is used for every token.
When another option is better
- If it fits in VRAM, keep it there: experts read from system RAM are slower. A 24 GB card runs Qwen3-30B-A3B at Q4 with no offload.
- For daily use of a 100B+ MoE, a 128 GB unified-memory machine avoids the PCIe and system-RAM bottleneck. See DGX Spark vs Mac Studio vs Strix Halo.
- Long context: quantize the cache before offloading more experts. See the KV-cache guide.
Hardware
Check current price on Amazon.
ASIN B08HR6ZBYJ. Amazon Associates link. The live price is on that page, not locked in here.
ASIN B0BG94PS2F. Amazon Associates link. The live price is on that page, not locked in here.
Sources
- llama.cpp llama-server README (cache types, --cpu-moe, --n-cpu-moe)
- Qwen3-30B-A3B model card (30.5B total, 3.3B active)
- gpt-oss-120b model card (117B total, 5.1B active)
- Qwen3-235B-A22B model card (235B total, 22B active)
- LLMHardware methodology: the VRAM formula
Questions
What does --n-cpu-moe do?
It keeps the MoE expert weights of the first N layers in system RAM, per the llama-server README, so the rest fits on the GPU.
Does offloading work for dense models?
Much less well. Every weight in a dense model is used for every token, so whatever sits in system RAM slows each token.
A correction or a hardware question goes to the contact page.