How to Run LLMs on Windows 10 and 11 with Ollama and LM Studio

By LLM Hardware team · Methodology ·

AI sketched the install order. The driver and WSL2 notes are general setup guidance; check the LM Studio and Ollama docs linked in Sources below for the current steps.

Key facts

NVIDIA path
Recent Game Ready or Studio driver, then Ollama or LM Studio
AMD and Intel
Vulkan or the vendor stack; do not assume CUDA
Checked
2026-10-13
Author
LLM Hardware team

Checked 2026-10-13. Author: LLM Hardware team. Amazon prices are not stored or shown on this site.

Citeable facts

Key fact: On Windows, qwen3:8b at Q4 is 6.8 GB. qwen3:14b at Q4 is 10.4 GB and fits 12 GB. A 14B model at Q8 is 18.8 GB and misses 16 GB. qwen3:32b at Q4 is 21.2 GB and fits 24 GB. Llama 3.3 70B at Q4 is 44 GB and does not fit a 24 GB card.

How we got it: VRAM_GB = params × bytes_per_param × 1.2 + 2. Q4 uses 0.5 bytes. Q8 uses 1 byte. Command syntax checked against the Ollama CLI docs and the qwen3 library tags on 10 November 2026: ollama pull qwen3:8b, ollama run qwen3:8b, then ollama ps. That is a documentation check, not a test run.

Checked:

Methodology explains the VRAM formula, the bytes per weight at each quantization, and the full source list.

Updated 10 November 2026 · LLM Hardware team · Ollama and LM Studio · NVIDIA, AMD, Intel Arc · Windows 10 and 11

To run an LLM on Windows 10 or 11, install Ollama from ollama.com/download/windows or LM Studio from lmstudio.ai. Both have native Windows installers and detect NVIDIA, AMD and Intel Arc GPUs once the driver is installed. In PowerShell, run ollama run qwen3:8b, then ollama ps: a PROCESSOR column reading 100% GPU means the model is on the card. Qwen3 8B at Q4 is 6.8 GB and fits an 8 GB GPU. A 32B model at Q4 is 21.2 GB and fits 24 GB. This guide covers both tools, which models to run for your VRAM tier, how to fix common Windows-specific issues, and optional extras like Open WebUI for a ChatGPT-style interface.

Quick Start: Running Your First Model in 2 Minutes

The fastest path to a working local LLM on Windows is Ollama. It runs in the background, detects supported GPUs once the driver is installed, and adds the ollama command to your PATH automatically.

  1. 1.

    Download and install Ollama for Windows

    Go to ollama.com/download/windows and run the installer. The installer does not need Administrator rights and installs in your home folder by default; Ollama then runs in the background. No Python, no virtual environments, no configuration files needed. After installation, open a new terminal window and the ollama command is immediately available.

  2. 2.

    Run your first model

    Open PowerShell or Command Prompt and run one of these commands. Ollama downloads the model on first run and caches it locally.

    8 GB VRAM: Qwen3 8B (6.8 GB at Q4)

    ollama run qwen3:8b

    12 GB VRAM: Qwen3 14B (10.4 GB at Q4)

    ollama run qwen3:14b

    No GPU / small VRAM — Phi-4-mini 3.8B (CPU-friendly)

    ollama run phi4-mini
  3. 3.

    Verify GPU is being used

    In a second terminal, run the command below. The PROCESSOR column should read 100% GPU when the whole model is on the card. Open Task Manager and click the GPU tab to see compute utilization spike during generation.

    ollama ps

System Requirements

Ollama supports Windows 10 22H2 or newer, Home or Pro, which includes Windows 11. LM Studio runs on x64 PCs with AVX2 and on ARM (Snapdragon X Elite) PCs, and its docs recommend at least 16 GB of RAM and 4 GB of dedicated VRAM. A discrete GPU is strongly recommended for any model larger than 3B parameters — CPU-only inference is functional but slow. Here are the minimum practical specs by use case.

Minimum (CPU only)

  • Windows 10/11
  • 16 GB RAM
  • No discrete GPU required
  • Models: Phi-4-mini, Qwen3 1.7B

Recommended (8 GB GPU)

  • Windows 10/11
  • 16 GB RAM
  • NVIDIA RTX 4060 8GB
  • Qwen3 8B Q4 is 6.8 GB

Ideal (12–24 GB GPU)

  • Windows 10/11
  • 32 GB RAM
  • RTX 4070 12GB or RTX 3090/4090 24GB
  • 14B Q4 is 10.4 GB. 32B Q4 is 21.2 GB. 70B Q4 is 44 GB

GPU Driver Requirements by Brand

GPU BrandRequired DriverDownloadNotes
NVIDIA GeForce Game Ready or Studio Driver nvidia.com/drivers Ollama Windows docs: 551.61 or newer. Ollama hardware support docs: cards with compute capability 5.0 to 6.2 need 570 or newer.
AMD Radeon ROCm v7 / HIP7-capable or Vulkan-capable driver amd.com/support Ollama uses ROCm where the driver supports it and Vulkan otherwise. Some RX 6000 cards need Vulkan.
Intel Arc (A/B series) Intel Arc Graphics - Windows driver intel.com/arc Covers Arc A-series and B-series. Ollama reaches Arc through Vulkan on Windows.
Integrated (Intel/AMD) Latest available Device Manager Very slow; small models only.

Keep GPU drivers updated. Ollama and LM Studio both release updates that may require newer driver features. When in doubt, update to the latest available driver before troubleshooting GPU detection issues.

Match your model to your VRAM. Running a model that exceeds your VRAM makes Ollama keep part of it in system RAM, and generation slows sharply. Always pick the largest quantization that fits fully in VRAM.

VRAMExample GPUsBest ModelsOllama CommandSpeed
8 GB RTX 4060 8GB Qwen3 8B Q4_K_M (6.8 GB) ollama run qwen3:8b not listed
12 GB Arc B580 12GB, RTX 4070 12GB Qwen3 14B Q4_K_M (10.4 GB), Phi-4 14B ollama run qwen3:14b not listed
16 GB RTX 4060 Ti 16GB, RTX 4070 Ti Super Qwen3 14B Q4_K_M (10.4 GB). 14B Q8 is 18.8 GB and misses 16 GB ollama run qwen3:14b not listed
24 GB+ RTX 4090, RTX 3090, RX 7900 XTX Qwen3 32B Q4_K_M (21.2 GB). Llama 3.3 70B Q4 is 44 GB and does not fit ollama run qwen3:32b not listed
CPU only 16+ GB RAM, no discrete GPU Qwen3 1.7B, Phi-4-mini 3.8B ollama run phi4-mini not listed

This page does not list speeds. Sizes are the site formula at Q4_K_M and short context, not measured allocations. Use the VRAM Calculator to check exact fit at your context length.

Ollama on Windows: Full Setup Details

Ollama's Windows installer handles everything: it installs per user without Administrator rights, runs Ollama in the background and makes the ollama command available in the terminal. Here is what you need to know beyond the quick start.

GPU detection: NVIDIA vs AMD vs Intel Arc

NVIDIA GPUs run through CUDA; the Ollama Windows docs ask for driver 551.61 or newer. AMD Radeon cards use ROCm v7 where the driver exposes it and Vulkan otherwise. Intel Arc and other GPUs use Vulkan, which Ollama enables by default when the backend is installed, and Windows GPU drivers usually include Vulkan support. After installing drivers, run "ollama run qwen3:8b", then "ollama ps", and check that the PROCESSOR column reads 100% GPU.

Model storage location on Windows

By default, Ollama stores models in C:\Users\%username%\.ollama\models. Models can be several gigabytes each. To use another drive, quit Ollama from the taskbar, open Settings (Windows 11) or Control Panel (Windows 10), search for environment variables, edit the variables for your account, add OLLAMA_MODELS with a path such as D:\ollama\models, then start Ollama again from the Start menu.

Running Ollama as a background API server

Ollama runs in the background after installation. It listens on http://localhost:11434 with an OpenAI-compatible API. This means any tool that supports the OpenAI API — VS Code extensions, Continue.dev, Cursor, Open WebUI — can connect to Ollama without any extra configuration. Just point the tool at http://localhost:11434 and it works.

Updating Ollama on Windows

To update, download the latest installer from ollama.com/download/windows and run it; the Ollama Windows docs say the installer helps you keep up to date.

LM Studio on Windows: GUI Alternative

LM Studio is a graphical desktop application for running local LLMs without any command-line work. It includes a model browser, built-in chat interface, and a GPU layers slider for fine-tuning how much VRAM to use. Ideal for beginners or anyone who prefers a visual interface.

1. Download LM Studio

Go to lmstudio.ai and download the Windows installer. Run it and follow the prompts — LM Studio installs like any standard Windows application. No dependencies required.

2. Find and download a model

Open the Discover tab and search for a model. For Qwen3 8B, search "qwen3" and look for "bartowski/Qwen3-8B-GGUF" — bartowski's quantizations are widely used. LM Studio shows file size and recommended VRAM next to each variant.

3. Configure GPU layers

When loading a model, LM Studio shows a GPU Layers slider. Drag it to the maximum value your VRAM supports. Higher layers = more work on GPU = faster generation. LM Studio estimates VRAM usage live as you adjust the slider.

4. Load a GGUF file from disk

Already have a GGUF model downloaded? Use My Models tab and click the import button to point LM Studio at any .gguf file on your system. Useful for models downloaded from Hugging Face or converted from other formats.

Ollama vs LM Studio: which should you use on Windows?

Use LM Studio if you want a visual interface, easy model browsing, and a built-in chat UI without touching the terminal. Use Ollama if you want a persistent API server for integrating with editors, scripts, or Open WebUI, or if you prefer working from the command line. Both run the same GGUF models and support the same GPUs. Many users install both — LM Studio for exploration, Ollama for daily use.

Fixing Common Windows Issues

CUDA not detected — model runs on CPU despite having an NVIDIA GPU

Usually a driver issue. Open Device Manager and verify your GPU is listed under Display Adapters without a warning icon. Update to the latest NVIDIA Game Ready or Studio Driver from nvidia.com. After updating, restart your machine and run "ollama run phi4-mini" — check Task Manager GPU tab during inference to confirm CUDA acceleration. If the GPU is still not detected, run "nvidia-smi" in a terminal; if that command fails, the driver is not installed correctly.

Model runs but generation is very slow

This is CPU offloading: the model does not fully fit in VRAM, so Ollama keeps part of it in system RAM. Run "ollama ps" and check the PROCESSOR column; a split such as 48%/52% CPU/GPU means part of the model is in system memory. Fix: switch to a smaller model or a smaller quantization. On an 8 GB GPU, Qwen3 14B at Q4 is 10.4 GB and does not fit, so switch to Qwen3 8B (6.8 GB). For higher quality on a 12 GB card, "ollama run qwen3:8b-q8_0" is a listed tag; Q8 is 11.6 GB by the site formula, so it needs 12 GB.

Port 11434 already in use — Ollama fails to start

Another process is using port 11434. Open PowerShell as Administrator and run: netstat -ano | findstr :11434 — note the PID in the last column, then run: taskkill /PID <pid> /F to terminate it. If Ollama is already running but not responding, quit it from the Ollama icon in the taskbar, then start it again from the Start menu.

Windows Defender quarantines the Ollama installer or model files

This is a false positive. Ollama and its model files are safe. To resolve: open Windows Security, go to Virus and Threat Protection, click Protection History, find the quarantined item, and click Allow. For the installer, right-click the downloaded .exe, select Properties, and check "Unblock" at the bottom before running. You can also add the Ollama install directory and model storage path to Windows Defender exclusions.

AMD GPU not detected by Ollama on Windows

Ollama on Windows uses ROCm v7 for AMD where the driver supports it, and Vulkan otherwise. Its docs note that some RDNA2 / Radeon RX 6000 cards may not expose ROCm v7 on current Windows drivers and recommend Vulkan for them; Vulkan is enabled by default. Update to the current Radeon driver from amd.com/support, restart Ollama, then check ollama ps. On a PC with integrated and discrete GPUs, the docs suggest setting GGML_VK_VISIBLE_DEVICES to the discrete GPU index if Ollama picks an unstable integrated GPU.

Optional: WSL2 for Advanced Users

The native Windows Ollama binary works well for most users. However, running Ollama inside WSL2 (Windows Subsystem for Linux) can offer slightly better performance and gives you access to the Linux Ollama ecosystem including ROCm for AMD GPUs and faster model loading in some configurations.

WSL2 Ollama setup (Ubuntu)

1. Enable WSL2 in PowerShell (run as Administrator)

wsl --install

2. Inside Ubuntu terminal, install Ollama for Linux

curl -fsSL https://ollama.com/install.sh | sh

3. Run your model

ollama run qwen3:8b

WSL2 has access to your Windows GPU via CUDA passthrough — NVIDIA GPUs work well. AMD GPU support in WSL2 requires ROCm which has limited Windows passthrough support as of 2026. For most Windows users, the native Ollama installer is simpler and performs nearly identically. WSL2 is worth trying if you encounter driver issues with the native install or need Linux-specific tooling.

Open WebUI: ChatGPT-Style Interface on Windows

Ollama's terminal interface is functional but minimal. Open WebUI adds a full browser-based chat interface with conversation history, model switching, file uploads, and web search — all running locally. It connects to your local Ollama server and requires Docker Desktop on Windows.

1. Install Docker Desktop for Windows

Download from docker.com/products/docker-desktop. Run the installer and enable the WSL2 backend when prompted. Restart your machine after installation.

2. Start Open WebUI with Docker

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway ghcr.io/open-webui/open-webui:main

3. Open your browser

Navigate to http://localhost:3000. Create an account on the first-run screen (this is local only — no internet account). Select your Ollama model from the dropdown at the top of the chat.

For a full Open WebUI setup walkthrough including configuration options, see the Open WebUI setup guide.

Frequently Asked Questions

Does Ollama work on Windows without a GPU?

Yes, Ollama runs on Windows without a GPU using CPU-only mode. It automatically falls back to CPU inference if no supported GPU is detected. CPU mode works but is much slower than GPU inference. For CPU-only use, stick to small models like Qwen3 1.7B or Phi-4-mini 3.8B.

Which GPU drivers do I need for LLMs on Windows?

For NVIDIA, the Ollama Windows docs ask for driver 551.61 or newer: use Manual Driver Search on nvidia.com/drivers and pick a Game Ready or Studio Driver for your card. For AMD Radeon, Ollama uses a ROCm v7 / HIP7-capable driver for ROCm acceleration or a Vulkan-capable Radeon driver for Vulkan, so install the current Radeon driver from AMD. For Intel Arc, install the Intel Arc Graphics Windows driver from intel.com; Ollama reaches Arc through Vulkan, which Windows GPU drivers usually include. Then run ollama ps to confirm the model is on the GPU.

How do I check if Ollama is using my GPU on Windows?

Two ways: run "ollama ps" in a terminal while a model is loaded and read the PROCESSOR column: 100% GPU means the whole model is on the GPU, 100% CPU means it is all in system memory, and a split such as 48%/52% CPU/GPU means part of it spilled to system RAM. You can also open Task Manager (Ctrl+Shift+Esc), click Performance and select your GPU to watch its activity during generation. If the model does not fit your VRAM, pick a smaller model or quantization.

Can I run LLMs on Windows with integrated graphics?

Technically yes, but performance is limited. Intel Iris Xe and UHD integrated graphics share system RAM, effectively giving 2 to 4 GB for model weights. Only very small models like Qwen3 1.7B or Phi-4-mini fit. AMD Radeon integrated graphics on Ryzen APUs performs similarly. A discrete GPU with 8 GB or more VRAM is strongly recommended for anything beyond small models.

What's the difference between Ollama and LM Studio on Windows?

Ollama is a CLI tool and headless background server. It exposes an OpenAI-compatible API at localhost:11434 and is ideal for developer integrations such as VS Code extensions, scripts and Open WebUI. LM Studio is a graphical desktop application with a built-in chat UI, model browser, and a visual GPU layers slider for tuning VRAM usage. LM Studio is easier for beginners; Ollama is better for developers who want a persistent API server. Both support the same GGUF models and NVIDIA, AMD, Intel Arc GPUs on Windows.

Check what models fit your Windows GPU or find the right hardware.

Sources & methodology

Behaviour, file-format and runtime details on this page are pulled from primary upstream docs and the sources listed below. The full sitewide methodology lives on the methodology page. For this guide we relied most on:

Spot a number that does not match the linked source? Contact us and we will update the guide.