Local AI PC guide (2026): VRAM and GPUs for running LLMs

For a local AI PC that runs large language models (LLMs) well, buy the GPU with the most VRAM you can afford, then build a balanced system around it. As a rule of thumb, a model quantized to 4 bits needs roughly 0.6 GB of VRAM per billion parameters, plus room for context. That means 16 GB comfortably runs models up to about 14 billion parameters, 32 GB reaches the 30-billion class, and 70-billion-parameter models need 48 GB or more, unless you accept much slower offloading to system RAM.

How much VRAM do you need for local LLMs?

Model weights take parameters × bits per weight ÷ 8 bytes. On top come the KV cache, which grows with context length and the number of simultaneous requests, and one to two gigabytes of runtime overhead. The table shows the weights alone, rounded, then a practical VRAM target for 4-bit use with a useful context window.

Quantization stores weights with fewer bits. 8-bit is close to the original quality; well-made 4-bit quantizations are the usual sweet spot for local use; below 4 bits quality falls faster, especially on smaller models. Popular 4-bit formats such as GGUF Q4_K_M average slightly more than 4 bits per weight, which is why the practical column adds headroom. You will meet GGUF in llama.cpp, Ollama and LM Studio, and GPTQ, AWQ, FP8 or FP4 variants in GPU-focused engines such as vLLM.

Mixture-of-experts (MoE) models activate only part of their parameters for each token, so they run faster than dense models of the same total size, but every expert still has to sit in memory. Engines such as llama.cpp can keep experts in system RAM and the rest on the GPU, which makes large MoE models usable on one consumer card with plenty of DDR5, at reduced speed. OpenAI's gpt-oss-120b, for example, was designed to fit on a single 80 GB GPU.

Memory bandwidth decides generation speed

When a dense model generates text, each new token reads all of the weights from memory. Generation speed therefore has a hard ceiling of roughly memory bandwidth ÷ model size in bytes: a card with 1,792 GB/s, the RTX 5090's specification, reading a 20 GB model cannot exceed about 90 tokens per second, before any other overhead. Processing a long prompt, by contrast, is limited by compute and favours GPUs with more tensor throughput. This is why a GPU with fast GDDR7 generates text far faster than a system with more, but slower, memory, as long as the model fits in VRAM.

GPU choices: consumer GeForce vs RTX PRO

Consumer GeForce cards give the most performance per dollar, and the RTX 5090's 32 GB is the most memory on a consumer card. RTX PRO cards add capacity (up to 96 GB on one card), ECC memory, certified drivers and coolers designed for multi-GPU workstations: the RTX PRO 6000 Blackwell comes as a 600 W Workstation Edition and a 300 W Max-Q version with a blower-style cooler that exhausts heat out of the case, which suits stacking several cards. NVIDIA's CUDA remains the best-supported platform for AI software; AMD and Intel cards work well with llama.cpp-based tools through Vulkan, ROCm or SYCL, but check support in the exact software you plan to use. Shop graphics cards, and see our graphics card buying guide for gaming considerations.

One GPU or two? Inference engines can split a model across GPUs, so two 32 GB cards give 64 GB of combined VRAM. Current GeForce and RTX PRO Blackwell cards have no NVLink and talk over PCIe: splitting by layers works well for single-user chat, while tensor parallelism benefits from PCIe 5.0 with eight or sixteen lanes per card. Mainstream AM5 and LGA1851 boards split their lanes to x8/x8 at best, while Threadripper and Xeon W platforms have enough lanes for three or four cards at full width; complete AI workstations are built around them.

Unified memory alternatives

Systems with large unified memory trade bandwidth for capacity. Mini PCs with AMD's Ryzen AI Max+ offer up to 128 GB of LPDDR5X at 256 GB/s, NVIDIA's DGX Spark pairs 128 GB with 273 GB/s, and a Mac Studio with M3 Ultra reaches 819 GB/s with up to 512 GB. They load models that would otherwise need several GPUs, but generate tokens more slowly than a high-end GPU on any model that fits in its VRAM. They suit large MoE models at modest speed, quiet desks and low power; a GPU workstation suits speed, fine-tuning and image or video generation.

System RAM and storage for AI

System RAM: fit at least as much as your total VRAM, ideally twice as much, so models load smoothly and the operating system and tools have room. For MoE offloading, 128 GB of DDR5 or more makes a real difference. On AM5 and LGA1851, four modules usually run at lower speeds than two, so prefer two large modules, such as 2 × 48 GB or 2 × 64 GB. Threadripper and Xeon W platforms add memory channels, which also speeds up CPU offloading. Our how much RAM guide explains capacities; shop memory.

Storage: model files are large (a 70B model at 4-bit is around 40 GB) and collections grow quickly, so plan for 2–4 TB of NVMe storage. A PCIe 4.0 or 5.0 drive loads models quickly; once a model sits in VRAM, drive speed no longer matters. See NVMe SSDs and our NVMe SSD guide.

Power and cooling for AI workloads

AI workloads hold GPUs at high load for long stretches, so size the power supply for sustained draw, not short gaming peaks:

• The RTX 5090 is rated at 575 W, and NVIDIA recommends a 1,000 W power supply. Choose a quality ATX 3.1 unit with a native 12V-2x6 cable and seat the connector fully.
• Two high-end GPUs plus a desktop CPU can exceed 1,400 W. In North America, a standard 15 A, 120 V circuit supports about 1,440 W of continuous load, so very large builds may need a 20 A or 240 V circuit.
• Power-limiting the GPU, for example to 70–80% of its rated power, is common for inference: memory-bound text generation loses little speed while heat and noise drop.

For cooling, open-air cards in multi-GPU builds need space between them and strong front-to-back airflow, while blower-style workstation cards push heat straight out of the case. Check card length and slot width against your case, and consider an AIO cooler or custom water cooling for dense, quiet builds. Our PSU sizing guide walks through the maths; browse power supplies.

Three balanced local AI builds

Check compatibility as you go with the PC Builder.