RTX 5090 for AI: What 32GB Runs, and Where It Still Stops Short

The RTX 5090 specs that matter for local AI

The 5090 is NVIDIA's current consumer flagship, and the first GeForce card with more than 24 GB. For running models locally, these are the parts of the spec sheet that count:

What fits on 32GB: the RTX 5090 model table

Same formula as the hardware calculator, run against 32 GB. "Fits" means under 90% of VRAM, leaving room for the driver and desktop. "Tight" means between 90% and just over 100%, where it may load or may spill into system RAM.

*Gemma 3 figures are on the cautious side. Gemma 3's published config keeps only a 1,024-token window for five of every six layers, but the calculator counts every layer at full context. Runtimes that store just that window for those layers need less memory than shown, especially at 32K.

On 32 GB, every model up to the 32B class fits at Q4 with a short chat, and the 32B models also fit at Q5. Long contexts and Q8 on the bigger models are where it still runs out. If 32 GB covers the models you want to run, here's an affiliate link to current RTX 5090 listings on Amazon (opens in new tab). Plan the power supply first; the before-you-buy list covers it.

What 8 more GB changes compared with a 24GB card

Against an RTX 3090 or 4090, the 5090's extra memory moves these from tight or won't-fit to comfortable, by the calculator's numbers:

• Qwen and DeepSeek-R1-Distill 32B at Q4, short chat: about 22.9 GB. Tight on 24 GB, comfortable on 32 GB.
• The same 32B models at Q5: about 27.1 GB. Won't fit on 24 GB, fits on 32 GB.
• Gemma 3 27B at Q5: about 24.0 GB. Tight on 24 GB, fits on 32 GB.
• Mistral Small 24B at Q8: about 29.4 GB. Won't fit on 24 GB, tight on 32 GB.

Put simply, a 24GB card makes you pick between a 32B model and a comfortable margin. The 5090 lets you have both, and step up a quantization level on 24B to 32B models. If 24 GB covers the models you actually use, the RTX 3090 and RTX 4090 pages cover those cards, and the RTX 4090 vs RTX 5090 comparison adds bandwidth, power and running cost side by side.

The 70B question

A single 5090 doesn't run Llama 3.3 70B at Q4. The calculator puts it at about 49.1 GB with a short chat, well past 32 GB. If 70B-class models are the goal, your realistic options are more total memory (two cards, or a Mac with enough unified memory; see the Apple Silicon guide), partial CPU offload, which is much slower, or a smaller model. For most people, a 32B model running fully on the card is the better experience.

Context length on a 32GB card

Ollama's documentation lists a 32K default context for cards with 24 to 48 GiB, which includes the 5090. Most models in the table handle that without trouble. The ones that don't: Qwen 32B at Q4 needs about 29.9 GB at 32K, a tight fit, and Gemma 3 27B shows as won't-fit at 32K, though see the note above on why that figure runs high. Run ollama ps and check that PROCESSOR reads 100% GPU. If it shows a CPU/GPU split, set a shorter context, for example with OLLAMA_CONTEXT_LENGTH=16384 ollama serve.

Before you buy an RTX 5090

• Plan the power. NVIDIA lists a 1,000 W minimum for the Founders Edition, and partner cards often run higher limits. A power supply with a native 600 W 16-pin cable avoids the four-plug adapter.
• Seat the 16-pin connector fully. Push the plug in until there's no gap, and avoid sharp bends right at the connector.
• Check the slot and the case. The Founders Edition is a 304 mm, two-slot card, but NVIDIA's own setup notes call for clearance for a three-slot card, and partner cards are often thicker. PCIe 5.0 isn't required; it works in a PCIe 4.0 slot.
• Compare against the real price. Don't assume the $1,999 launch price. Check what 5090s actually sell for, and compare that with a used 24GB card if 32B models at Q4 are the most you need.
• Test it before the return window closes. Load Qwen 32B at Q4 with a short context, confirm ollama ps shows 100% GPU, and let it generate for a while as you watch temperatures.