RTX 4090 for AI: Same 24GB as the 3090, So What Are You Paying For?

The RTX 4090 specs that matter for local AI

The 4090 is still one of the fastest consumer cards you can run a local model on. For AI work, the useful parts of its spec sheet are these:

What fits on 24GB: the RTX 4090 model table

Same formula as the hardware calculator, run against 24 GB. "Fits" means under 90% of VRAM, leaving room for the driver and desktop. "Tight" means between 90% and just over 100%, where it may load or may spill into system RAM.

*Gemma 3 figures are on the cautious side. Gemma 3's published config keeps only a 1,024-token window for five of every six layers, but the calculator counts every layer at full context. Runtimes that store just that window for those layers need less memory than shown, especially at 32K.

These numbers are identical to the 3090's, because capacity is the only thing the calculator checks. The RTX 3090 page adds a Q5 column if you're weighing quantization levels on a 24GB card.

If 24 GB covers the models you want to run and the extra speed is worth it to you, here's an affiliate link to current RTX 4090 listings on Amazon (opens in new tab). The buying checks below cover what to inspect.

RTX 4090 vs RTX 3090: what the extra money buys

For running local models, the honest summary is that the 4090 buys speed, not capacity. Token generation is mostly limited by memory bandwidth, and the 4090's is about 8% higher than the 3090's. Its newer architecture and much higher CUDA core count help more with prompt processing, where the card chews through your input before it starts answering. That shows up most with long documents and big pasted code files.

What it costs you: 100 W more power, a bigger recommended power supply, and no NVLink. We haven't benchmarked either card, so this page doesn't put a number on the speed gap.

Lean toward the 4090 if you feed models long inputs often, or you also use the card for image generation or gaming where its extra compute counts. Lean toward the 3090 if your goal is fitting a model, since both hold the same 24 GB. The RTX 3090 page has its full fit table and a used-buying checklist, and the full RTX 3090 vs RTX 4090 comparison also weighs two 3090s against one 4090.

RTX 4090 vs RTX 5090

The 5090 is the card that actually adds room. Its extra 8 GB moves 32B models at Q4 from tight to comfortable with short chats, and its bandwidth is about 78% higher. Neither card runs a 70B model at Q4 on its own. The RTX 5090 page has the 32GB fit table, and the RTX 4090 vs RTX 5090 comparison lists every model and setting where the two cards disagree.

The 32K context catch

Ollama sets its default context length from your VRAM. Its documentation lists 4K under 24 GiB and 32K from 24 to 48 GiB, and a 24GB card sits right at the bottom edge of that bigger band. At 32K, Qwen 32B at Q4 needs about 29.9 GB by the calculator's math, more than the card holds. Run ollama ps and check that PROCESSOR reads 100% GPU. If it shows a CPU/GPU split, set a shorter context, for example with OLLAMA_CONTEXT_LENGTH=8192 ollama serve. The RTX 3090 page walks through this in more detail; it applies the same way here.

Buying an RTX 4090 now

NVIDIA reportedly wound down 4090 production in late 2024 ahead of the RTX 50 series, so most 4090s for sale now are used, renewed, or leftover stock. The general used-GPU checklist applies. These are the checks specific to this card:

• Confirm it's a 24GB 4090. The 4080, 4080 Super and 4070 Ti Super have 16 GB. Once it's installed, nvidia-smi or GPU-Z should report about 24 GB.
• Inspect the 16-pin power connector. After reports of melted adapters in 2022, NVIDIA said the failures it examined came from connectors that weren't fully plugged in. On a used card, look at the socket for discoloration or deformed pins, and make sure the plug seats all the way with no gap.
• Check your power supply. NVIDIA lists 850 W for the Founders Edition. Partner cards often run higher power limits, so read the exact model's spec page. A power supply with a native 16-pin cable avoids the adapter altogether.
• Measure your case. The Founders Edition is 304 mm long and three slots thick, and many partner cards are bigger.
• Test it before the return window closes. Load a model that fills most of the card, such as Mistral Small 24B at Q4 (about 17 GB), confirm ollama ps shows 100% GPU, and let it generate for a while as you watch temperatures.