RTX 3090 for AI: What 24GB of VRAM Runs, and What to Check Before Buying Used

The RTX 3090 specs that matter for local AI

For running models locally, three numbers on this card do most of the work: how much memory it has, how fast that memory moves data, and how much power it pulls. The rest of the spec sheet matters more for gaming.

What fits on 24GB: the RTX 3090 model table

These figures come from the same formula the hardware calculator uses, run against the 3090's 24 GB. "Fits" means the model plus its context memory stays under 90% of VRAM, leaving room for the driver and your desktop. "Tight" means it lands between 90% and just over 100%, where it may load or may spill into system RAM depending on your setup.

*Gemma 3 figures are on the cautious side. Gemma 3's published config keeps only a 1,024-token window for five of every six layers, but the calculator counts every layer at full context. Runtimes that store just that window for those layers need less memory than shown, especially at 32K.

The practical sweet spot on a 3090 is the 24B to 27B range at Q4: Mistral Small 24B and Gemma 3 27B both fit with room for a normal conversation. The 32B models are where 24 GB starts to pinch. They load at Q4 with short chats, but there's little room left over, and a longer context pushes them past the card. Not sure what Q4 and Q8 trade away? The quantization explainer covers it, and the quantization calculator shows every level's size side by side.

If 24 GB covers the models you want to run, here's an affiliate link to current RTX 3090 listings on Amazon (opens in new tab). Check the used-buying checklist before you pick one.

The 32K context catch on a 24GB card

Ollama picks a default context length based on how much VRAM it finds. Its documentation lists 4K for cards under 24 GiB and 32K for cards from 24 to 48 GiB. A 24GB card sits right at the bottom edge of that bigger band.

That matters because context memory grows with context length. By the calculator's math, Qwen 32B at Q4 needs about 22.9 GB at 4K but about 29.9 GB at 32K, which is more than the card holds. When that happens, part of the model lands in system RAM and generation slows down noticeably.

To see what you actually got, run ollama ps while the model is loaded. The PROCESSOR column should read 100% GPU. A split like 30%/70% CPU/GPU means it spilled over. Ollama's docs show how to set the context yourself, for example by starting the server with OLLAMA_CONTEXT_LENGTH=8192 ollama serve, or with the context slider in the Ollama app's settings. For 32B models on a 3090, a shorter context is the trade that keeps everything on the card.

RTX 3090 vs newer cards for local AI

Capacity decides what you can run. Bandwidth decides how fast it runs once it fits. Here's how the 3090 lines up against the two newer flagships on both:

The 3090 and 4090 hold exactly the same models, so if room for bigger models is what you're paying for, the 4090 doesn't add any. It does have about 8% more memory bandwidth and a newer architecture, at 100 W more power. The 5090 is the real step up in capacity: its extra 8 GB is what moves 32B models from tight to comfortable. The RTX 4090 and RTX 5090 pages have their own fit tables. For head-to-head breakdowns, see RTX 3090 vs RTX 4090 and RTX 3090 vs RTX 4080. We haven't measured tokens per second on these cards ourselves, so this page doesn't rank their speed. Compare current prices on all three before deciding, since used and new pricing has moved a lot in 2026. The hardware guide covers the 12GB and 16GB tiers if 24 GB is more than you need.

Two RTX 3090s: what 48GB gets you

The 3090 and 3090 Ti were the last GeForce cards with an NVLink connector, which is part of why people pair them. The RTX 40 series dropped it. You don't need the bridge to run one model across two cards, though. Ollama's documentation says that when a model doesn't fit on any single GPU, it spreads the model across all available GPUs.

Two cards give you 48 GB in total. That's still short of comfortable for Llama 3.3 70B at Q4, which needs about 49 GB by the calculator's single-pool math, and splitting a model across cards adds some overhead the calculator doesn't model. Treat a 70B model on two 3090s as tight at best. For the power side, size the supply for two 350 W cards plus the rest of the system, using the exact cards' specifications.

Buying a used RTX 3090: what to check

The general used-GPU checklist applies here too. These are the checks specific to the 3090:

• Confirm it's really a 24GB 3090. Listings sometimes mix up the 3090, 3090 Ti, 3080 and 3080 Ti, and the 3080 cards carry much less VRAM. Once it's installed, nvidia-smi or GPU-Z should report about 24 GB (24,576 MiB) of memory.
• Watch the memory temperature under load. The 3090 carries 24 one-gigabyte GDDR6X chips on both sides of the board, so the back of the card gets warm. HWiNFO shows a "GPU Memory Junction Temperature" sensor. Check it while a model is generating; unusually high readings on a used card can point to tired thermal pads, which is a fix that means opening the card.
• Ask how it was used. A card that ran around the clock for mining or rendering has more wear than one used for gaming in the evenings.
• Check power and connectors for the exact model. The Founders Edition uses a 12-pin plug and ships with an adapter from two 8-pin cables; make sure that adapter comes with it. Partner cards use their own 8-pin layouts and often run higher power limits than the 350 W reference, so read that model's spec page before trusting your current power supply.
• Measure your case. The Founders Edition is 313 mm long and three slots thick, and many partner cards are bigger.
• Test it before the return window closes. Load a model that fills most of the card, such as Mistral Small 24B at Q4 (about 17 GB), confirm ollama ps shows 100% GPU, and let it generate for a while as you watch temperatures.