Run Qwen Locally: Picking the Right Version

By , Ready Utilities

Last updated: September 2026

Qwen shows up as the top pick at nearly every hardware tier on this site when you run it locally, so it earns its own guide. This covers which Qwen sub-family to actually run, the commands, and two gotchas that trip people up more than any hardware question does.

Which Qwen should you run?

"Qwen" isn't one model, it's a family, and the sub-family matters more than most guides let on.

Sub-familyGood forOllama tag prefix
Qwen3General reasoning, multilingual, agentic tasks. The default choice on this site's own recommendations.qwen3:
Qwen2.5General reasoning and RAG, still widely used and well-supported.qwen2.5:
Qwen2.5-CoderCode generation and debugging specifically; see the coding guide for more.qwen2.5-coder:
Qwen3.6The newest flagship-tier release: a 27B dense model with a 256K context window, Apache 2.0 licensed.qwen3.6:

Unless you have a specific reason otherwise, Qwen3 is the right starting point, matching the picks in best local LLM by hardware tier: 8B for 8GB VRAM, 14B for 12-16GB, 32B for 24GB. A newer Qwen3.8 dense variant has started showing up in community rankings as this guide gets checked; it's worth a look if you want the newest option, but Qwen3 remains a solid, well-supported default and isn't going away.

The commands

ollama run qwen3:8b
ollama run qwen3:14b
ollama run qwen3:32b

Check your exact fit for any of these in the hardware calculator before downloading, especially at 14B or 32B where headroom for context matters.

The tag-naming trap

Typing ollama pull qwen3 with no size tag doesn't ask you which one you want. It silently downloads the 8B model. If you meant to grab the 14B or 32B version and left the tag off, you'll end up running a smaller model than you intended with no error or warning. Always include the size explicitly: qwen3:14b, not just qwen3.

The context window gotcha

Qwen models, like most, ship with a runtime default context window that's scaled to conserve memory, and it's often smaller than the model's actual maximum. If you paste in a long document or have an extended conversation, Ollama can silently truncate the beginning of it rather than warning you, so responses start ignoring things you said earlier. The fix is a custom Modelfile that sets the context length explicitly:

FROM qwen3:14b PARAMETER num_ctx 32768

Save that as a file named Modelfile, then build and run it as a named variant:

ollama create qwen3-32k -f Modelfile
ollama run qwen3-32k

A larger context window uses more memory (see the calculator's context length setting for the real cost), so don't set it larger than you'll actually use.

Sub-family details and the num_ctx behavior are cross-checked against multiple current Qwen setup guides (see the fact-check log); the tag-naming default is confirmed the same way across independent sources. This page gets rechecked as new Qwen versions ship, since the family updates often.

Sources

  1. PromptQuorum (opens in new tab)
  2. LocalAIMaster (opens in new tab)
  3. PromptQuorum (Ollama model rankings) (opens in new tab)

Frequently asked questions

Why does Qwen seem to forget the start of a long conversation?

This is almost always the context window default being smaller than you expect, not the model failing. Set num_ctx explicitly in a custom Modelfile, as shown above, to fix it.

Is Qwen better than Llama or DeepSeek?

There's no universal answer; it depends on the task and your hardware. This site recommends Qwen3 as the default starting point at most tiers, but see the hardware-tier guide for the full comparison.

What happens if I run ollama pull qwen3 without a size?

It downloads the 8B model with no warning. If you wanted a different size, you have to explicitly include it, like qwen3:32b.