- Context window
- The amount of text (measured in tokens) a model can "see" at once, including your prompt, any earlier conversation, and its own response so far. A larger context window lets you paste in longer documents or hold longer conversations before the model starts losing track of earlier parts, but it also costs more memory. See the hardware calculator's context length setting.
- Fine-tuning
- Further training an existing model on your own data so it specializes in a particular task or style, rather than training a model from scratch. Most local AI use doesn't involve fine-tuning at all; downloading and running a model as-is covers the large majority of use cases on this site.
- GGUF
- The file format most local AI models are distributed in. A single .gguf file packages the model's weights, its quantization level, and metadata together, which is why you'll see model downloads named things like "Llama-3.1-8B-Q4_K_M.gguf."
- GPU offload
- Running part of a model on your GPU and the rest on your CPU/system RAM, used when a model doesn't fully fit in VRAM. It lets you run something otherwise too large, at a real speed cost since the CPU-handled portion is much slower than the GPU portion.
- Inference
- The process of actually running a model to generate a response, as opposed to training it. Everything this site's guides walk you through, installing Ollama, downloading a model, chatting with it, is inference. You're never training anything by default.
- KV cache (key-value cache)
- Memory the model uses to "remember" the conversation so far without recomputing everything from scratch on every new word. It grows with your context window and depends on the model's specific architecture (layer count and attention head count), which is why two models the same size can need noticeably different amounts. The hardware calculator's methodology section explains this in more depth.
- Mixture of experts (MoE)
- An architecture where a model is built from many smaller "expert" sub-networks, and only a handful activate for any given word, instead of the whole model computing every time. This speeds up generation, but the entire parameter set still has to be loaded into memory regardless of how few experts activate per token, so it doesn't reduce your hardware requirement the way the "active parameters" figure might suggest.
- Parameters
- The individual numeric values a model learned during training; roughly, its "size." A model labeled "8B" has about 8 billion of these. More parameters generally means more capability, and always means more memory to store and run the model.
- Quantization
- Storing a model's parameters at lower precision to shrink its file size and memory footprint, at some cost to output quality. Q4_K_M (4-bit) is the most common default for local use; FP16 (16-bit) is full, unquantized precision. See quantization explained for the full ladder of options and what each one costs in quality.
- Token
- A chunk of text a model processes as one unit, roughly (not exactly) a word. "Tokens per second" is the standard way local AI speed is measured; context windows are also measured in tokens.
- Unified memory
- Apple Silicon's approach to memory, where the CPU, GPU, and Neural Engine share one pool instead of having separate VRAM and system RAM. See the Apple Silicon guide for how this changes hardware planning on a Mac.
- VRAM
- The dedicated memory on a discrete graphics card, separate from your computer's regular system RAM. VRAM capacity is usually the single most important spec for local AI on a Windows or Linux PC. See the hardware buying guide for VRAM by GPU tier.
Local AI Glossary
Last updated: September 2026
This local AI glossary defines the terms used across this site's guides and calculator. Each entry has its own link, so other pages jump straight to the definition you need.
Definitions here match the terminology and figures used consistently across this site's guides and calculator, rather than introducing new claims. Where a term's real-world behavior is more nuanced (KV cache, mixture of experts), the entry links to the fuller explanation elsewhere on the site rather than oversimplifying.
Sources
No external sources for this page -- every term matches usage already sourced elsewhere on this site (see the calculator and hardware guide).