By Web Asset House · data reviewed 2026-10-22
A model’s supported context limit is not a promise that every computer can allocate that context. During generation, many transformer runtimes keep key and value tensors for earlier tokens. The capacity needed for that cache grows with the selected context and the number of sequences.
The standard attention calculation
KV bytes = 2 × layers × KV heads × head dimension × context tokens × bytes per element
The default uses one sequence and two bytes per element. KV heads matter: grouped-query attention can use fewer KV heads than query heads. Using the query-head count in its place can substantially distort the estimate. We read the relevant model configuration when available.
Know when the simple formula does not apply
Latent attention, state-space components, sliding windows, mixed attention layers, runtime cache quantization, and vision-specific state can require a different memory model. When the available fields do not establish a supported standard-attention calculation, the estimator reports unknown rather than treating the missing cache as zero.
Plan for the actual workflow
Use a context length that covers the prompt and generated output together. A long document, tool history, retrieved passages, and coding conversation all consume space. Recheck the memory result when increasing context. Our speed range does not model the additional attention work or prompt-processing cost caused by a longer sequence.
A calculator result is a starting point
Leave headroom for the operating system and other applications, then verify the real allocation and responsiveness in the intended runtime. The calculator labels results above 90% of its usable budget as borderline so a narrow margin is visible.
Primary sources
- Ollama: memory, context, local and cloud behavior
- llama.cpp: supported build backends
- Hugging Face: GGUF documentation
Questions, answered.
Does doubling context double all memory usage?
No. It doubles the standard-attention KV-cache estimate at fixed settings, while the model weights remain the same.
Is unknown cache size treated as zero?
No. Missing or unsupported cache metadata produces an unknown fit result rather than a false positive.