Tokenization and context windows
Models read and bill in tokens, not words. The context window is the hard limit on input plus output tokens per request.
Stable: this rarely changes. Reviewed 17 Oct 2026.
Key ideas
- Most models use BPE-style subword tokenizers. English averages roughly 0.75 words per token; code, non-Latin scripts and numbers cost more.
- Each provider tokenizes differently, so the same prompt has a different token count and price across models.
- Context windows of 200K to 1M+ tokens are common, but quality degrades for facts buried in the middle of very long inputs.
- Output tokens are usually priced several times higher than input tokens and are generated serially, so they dominate latency.
- Count tokens with the provider's tokenizer or token-counting endpoint before shipping cost estimates.
Common mistakes
- Treating a big context window as a substitute for retrieval. Bigger context raises cost and can lower accuracy.
- Forgetting that tool definitions and system prompts consume context on every call.