How models are trained
Pretraining builds raw capability; post-training turns it into a helpful, tool-using, reasoning assistant.
Evolving: the basics hold, details shift. Reviewed 17 Oct 2026.
Key ideas
- Pretraining: next-token prediction on trillions of tokens. Sets knowledge and the knowledge cutoff.
- SFT (supervised fine-tuning): demonstrations of desired behaviour, chat format, tool calls.
- Preference tuning: RLHF, RLAIF or DPO to align outputs with preferences and safety policies.
- RL with verifiable rewards (RLVR): rewarding correct answers on math, code and tasks with checkers. This is what produced modern reasoning models.
- Reasoning models spend extra tokens thinking before answering; you trade latency and cost for accuracy on hard problems.
Common mistakes
- Expecting the model to know events after its cutoff. Give it current facts through context or tools.