Public benchmarks
Benchmarks signal general capability but saturate fast. Use them to shortlist models, not to choose one.
Changes fast: check the latest updates too. Reviewed 17 Oct 2026.
Key ideas
- Coding and agents: SWE-bench Verified, Terminal-Bench, tau-bench (tool use with policies).
- Reasoning and knowledge: GPQA Diamond, Humanity's Last Exam, ARC-AGI.
- Computer use: OSWorld. Long context: needle and multi-hop variants.
- Check whether reported scores used tools, extra compute or custom scaffolds.
Common mistakes
- Comparing numbers from different harnesses or settings.