Research
5 areas, 0 updates so far. Checked every morning; 2 slower-moving areas are checked on Mondays.
Nothing here yet. This area is checked every morning, and new updates show up as soon as something real happens.
Evaluation
Knowing whether your AI system works, and catching regressions before users do.
- Eval strategyStableEvals are the unit tests of AI systems. Start from real failure modes, write targeted checks and run them on every prompt, model or tool change.
- LLM-as-judgeEvolvingA model grades outputs against a rubric. Cheap and scalable, but it must be calibrated against human labels.
- Evaluating agentsChanges fastAgents need evals of the final outcome and of the path: tool choice, arguments, number of steps and cost.
- Public benchmarksChanges fastBenchmarks signal general capability but saturate fast. Use them to shortlist models, not to choose one.
- Eval and tracing toolsChanges fastTools to store datasets, run experiments, trace production calls and review outputs.