Foundations & Models
5 areas, 0 updates so far. Checked every morning; 1 slower-moving area is checked on Mondays.
Nothing here yet. This area is checked every morning, and new updates show up as soon as something real happens.
LLM Foundations
How large language models work, from tokens to training stages, and how to pick one.
- Transformer architectureStableAlmost every modern LLM is a decoder-only transformer: a stack of self-attention and feed-forward blocks that predicts the next token from all previous tokens.
- Tokenization and context windowsStableModels read and bill in tokens, not words. The context window is the hard limit on input plus output tokens per request.
- Decoding and samplingStableThe model outputs a probability distribution for the next token; the decoding strategy decides which token is picked.
- EmbeddingsEvolvingEmbedding models map text (or images) to dense vectors so that semantic similarity becomes geometric distance. They power search, clustering, dedup and RAG.
- How models are trainedEvolvingPretraining builds raw capability; post-training turns it into a helpful, tool-using, reasoning assistant.
- Model landscape and selectionChanges fastModel rankings change monthly. Select models with your own evals against quality, latency, cost, context, tool use and deployment constraints.