Module 6 of 13 in the AI Track, a free, self-directed curriculum for building AI that moves revenue.
The idea
Enough of the machinery to be dangerous. Transformers and attention as intuition, not matrix math. Tokens, and why a model can’t count the letters in a word, and why a long context costs real money. The training stages, from pretraining to fine-tuning to instruction tuning, and what each one buys. Then the knobs you’ll actually turn in production: context window, temperature, top-p, max tokens.
The payoff is understanding why models hallucinate. They predict the next plausible token, not the truth. Once you really get that, everything built around grounding and verification makes sense, because all of it exists to fight that one tendency.
Why it matters
“No hallucinations by design” only means something if you can explain the failure it’s designed against. This is what turns a slogan into something you can defend to an engineer who’s trying to poke holes in it.
Build this
The piece you’ll point people to: “Why LLMs hallucinate, and how we stop it.” Written for a smart buyer who isn’t technical. It takes the hardest thing to explain and makes it something they can read in five minutes.
How to work through it
- Jay Alammar’s Illustrated Transformer
- Karpathy’s Intro to LLMs and Let’s build GPT
- Read the Anthropic and OpenAI docs on context and parameters
- Draft the hallucination explainer
Resources
- Jay Alammar’s Illustrated Transformer
- Karpathy’s Intro to LLMs and Let’s build GPT
- Anthropic and OpenAI model docs