Learn AI Layer by Layer
An interactive guide to understanding AI from first principles.
Subscribe to messyprogress.substack.com to find out when new chapters get released.
Introduction
What this is and how to use it
Everything Is Numbers
Computation
The Power of Incremental Improvement
Optimization
Building a Brain
Neural networks
Describing the World with Numbers
Vectors
From Words to Meanings
Embeddings
Understanding by Predicting
Next-word prediction
Paying Attention
Attention
Where Am I?
Positional encoding
One Architecture to Rule Them All
Transformers
Thinking by Rotating
Matrix math
Coming soon
Why Training Almost Doesn't Work
Making training work
Coming soon
Only Wake the Specialists You Need
Mixture of experts
Coming soon
Remembering a Million Words
Long context
Coming soon
Running Models Fast
Inference and hardware
Coming soon
Looking Inside the Mind
Interpretability
Coming soon
Learning from Experience
Reinforcement learning
Coming soon
Getting Better by Beating Yourself
Self-play
Coming soon
Thinking by Talking to Yourself
Reasoning models
Coming soon
Teaching AI Right from Wrong
Alignment
Coming soon
Models Teaching Models
Distillation and synthetic data
Coming soon
Teaching Machines to See
Image comprehension
Coming soon
Drawing Pictures
Image generation
Coming soon
Simulating Reality
World models
Coming soon
Listening, Speaking, and Singing
Audio
Coming soon
Getting Things Done
Agents and tool use
Coming soon
Making Stuff Up
Hallucination and grounding
Coming soon
Getting the Right Information
Context management
Coming soon