How a neural network learns
3Blue1Brown's deep learning series as a rabbit hole. The main track goes from neurons to transformers; go deeper wherever the maths gets ahead of you.
The path, clip by clip
10 videos: the main track runs top to bottom; indented clips are the go-deeper side paths. Every video plays in YouTube's own player and belongs to its creator; follow the links to watch on YouTube.
- But what is a neural network? by 3Blue1Brown
Start here: neurons, layers and weights, with handwritten-digit recognition as the running example.Go deeper:
- Vectors (Essence of linear algebra, chapter 1) by 3Blue1Brown
Rusty on vectors? This is the picture everything else builds on. - Linear transformations and matrices (chapter 3) by 3Blue1Brown
Why one layer of a network is a matrix multiplication.
- Vectors (Essence of linear algebra, chapter 1) by 3Blue1Brown
- Gradient descent, how neural networks learn by 3Blue1Brown
The cost function, and how a network improves by walking downhill on it.Go deeper:
- The essence of calculus by 3Blue1Brown
The intuition for derivatives that gradient descent depends on. - The paradox of the derivative (Essence of calculus, chapter 2) by 3Blue1Brown
What an instantaneous rate of change really means.
- The essence of calculus by 3Blue1Brown
- Backpropagation, intuitively by 3Blue1Brown
How the network works out which weights to nudge, and by how much.Go deeper:
- Backpropagation calculus by 3Blue1Brown
The same idea with the chain rule written out.
- Backpropagation calculus by 3Blue1Brown
- Transformers, the tech behind LLMs by 3Blue1Brown
From recognising digits to predicting the next word: the architecture behind large language models.Go deeper:
- Attention in transformers, step-by-step by 3Blue1Brown
The mechanism that lets each word look at every other word.
- Attention in transformers, step-by-step by 3Blue1Brown