Rabbit Hole

Rabbit Hole › Demo paths

How a neural network learns

3Blue1Brown's deep learning series as a rabbit hole. The main track goes from neurons to transformers; go deeper wherever the maths gets ahead of you.

The path, clip by clip

10 videos: the main track runs top to bottom; indented clips are the go-deeper side paths. Every video plays in YouTube's own player and belongs to its creator; follow the links to watch on YouTube.

  1. But what is a neural network? by 3Blue1Brown
    Start here: neurons, layers and weights, with handwritten-digit recognition as the running example.

    Go deeper:

    1. Vectors (Essence of linear algebra, chapter 1) by 3Blue1Brown
      Rusty on vectors? This is the picture everything else builds on.
    2. Linear transformations and matrices (chapter 3) by 3Blue1Brown
      Why one layer of a network is a matrix multiplication.
  2. Gradient descent, how neural networks learn by 3Blue1Brown
    The cost function, and how a network improves by walking downhill on it.

    Go deeper:

    1. The essence of calculus by 3Blue1Brown
      The intuition for derivatives that gradient descent depends on.
    2. The paradox of the derivative (Essence of calculus, chapter 2) by 3Blue1Brown
      What an instantaneous rate of change really means.
  3. Backpropagation, intuitively by 3Blue1Brown
    How the network works out which weights to nudge, and by how much.

    Go deeper:

    1. Backpropagation calculus by 3Blue1Brown
      The same idea with the chain rule written out.
  4. Transformers, the tech behind LLMs by 3Blue1Brown
    From recognising digits to predicting the next word: the architecture behind large language models.

    Go deeper:

    1. Attention in transformers, step-by-step by 3Blue1Brown
      The mechanism that lets each word look at every other word.