Backpropagation explained

How a neural network shares out the blame for a wrong answer: the chain rule in plain words, one weight's share worked out in full, and why it is gradient descent one layer at a time. A tiny network on a robot's sensor readings is trained by hand-written backpropagation in your browser, with the loss charted and every weight printed.

Guidefree, runs in your browser

Backpropagation is how a neural network works out which of its weights to blame for a wrong answer, and by how much. It runs the error backwards through the network, layer by layer, and hands every weight its own share. Once each weight knows its share, the network learns by the rule in the gradient descent guide: move each weight a little the other way. On this page a network small enough to read, with two of the robot's sensor readings going in and one steering command coming out, is trained by backpropagation written out in full, and each demo is a program you can change and run.

This page assumes you know what a weight, a bias and an activation are. The neural network guide covers those from nothing.

The idea in one line

this weight's share of the error = the error at the output
                                 × how much the output moves when this weight moves

The second part is the hard part, because a weight in the first layer does not touch the output. It changes a hidden number, which changes the total at the output, which changes the output. So follow the chain and multiply.

That is the chain rule, and in plain words it is this. If turning the tap changes the flow twice as fast as the tap turns, and the flow fills the bucket three times as fast as the flow rises, then turning the tap fills the bucket six times as fast. Rates multiply along a chain. A network is a long chain of small steps, each one a multiply, an add or a squash, and every one of them has a rate that is easy to write down. Backpropagation is nothing more than multiplying those rates together, starting at the output and working back.

The network on this page

The robot's depth sensor reports 64 distances. This network takes two numbers from them: left, the nearest thing in the left half of the view, and right, the nearest in the right half. Both are scaled the way lesson A tiny network scales them, so that 30 cm becomes 0 and 50 cm or more becomes 1:

x = (min(reading, 50) − 30) / 20

Those two numbers feed two hidden neurons. Each one multiplies, adds its bias and squashes with tanh. The two hidden numbers feed one output neuron, which does the same, and its answer is the turn: -1 is a hard left, 0 is straight on, +1 is a hard right. That is 4 + 2 weights in the first layer and 2 + 1 in the second: nine numbers in all, few enough to print and read.

One weight's share of the error

Take one view the robot recorded: 16 cm on the left, 36 cm on the right. The wall ends on the right, so the right answer is +1, turn right. Scaled, the inputs are -0.7 and 0.3.

Run it forwards through the weights in the diagram and the output is -0.155, a small turn to the left: wrong, and wrong in the wrong direction. The loss, the square of the miss, is 1.334.

Now work backwards. Each link in the chain is a rate, and each one is a single line of arithmetic:

• The loss to the output. The loss is (y − 1)², so its rate of change with y is 2 × (y − 1), which is -2.310.
• The output to its total. The output is tanh(z), whose rate is 1 − y², which is 0.976. This is where tanh matters: near 0 the rate is almost 1, and out at the flat ends it is almost nothing.
• The total to the hidden number. The total is h1 × 0.7 + h2 × -0.5 + 0.2, so its rate with h1 is the weight itself, 0.7.
• The hidden number to its own total. Another tanh: 1 − h1², which is 0.893.
• That total to the weight. The total is x1 × w + ..., so its rate with w is the input itself, -0.7.

Multiply the chain and you have the share of the weight that joins the left reading to the first hidden neuron: -2.310 × 0.976 × 0.7 × 0.893 × -0.7 = 0.986. Stop three links earlier and you have the share of the weight that joins the first hidden neuron to the output: -2.310 × 0.976 × -0.327 = 0.738, where -0.327 is h1 itself.

Both are positive, which means the loss grows if either weight grows, so both weights should shrink. The first layer's weight has the larger share, so it moves further.

The demo does the same arithmetic and then checks it the blunt way: it nudges the weight by 0.001, runs the network again, and measures what the loss actually did. If backpropagation is right, the two numbers agree.

from bugbot import *
import math
connect()

# one view the robot recorded: something 16 cm away on the left of its view and
# 36 cm on the right. The wall ends on the right, so the right answer is +1.
LEFT, RIGHT, TARGET = 16, 36, 1.0

W1 = [[0.5, -0.4],        # left reading into hidden 1, hidden 2
      [-0.3, 0.6]]        # right reading into hidden 1, hidden 2
B1 = [0.1, -0.2]
W2 = [0.7, -0.5]          # hidden 1, hidden 2 into the output
B2 = 0.2

def scale(cm):
    return (min(cm, 50) - 30) / 20

def forward(x1, x2, w1_00=None, w2_0=None):
    a = W1[0][0] if w1_00 is None else w1_00      # so one weight can be changed
    b = W2[0] if w2_0 is None else w2_0
    z1 = x1 * a + x2 * W1[1][0] + B1[0]
    h1 = math.tanh(z1)
    z2 = x1 * W1[0][1] + x2 * W1[1][1] + B1[1]
    h2 = math.tanh(z2)
    z = h1 * b + h2 * W2[1] + B2
    return h1, h2, z, math.tanh(z)

x1, x2 = scale(LEFT), scale(RIGHT)
h1, h2, z, y = forward(x1, x2)
loss = (y - TARGET) ** 2
print("inputs:      x1", round(x1, 3), " x2", round(x2, 3))
print("hidden:      h1", round(h1, 4), " h2", round(h2, 4))
print("output:      z", round(z, 4), " y", round(y, 4), " wanted", TARGET)
print("loss:       ", round(loss, 4))

dy = 2 * (y - TARGET)                 # how the loss changes if the output changes
dz = dy * (1 - y * y)                 # back through the output's tanh
dW2_0 = dz * h1                       # the output weight's own share
dh1 = dz * W2[0]                      # the same error, passed back to hidden 1
dz1 = dh1 * (1 - h1 * h1)             # back through hidden 1's tanh
dW1_00 = dz1 * x1                     # the input weight's share
print("dy", round(dy, 4), " dz", round(dz, 4), " dW2_0", round(dW2_0, 4))
print("dh1", round(dh1, 4), " dz1", round(dz1, 4), " dW1_00", round(dW1_00, 4))

STEP = 0.001                          # the check: nudge each weight and see what the loss does
for name, gradient, nudged in (("W2[0]", dW2_0, forward(x1, x2, w2_0=W2[0] + STEP)),
                               ("W1[0][0]", dW1_00, forward(x1, x2, w1_00=W1[0][0] + STEP))):
    measured = ((nudged[3] - TARGET) ** 2 - loss) / STEP
    print(name, " chain rule", round(gradient, 4), " nudge test", round(measured, 4))

for i in range(41):                   # the loss for every value each weight could take
    s = -2.0 + i * 0.1
    plot("output weight", (forward(x1, x2, w2_0=s)[3] - TARGET) ** 2)
    plot("input weight", (forward(x1, x2, w1_00=s)[3] - TARGET) ** 2)
    wait(0.05)

The two lines on the chart are the loss for this one view as each weight is swept from -2 to +2, and each line is flat where tanh has saturated and steep in between. The weights the network is actually carrying, 0.7 and 0.5, are a little right of the middle of the sweep, on the way up. The gradients printed above are the slopes of those two lines at exactly those points.

The wire in red is the one the text works through. Here are its five rates in the order they are multiplied, with the running product underneath.

Why this is gradient descent, one layer at a time

Nothing in that chain is special to networks. It is the slope of the loss with respect to one weight, which is what the gradient descent guide calls the gradient, and once you have it the step is the same one line: weight = weight − rate × share.

What backpropagation adds is order. Work out the share of every weight separately, by nudging each one and running the whole network again, and a network with a million weights needs a million passes. Backpropagation gets all of them from one pass forwards and one pass backwards, because the middle of the chain is shared. dz, the error at the output's total, is worked out once and used by every weight behind it. dh1 is worked out once and used by every weight feeding hidden 1. That is the whole trick, and it is the reason large networks can be trained at all.

Training the network

Eight views, recorded by the robot on the mat used in the k-nearest neighbours guide: open space twice, a wall straight ahead twice, the wall ending on the left twice and on the right twice. Each one has the turn it should have made.

The weights start small and random. They cannot all start at zero, or at the same value: two hidden neurons that start identical get identical shares of the error for ever, and stay identical, so the second one is wasted. A little randomness breaks that.

Each epoch runs all eight views forwards, adds up each weight's share of the error over all eight, and takes one step. The backward part is the seven lines under # backward, which are the chain above written for every weight at once.

from bugbot import *
import math, random
connect()

# eight views the robot has recorded, each as two numbers: how near the nearest
# thing is on the left half of its view and on the right half, in cm, and the
# turn it should have made (-1 hard left, 0 straight ahead, +1 hard right)
EXAMPLES = [
    (51, 51, 0.0),      # open space
    (37, 37, 0.0),      # open space, the far wall in view
    (16, 16, 1.0),      # wall straight ahead: turn away
    (25, 25, 1.0),
    (36, 16, -1.0),     # the wall ends on the left: go left
    (42, 29, -1.0),
    (16, 36, 1.0),      # the wall ends on the right: go right
    (29, 42, 1.0),
]

LR = 0.5             # the learning rate
EPOCHS = 400
H = 2                # hidden neurons: try 1, and try 4

def scale(cm):
    return (min(cm, 50) - 30) / 20        # 30 cm becomes 0, 50 cm or more becomes 1

DATA = [([scale(l), scale(r)], t) for l, r, t in EXAMPLES]

rnd = random.Random(1)                     # small random weights to start
W1 = [[rnd.uniform(-0.6, 0.6) for j in range(H)] for i in range(2)]
B1 = [0.0] * H
W2 = [rnd.uniform(-0.6, 0.6) for j in range(H)]
B2 = 0.0
print("W1 at the start:", [[round(w, 3) for w in row] for row in W1])
print("W2 at the start:", [round(w, 3) for w in W2])

def forward(x):
    h = [math.tanh(x[0] * W1[0][j] + x[1] * W1[1][j] + B1[j]) for j in range(H)]
    z = sum(h[j] * W2[j] for j in range(H)) + B2
    return h, math.tanh(z)

for epoch in range(1, EPOCHS + 1):
    loss = 0.0
    gW1 = [[0.0] * H for i in range(2)]
    gB1 = [0.0] * H
    gW2 = [0.0] * H
    gB2 = 0.0
    for x, target in DATA:
        h, y = forward(x)                       # forward: what the network says
        loss = loss + (y - target) ** 2
        # backward
        dz = 2 * (y - target) * (1 - y * y) / len(DATA)   # the error at the output
        for j in range(H):
            gW2[j] = gW2[j] + dz * h[j]                   # each output weight's share
            dh = dz * W2[j] * (1 - h[j] * h[j])           # the error, passed back a layer
            gW1[0][j] = gW1[0][j] + dh * x[0]             # each input weight's share
            gW1[1][j] = gW1[1][j] + dh * x[1]
            gB1[j] = gB1[j] + dh
        gB2 = gB2 + dz
    loss = loss / len(DATA)
    for j in range(H):                          # the step: every weight a little downhill
        W2[j] = W2[j] - LR * gW2[j]
        B1[j] = B1[j] - LR * gB1[j]
        W1[0][j] = W1[0][j] - LR * gW1[0][j]
        W1[1][j] = W1[1][j] - LR * gW1[1][j]
    B2 = B2 - LR * gB2
    plot("loss", loss)
    wait(0.02)                                  # so the chart spreads the epochs out
    if epoch % 50 == 0:
        print("epoch", epoch, " loss", round(loss, 4))

print("W1 at the end:", [[round(w, 3) for w in row] for row in W1])
print("B1:", [round(b, 3) for b in B1], " W2:", [round(w, 3) for w in W2], " B2:", round(B2, 3))
for (l, r, t), (x, _) in zip(EXAMPLES, DATA):
    h, y = forward(x)
    print("left", l, " right", r, " wanted", t, " said", round(y, 2), " hidden", [round(v, 2) for v in h])

The loss starts at 0.73, which is what you get from a network that says much the same thing whatever it sees. It is down to 0.05 by epoch 20, and then it nearly stops: 0.037 at epoch 50, 0.034 at epoch 100. After epoch 120 it falls away again, to 0.007 at 200 and 0.0007 at 400. That flat stretch is worth noticing. The weights were still moving, still downhill, through a part of the landscape where the loss hardly changes. Training runs on real networks spend a lot of their time there, and it is why people do not stop a run at the first sign of a flat line.

The nine weights come out as:

W1 = [[-1.026, 2.366],     left reading into hidden 1, hidden 2
      [1.478, -0.019]]     right reading into hidden 1, hidden 2
B1 = [0.347, 0.21]
W2 = [2.052, -1.999]       hidden 1, hidden 2 into the output
B2 = 0.603

Small enough to read, so read them. Hidden 2 has a weight of 2.366 on the left reading and -0.019 on the right: it has learned to watch the left side alone, and its value tracks the left reading exactly, from -0.89 when the left is 16 cm to 0.99 when it is 51. Hidden 1 has weights of opposite sign, -1.026 and 1.478, so it measures which side is nearer. The output adds 2.052 of the first and subtracts 1.999 of the second, with a bias of 0.603.

Read that as a sentence: turn towards the side with more room, but stop turning when the left is wide open. Nobody wrote that rule. It came out of eight examples and 400 steps.

Why two hidden neurons

Change H to 1 and run it again. The loss stops falling at 0.039 and stays there. With one hidden neuron the whole network is one squashed weighted sum, and the eight views cannot all be got right with one: open space comes out as -0.39, a left turn nobody asked for. Change H to 4 and the loss reaches 0.0005, barely better than 2. Two is what this problem needs, and the neural network guide has the picture of why one is not enough.

Driving with it

The same program, with 15 seconds of driving on the end. Every tenth of a second the robot reads its depth grid, takes the nearest reading in each half of the level rows, puts the two numbers through the trained network and turns at 60 times the answer while driving forward at 50. The path is drawn on the mat as a blue line, and the chart shows the network's output and the nearest reading in centimetres.

from bugbot import *
import math, random
connect()

EXAMPLES = [
    (51, 51, 0.0),      # open space
    (37, 37, 0.0),      # open space, the far wall in view
    (16, 16, 1.0),      # wall straight ahead: turn away
    (25, 25, 1.0),
    (36, 16, -1.0),     # the wall ends on the left: go left
    (42, 29, -1.0),
    (16, 36, 1.0),      # the wall ends on the right: go right
    (29, 42, 1.0),
]

LR = 0.5
EPOCHS = 400
H = 2

def scale(cm):
    return (min(cm, 50) - 30) / 20

DATA = [([scale(l), scale(r)], t) for l, r, t in EXAMPLES]

rnd = random.Random(1)
W1 = [[rnd.uniform(-0.6, 0.6) for j in range(H)] for i in range(2)]
B1 = [0.0] * H
W2 = [rnd.uniform(-0.6, 0.6) for j in range(H)]
B2 = 0.0

def forward(x):
    h = [math.tanh(x[0] * W1[0][j] + x[1] * W1[1][j] + B1[j]) for j in range(H)]
    z = sum(h[j] * W2[j] for j in range(H)) + B2
    return h, math.tanh(z)

for epoch in range(1, EPOCHS + 1):
    loss = 0.0
    gW1 = [[0.0] * H for i in range(2)]
    gB1 = [0.0] * H
    gW2 = [0.0] * H
    gB2 = 0.0
    for x, target in DATA:
        h, y = forward(x)
        loss = loss + (y - target) ** 2
        dz = 2 * (y - target) * (1 - y * y) / len(DATA)
        for j in range(H):
            gW2[j] = gW2[j] + dz * h[j]
            dh = dz * W2[j] * (1 - h[j] * h[j])
            gW1[0][j] = gW1[0][j] + dh * x[0]
            gW1[1][j] = gW1[1][j] + dh * x[1]
            gB1[j] = gB1[j] + dh
        gB2 = gB2 + dz
    loss = loss / len(DATA)
    for j in range(H):
        W2[j] = W2[j] - LR * gW2[j]
        B1[j] = B1[j] - LR * gB1[j]
        W1[0][j] = W1[0][j] - LR * gW1[0][j]
        W1[1][j] = W1[1][j] - LR * gW1[1][j]
    B2 = B2 - LR * gB2
    plot("loss", loss)
    wait(0.02)
    if epoch % 100 == 0:
        print("epoch", epoch, " loss", round(loss, 4))

# now drive with it
trail = []
for tick in range(150):              # 15 seconds
    view = tof_grid()[16:32]
    left = min(view[0:4] + view[8:12])      # nearest thing on the left of the view
    right = min(view[4:8] + view[12:16])    # and on the right
    h, turn = forward([scale(left), scale(right)])
    drive(50, 0, 60 * turn)
    plot("turn", turn)
    plot("nearest", min(view))
    x, y = position()
    trail.append((50 + x, 10 + y))
    draw("path", trail, "blue", "line")
    if tick % 15 == 0:
        print("t", tick / 10, " left", left, " right", right, " turn", round(turn, 2))
    wait(0.1)
stop()

The robot drives straight while both readings are far, turns hard when the wall comes up, and goes round in loops in front of it. The turn line sits at about -0.01 in the open and jumps to 0.9 or more as the wall closes. It is much like the behaviour the hand-set weights in the neural network guide produce, except that nobody set these weights.

Two numbers are a thin summary of a view, and a network can only be as good as the numbers it is given. Among the 34 views recorded in lesson 8.2, one labelled gap-left reads 24 cm on the left and 29 on the right, and one labelled gap-right reads 22 and 24: nearly the same pair of numbers, opposite answers. No network reading only these two numbers can tell those apart, however long it trains. That is a choice about the inputs, not a failure of backpropagation, and it is why the network in the neural network guide takes all 16 readings.

What backpropagation is not

• It is not the learning. It works out the gradients. Gradient descent, or Adam, or whatever optimiser you use, does the learning with them.
• It is not only for neural networks. Any calculation built from steps with known rates can be differentiated the same way. Modern libraries call it automatic differentiation and apply it to anything you write.
• It is not how brains work. The name suggests a signal travelling back down the same wires, and no one has found that in a brain.

The one thing that does go wrong on purpose is worth knowing. Every link in the chain multiplies, and tanh contributes 1 − y², which is less than 1 and close to 0 at the flat ends. Multiply thirty of those together and a weight in an early layer gets a share of almost nothing, so it barely moves. This is the vanishing gradient, it is why deep networks were stuck for years, and it is most of the reason ReLU, which has a rate of exactly 1 for positive inputs, replaced the squashing activations in deep networks.

Where this is taught

• A tiny network trains a larger network the same way, with the four backward lines written in numpy.
• See as numbers is where the depth readings this network eats come from.
• Collecting data records views with labels, which is what training needs before anything else.
• Project: situations puts a trained network in charge of a robot.
• Learning a behaviour sets out when a robot should learn a rule rather than be given one.
• Generalisation is about whether a trained network is right on views it has never seen.

Questions

What is backpropagation in simple terms?

It is the method a network uses to find out how much each weight contributed to its error. The error is worked out at the output and passed backwards layer by layer, and each weight ends up with a number saying which way it should move and how strongly.

How does backpropagation work, step by step?

Run the inputs forwards and keep every intermediate value. Work out the error at the output. Multiply it by the rate of the output's activation to get the error at the output's total. Hand each incoming weight its share, which is that error times the value on its wire. Pass the error back through the weights to the layer before, multiply by that layer's activation rate, and repeat until the first layer.

What is the chain rule in backpropagation?

The rule that rates multiply along a chain. A weight in the first layer changes a hidden total, which changes a hidden number, which changes the output's total, which changes the output, which changes the loss. Multiply those five rates and you have how much the loss changes when that weight changes.

What is the difference between backpropagation and gradient descent?

Backpropagation finds the gradients. Gradient descent uses them to change the weights. They are always used together, which is why the two names get swapped, but one is the arithmetic of blame and the other is the arithmetic of the step.

Why do weights need random starting values?

Because two neurons in the same layer that start with the same weights get the same share of the error, take the same step, and stay identical for ever. Random starting values break that symmetry so the neurons can learn different things.

What is a forward pass and a backward pass?

The forward pass runs the inputs through the network to get an answer, keeping the intermediate values. The backward pass runs the error back through the same network to get the gradients. Both are needed once per training step.

What is the vanishing gradient problem?

Every layer's activation multiplies the gradient by its own rate, which for tanh or a sigmoid is less than 1. Over many layers the product shrinks towards zero and the early layers stop learning. ReLU activations, skip connections and careful starting weights are the usual answers.

How do you check backpropagation is right?

Nudge one weight by a tiny amount, run the network forwards again, and see how much the loss changed. Divide by the nudge. That number should match the gradient backpropagation gave you, to several decimal places. The first demo on this page does exactly that, and it is what you should do to any gradient code you write yourself.

Who invented backpropagation?

The mathematics is older, but the 1986 paper by David Rumelhart, Geoffrey Hinton and Ronald Williams is the one that showed it learning useful hidden features and made it standard. Hinton shared the 2024 Nobel Prize in Physics with John Hopfield, for the discoveries that made machine learning with neural networks possible.

Do I have to write backpropagation myself?

No. PyTorch, TensorFlow and JAX work the gradients out for you from whatever calculation you write. Writing it once by hand, as this page does, is how you find out there is nothing hidden in it.

Learn it step by step

These lessons build the same ideas one at a time, each with tasks the simulator marks.

1. 8.4 A tiny network Learning, Robot club
2. 8.1 See as numbers Learning, Robot club
3. 8.2 Collecting data Learning, Robot club
4. 8.8 Project: situations Learning, Robot club
5. U12.1 Learning a behaviour Learning, and the capstone, University
6. U12.3 Generalisation Learning, and the capstone, University