A tiny network
Weights, a loss, and watching it fall: training in the browser.
Do this lesson in the simulatorNearest neighbour keeps all the data and compares against it every time. A neural network keeps a set of numbers called weights instead, learned from the data once, and the data can then be thrown away. This lesson trains one, in the browser, in about a second.
The dataset is bigger now: the four samples from 8.2 plus thirty more recorded around each spot, the kind of thing you get by recording as you drive about. It is long, so it sits at the top of each cell.
Numbers in, numbers out
The network here is two layers. Sixteen inputs (the level readings) are mixed by a table of weights into 8 hidden numbers, each squashed by tanh into the range -1 to 1. Those 8 are mixed again into 4 outputs, one per label. The biggest output is the answer. That is all a network is: multiply, add, squash, repeat.
numpy does the multiplying. X @ W1 multiplies a whole table of samples by a whole table of weights in one go.
# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()
DATA = [
([54, 52, 51, 51, 51, 51, 52, 54, 54, 52, 51, 51, 51, 51, 52, 54], 'open'),
([16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16], 'wall-ahead'),
([36, 47, 46, 46, 46, 16, 16, 16, 36, 47, 46, 46, 46, 16, 16, 16], 'gap-left'),
([16, 16, 16, 46, 46, 46, 47, 36, 16, 16, 16, 46, 46, 46, 47, 36], 'gap-right'),
([400, 400, 61, 61, 61, 61, 400, 400, 96, 62, 61, 61, 61, 61, 62, 64], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([400, 400, 61, 61, 61, 61, 400, 400, 64, 62, 61, 61, 61, 61, 62, 64], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([400, 400, 61, 61, 61, 61, 400, 400, 64, 62, 61, 61, 61, 61, 62, 96], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([24, 34, 55, 59, 59, 59, 29, 30, 24, 33, 55, 59, 59, 59, 29, 30], 'gap-left'),
([24, 34, 51, 51, 51, 51, 25, 22, 24, 33, 51, 51, 51, 51, 25, 22], 'gap-left'),
([24, 34, 43, 43, 43, 43, 44, 18, 24, 33, 43, 43, 43, 43, 44, 18], 'gap-left'),
([42, 58, 59, 59, 29, 29, 29, 30, 42, 58, 59, 59, 29, 29, 29, 30], 'gap-left'),
([42, 52, 51, 51, 21, 21, 21, 22, 42, 52, 51, 51, 21, 21, 21, 22], 'gap-left'),
([42, 44, 43, 43, 13, 13, 13, 13, 42, 44, 43, 43, 13, 13, 13, 13], 'gap-left'),
([30, 29, 29, 29, 59, 59, 58, 42, 30, 29, 29, 29, 59, 59, 58, 42], 'gap-right'),
([22, 21, 21, 21, 51, 51, 52, 42, 22, 21, 21, 21, 51, 51, 52, 42], 'gap-right'),
([13, 13, 13, 13, 43, 43, 44, 42, 13, 13, 13, 13, 43, 43, 44, 42], 'gap-right'),
([30, 29, 59, 59, 59, 55, 34, 24, 30, 29, 59, 59, 59, 55, 33, 24], 'gap-right'),
([22, 25, 51, 51, 51, 51, 34, 24, 22, 25, 51, 51, 51, 51, 33, 24], 'gap-right'),
([18, 44, 43, 43, 43, 43, 34, 24, 18, 44, 43, 43, 43, 43, 33, 24], 'gap-right'),
]
import numpy as np
labels = sorted(set(l for _, l in DATA))
# centred near 0, roughly -1 to +1
X = (np.array([f for f, _ in DATA], dtype=float) - 30) / 20
Y = np.zeros((len(DATA), len(labels)))
for i, (_, l) in enumerate(DATA):
# one-hot: a 1 in the right label's column
Y[i, labels.index(l)] = 1
print("labels:", labels)
print("X is", X.shape, "Y is", Y.shape)
print("first sample:", np.round(X[0], 2), "->", Y[0])
Two things to notice. The readings are shifted and scaled so they sit around zero: networks train badly on raw numbers like 400. And the labels become one-hot rows, a 1 in the column of the right answer, so the network's 4 outputs have something to be compared with.
Untrained
Random weights give random answers:
# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()
DATA = [
([54, 52, 51, 51, 51, 51, 52, 54, 54, 52, 51, 51, 51, 51, 52, 54], 'open'),
([16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16], 'wall-ahead'),
([36, 47, 46, 46, 46, 16, 16, 16, 36, 47, 46, 46, 46, 16, 16, 16], 'gap-left'),
([16, 16, 16, 46, 46, 46, 47, 36, 16, 16, 16, 46, 46, 46, 47, 36], 'gap-right'),
([400, 400, 61, 61, 61, 61, 400, 400, 96, 62, 61, 61, 61, 61, 62, 64], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([400, 400, 61, 61, 61, 61, 400, 400, 64, 62, 61, 61, 61, 61, 62, 64], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([400, 400, 61, 61, 61, 61, 400, 400, 64, 62, 61, 61, 61, 61, 62, 96], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([24, 34, 55, 59, 59, 59, 29, 30, 24, 33, 55, 59, 59, 59, 29, 30], 'gap-left'),
([24, 34, 51, 51, 51, 51, 25, 22, 24, 33, 51, 51, 51, 51, 25, 22], 'gap-left'),
([24, 34, 43, 43, 43, 43, 44, 18, 24, 33, 43, 43, 43, 43, 44, 18], 'gap-left'),
([42, 58, 59, 59, 29, 29, 29, 30, 42, 58, 59, 59, 29, 29, 29, 30], 'gap-left'),
([42, 52, 51, 51, 21, 21, 21, 22, 42, 52, 51, 51, 21, 21, 21, 22], 'gap-left'),
([42, 44, 43, 43, 13, 13, 13, 13, 42, 44, 43, 43, 13, 13, 13, 13], 'gap-left'),
([30, 29, 29, 29, 59, 59, 58, 42, 30, 29, 29, 29, 59, 59, 58, 42], 'gap-right'),
([22, 21, 21, 21, 51, 51, 52, 42, 22, 21, 21, 21, 51, 51, 52, 42], 'gap-right'),
([13, 13, 13, 13, 43, 43, 44, 42, 13, 13, 13, 13, 43, 43, 44, 42], 'gap-right'),
([30, 29, 59, 59, 59, 55, 34, 24, 30, 29, 59, 59, 59, 55, 33, 24], 'gap-right'),
([22, 25, 51, 51, 51, 51, 34, 24, 22, 25, 51, 51, 51, 51, 33, 24], 'gap-right'),
([18, 44, 43, 43, 43, 43, 34, 24, 18, 44, 43, 43, 43, 43, 33, 24], 'gap-right'),
]
import numpy as np
np.random.seed(1)
labels = sorted(set(l for _, l in DATA))
X = (np.array([f for f, _ in DATA], dtype=float) - 30) / 20
Y = np.zeros((len(DATA), len(labels)))
for i, (_, l) in enumerate(DATA):
Y[i, labels.index(l)] = 1
W1 = np.random.randn(16, 8) * 0.5; b1 = np.zeros(8)
W2 = np.random.randn(8, len(labels)) * 0.5; b2 = np.zeros(len(labels))
# the forward pass
out = np.tanh(X @ W1 + b1) @ W2 + b2
acc = np.mean(out.argmax(axis=1) == Y.argmax(axis=1))
print(f"untrained accuracy: {round(acc * 100)}%")
Under half right, and that is luck: with random weights the network is guessing.
Training
Training means: measure how wrong the outputs are (the loss), work out which way each weight should move to make it less wrong (the gradient), and move every weight a little that way. Repeat. Each pass over the data is an epoch.
# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()
DATA = [
([54, 52, 51, 51, 51, 51, 52, 54, 54, 52, 51, 51, 51, 51, 52, 54], 'open'),
([16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16], 'wall-ahead'),
([36, 47, 46, 46, 46, 16, 16, 16, 36, 47, 46, 46, 46, 16, 16, 16], 'gap-left'),
([16, 16, 16, 46, 46, 46, 47, 36, 16, 16, 16, 46, 46, 46, 47, 36], 'gap-right'),
([400, 400, 61, 61, 61, 61, 400, 400, 96, 62, 61, 61, 61, 61, 62, 64], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([400, 400, 61, 61, 61, 61, 400, 400, 64, 62, 61, 61, 61, 61, 62, 64], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([400, 400, 61, 61, 61, 61, 400, 400, 64, 62, 61, 61, 61, 61, 62, 96], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([24, 34, 55, 59, 59, 59, 29, 30, 24, 33, 55, 59, 59, 59, 29, 30], 'gap-left'),
([24, 34, 51, 51, 51, 51, 25, 22, 24, 33, 51, 51, 51, 51, 25, 22], 'gap-left'),
([24, 34, 43, 43, 43, 43, 44, 18, 24, 33, 43, 43, 43, 43, 44, 18], 'gap-left'),
([42, 58, 59, 59, 29, 29, 29, 30, 42, 58, 59, 59, 29, 29, 29, 30], 'gap-left'),
([42, 52, 51, 51, 21, 21, 21, 22, 42, 52, 51, 51, 21, 21, 21, 22], 'gap-left'),
([42, 44, 43, 43, 13, 13, 13, 13, 42, 44, 43, 43, 13, 13, 13, 13], 'gap-left'),
([30, 29, 29, 29, 59, 59, 58, 42, 30, 29, 29, 29, 59, 59, 58, 42], 'gap-right'),
([22, 21, 21, 21, 51, 51, 52, 42, 22, 21, 21, 21, 51, 51, 52, 42], 'gap-right'),
([13, 13, 13, 13, 43, 43, 44, 42, 13, 13, 13, 13, 43, 43, 44, 42], 'gap-right'),
([30, 29, 59, 59, 59, 55, 34, 24, 30, 29, 59, 59, 59, 55, 33, 24], 'gap-right'),
([22, 25, 51, 51, 51, 51, 34, 24, 22, 25, 51, 51, 51, 51, 33, 24], 'gap-right'),
([18, 44, 43, 43, 43, 43, 34, 24, 18, 44, 43, 43, 43, 43, 33, 24], 'gap-right'),
]
import numpy as np
np.random.seed(1)
labels = sorted(set(l for _, l in DATA))
X = (np.array([f for f, _ in DATA], dtype=float) - 30) / 20
Y = np.zeros((len(DATA), len(labels)))
for i, (_, l) in enumerate(DATA):
Y[i, labels.index(l)] = 1
W1 = np.random.randn(16, 8) * 0.5; b1 = np.zeros(8)
W2 = np.random.randn(8, len(labels)) * 0.5; b2 = np.zeros(len(labels))
# how far to move each step
lr = 0.5
for epoch in range(1, 301):
# forward
H = np.tanh(X @ W1 + b1)
Z = H @ W2 + b2
# outputs as probabilities
P = np.exp(Z - Z.max(axis=1, keepdims=True)); P /= P.sum(axis=1, keepdims=True)
loss = -np.mean(np.log(P[np.arange(len(DATA)), Y.argmax(axis=1)] + 1e-9))
# backward: how wrong, per output
dZ = (P - Y) / len(DATA)
dW2 = H.T @ dZ; db2 = dZ.sum(axis=0)
dH = dZ @ W2.T * (1 - H ** 2)
dW1 = X.T @ dH; db1 = dH.sum(axis=0)
# the step
W1 -= lr * dW1; b1 -= lr * db1; W2 -= lr * dW2; b2 -= lr * db2
if epoch % 50 == 0:
print(f"epoch {epoch}: loss {loss:.3f}")
pred = np.tanh(X @ W1 + b1) @ W2 + b2
print(f"accuracy: {round(np.mean(pred.argmax(axis=1) == Y.argmax(axis=1)) * 100)}%")
Watch the loss fall. The four backward lines are the calculus done for you: each is the gradient of the line above it, and you do not need to follow them to use them. The shape is what matters: forward, measure, backward, step.
lr is the learning rate. Too small and the loss creeps; too big and it bounces and never settles. It is the gain from Module 3 again, in a new costume.
Why the centring matters
Change the scaling line to X = np.array(...) / 100 and train again: it sticks at about 88%, confusing gap-left with gap-right. Same network, same data, worse numbers to work with. Getting the inputs into a sensible range is not a detail.
Task: a tiny network
Train the network on DATA, printing epoch <n>: loss <value> every 50 epochs, then print accuracy: <n>% and get it above 90. Do not drive.
# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()
DATA = [
([54, 52, 51, 51, 51, 51, 52, 54, 54, 52, 51, 51, 51, 51, 52, 54], 'open'),
([16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16], 'wall-ahead'),
([36, 47, 46, 46, 46, 16, 16, 16, 36, 47, 46, 46, 46, 16, 16, 16], 'gap-left'),
([16, 16, 16, 46, 46, 46, 47, 36, 16, 16, 16, 46, 46, 46, 47, 36], 'gap-right'),
([400, 400, 61, 61, 61, 61, 400, 400, 96, 62, 61, 61, 61, 61, 62, 64], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([400, 400, 61, 61, 61, 61, 400, 400, 64, 62, 61, 61, 61, 61, 62, 64], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([400, 400, 61, 61, 61, 61, 400, 400, 64, 62, 61, 61, 61, 61, 62, 96], 'open'),
([52, 50, 49, 49, 49, 49, 50, 52, 51, 50, 49, 49, 49, 49, 50, 51], 'open'),
([39, 38, 37, 37, 37, 37, 38, 39, 39, 38, 37, 37, 37, 37, 38, 39], 'open'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([26, 25, 25, 25, 25, 25, 25, 26, 26, 25, 25, 25, 25, 25, 25, 26], 'wall-ahead'),
([18, 17, 17, 17, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17], 'wall-ahead'),
([9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 'wall-ahead'),
([24, 34, 55, 59, 59, 59, 29, 30, 24, 33, 55, 59, 59, 59, 29, 30], 'gap-left'),
([24, 34, 51, 51, 51, 51, 25, 22, 24, 33, 51, 51, 51, 51, 25, 22], 'gap-left'),
([24, 34, 43, 43, 43, 43, 44, 18, 24, 33, 43, 43, 43, 43, 44, 18], 'gap-left'),
([42, 58, 59, 59, 29, 29, 29, 30, 42, 58, 59, 59, 29, 29, 29, 30], 'gap-left'),
([42, 52, 51, 51, 21, 21, 21, 22, 42, 52, 51, 51, 21, 21, 21, 22], 'gap-left'),
([42, 44, 43, 43, 13, 13, 13, 13, 42, 44, 43, 43, 13, 13, 13, 13], 'gap-left'),
([30, 29, 29, 29, 59, 59, 58, 42, 30, 29, 29, 29, 59, 59, 58, 42], 'gap-right'),
([22, 21, 21, 21, 51, 51, 52, 42, 22, 21, 21, 21, 51, 51, 52, 42], 'gap-right'),
([13, 13, 13, 13, 43, 43, 44, 42, 13, 13, 13, 13, 43, 43, 44, 42], 'gap-right'),
([30, 29, 59, 59, 59, 55, 34, 24, 30, 29, 59, 59, 59, 55, 33, 24], 'gap-right'),
([22, 25, 51, 51, 51, 51, 34, 24, 22, 25, 51, 51, 51, 51, 33, 24], 'gap-right'),
([18, 44, 43, 43, 43, 43, 34, 24, 18, 44, 43, 43, 43, 43, 33, 24], 'gap-right'),
]
import numpy as np
X = (np.array([f for f, _ in DATA], dtype=float) - 30) / 20
print('samples:', X.shape)
Challenges
- Try 4 hidden numbers, then 16. Does it still reach 100%? How fast?
- Try
lrof 0.05 and of 3. Print the loss every 10 epochs to see what each does. - Classify the current view: put
level_rows()through the same scaling and the trained weights, and print the label with the biggest output.