Overfitting explained
What overfitting and underfitting look like, and how more data, fewer parameters and early stopping each help, shown on a robot that measures how fast it turns. Fit a model to a short recording, press Run, and watch it fail on a fresh drive.
Overfitting is when a model learns the data you gave it instead of the job you wanted it to do. It gets the examples in front of it perfectly right and the next thing you ask it badly wrong. It is the reason machine learning is always tested on data the model has never seen, and the reason a model that scores 100 percent is more often a warning than a result.
On this page a small robot measures how fast it turns. It spins on the spot at a few commands, times the turn with its gyro, and fits a model of turn rate against command. Then it spins again at commands the model has never been given, and the model has to say what will happen. Each demo below is a real program you can change and run.
The chart under each demo has three lines. measured is what the robot actually did, on every command it tried. line and bumps are what the two models said would happen, drawn only for the second half of the run, the fresh commands. A model that has learned the job keeps close to the measured line there. A model that has learned the data does not.
The two sets
Everything here rests on one split.
the recording (the training set): the measurements the model is fitted to
the fresh drive (the test set): measurements it has never seen, used to judge it
The error on the recording is called the training error. It is easy to make small and it proves nothing on its own, because the model was built out of those very numbers. The error on the fresh drive is the one that means something, and the rule is absolute: the fresh measurements may not touch the fit in any way.
The job, and three models
One measurement is a spin: hold a turn command for 0.4 s so the robot is up to speed, then read the gyro's heading, hold for another 0.8 s, read it again, and divide the change by 0.8. That gives the turn rate in degrees a second. The robot stays on its own spot the whole time, so the mat never gets in the way.
The models take a command from 20 to 100 and answer with a turn rate.
- flat, one parameter: the average of every rate in the recording. It says the same thing whatever you ask.
- line, two parameters:
rate = slope × command + intercept, fitted by least squares. The lesson Fitting a model to data does this fit on the forward drive. - bumps, six parameters: six bumps spread across the command range, one weight each. A bump is 1 at its own command and fades to nothing about 10 commands away, so each weight is almost free to say what it likes about its own command and says nothing about the rest. The weights are trained by going over the recording again and again, nudging each one down its error, which is what the lesson A tiny network does with a network's weights.
Counting parameters is counting the numbers the fit is allowed to choose. Six points and six free numbers means the model can pass exactly through every point it has.
The recording, then the fresh drive
The recording is six commands: 20, 35, 50, 65, 80 and 95. The fresh drive is the six in between: 25, 40, 55, 70, 85 and 100.
The program
from bugbot import *
import math
connect()
# change these and press Run
RECORD = [20, 35, 50, 65, 80, 95] # the commands the recording uses
FRESH = [25, 40, 55, 70, 85, 100] # the commands of the fresh drive
CENTRES = [20, 35, 50, 65, 80, 95] # the bumpy model has one bump at each
WIDTH = 8.0 # how wide each bump is, in command units
STEP = 0.1 # how far each pass moves the weights
PASSES = 300 # how many passes over the recording
DT = 0.1
def wrap(a):
return (a + 180) % 360 - 180
def turn_rate(cmd):
# spin at this command, let it settle, then time the turn with the gyro
for i in range(4):
drive(0, 0, cmd)
wait(DT)
start = imu()[0]
for i in range(8):
drive(0, 0, cmd)
wait(DT)
return wrap(imu()[0] - start) / 0.8
def fit_flat(data):
# one parameter: the average of every rate in the recording
return sum(v for c, v in data) / len(data)
def fit_line(data):
# two parameters, by least squares: the best slope and intercept
n = len(data)
sc = sum(c for c, v in data)
sv = sum(v for c, v in data)
scc = sum(c * c for c, v in data)
scv = sum(c * v for c, v in data)
slope = (n * scv - sc * sv) / (n * scc - sc * sc)
return slope, (sv - slope * sc) / n
def bumps(c):
# how strongly each bump answers to this command
return [math.exp(-((c - m) / WIDTH) ** 2) for m in CENTRES]
def bumpy(w, c):
return sum(wi * b for wi, b in zip(w, bumps(c)))
def fit_bumps(data):
# one weight per bump, nudged down its error again and again
w = [0.0] * len(CENTRES)
for p in range(PASSES):
for c, v in data:
b = bumps(c)
wrong = bumpy(w, c) - v
for j in range(len(w)):
w[j] = w[j] - STEP * wrong * b[j]
return w
def rms(model, data):
return math.sqrt(sum((model(c) - v) ** 2 for c, v in data) / len(data))
# the recording
for i in range(6):
drive(0, 0, 60)
wait(DT)
recording = []
for c in RECORD:
v = turn_rate(c)
recording.append((c, v))
plot("measured", v)
stop()
flat = fit_flat(recording)
slope, intercept = fit_line(recording)
w = fit_bumps(recording)
models = [("flat", lambda c: flat), ("line", lambda c: slope * c + intercept),
("bumps", lambda c: bumpy(w, c))]
print("recording:", len(recording), "points")
print("flat: ", round(flat, 1), " line:", round(slope, 3), "x command +", round(intercept, 2))
print("bumps:", [round(x) for x in w])
for name, f in models:
print("on the recording,", name, "is out by", round(rms(f, recording), 2))
# the fresh drive
fresh = []
for c in FRESH:
v = turn_rate(c)
fresh.append((c, v))
plot("measured", v)
plot("line", slope * c + intercept)
plot("bumps", bumpy(w, c))
print("command", c, "measured", round(v, 1),
" flat", round(flat, 1), " line", round(slope * c + intercept, 1),
" bumps", round(bumpy(w, c), 1))
stop()
for name, f in models:
print("on the fresh drive,", name, "is out by", round(rms(f, fresh), 2))
print("clock", round(clock(), 1))
The two errors, in degrees a second, are the whole story:
| Model | Parameters | Out by, on the recording | Out by, on the fresh drive |
|---|---|---|---|
| flat | 1 | 30.9 | 33.11 |
| line | 2 | 2.7 | 4.16 |
| bumps | 6 | 0.0 | 25.11 |
The bumpy model wins the recording and loses everything else. It passes through all six points exactly, because it has one free number for each of them, and then between them it has nothing to say: at command 40 it answers 37.9 where the robot did 50.2, and at command 100, past the end of the recording, it answers 78.5 where the robot did 126.9. It is not confused. It was never asked about those commands, and it was given enough freedom to ignore them.
The flat model is the opposite mistake, and it has its own name: underfitting. One parameter cannot represent a rate that climbs from 30 to 125, so it is out by about 30 wherever you ask. The tell is that its two errors are the same. A model that is equally bad on the data it was fitted to and on fresh data is not overfitting, it is too simple.
Being wrong in these two ways has a standard name, the bias and variance trade-off. Too few parameters and the model cannot follow the truth, so it is wrong in the same way every time: that is bias. Too many and it follows the noise in whatever measurements it happened to get, so it is wrong in a different way every time you fit it: that is variance. The error on fresh data is the sum of the two, and the best model is somewhere in the middle.
Overfitting looks like success
This is the part worth remembering. Nothing in the first demo's fit looks wrong. The bumpy model is more flexible than the line, which is the reason anyone would use it, and the robot's real turn rate is not quite straight, so there is something for the extra flexibility to find. The fit converges. The training error goes to zero.
If you never drive again, you will never know. Overfitting is invisible from inside the recording. The only thing that shows it is a measurement the model was not fitted to.
More data, same model
The flexibility only becomes freedom when there are more parameters than the data can pin down. Give the same six bumps a longer recording, sixteen commands instead of six, and each weight now has several measurements pulling on it.
The program
from bugbot import *
import math
connect()
# change these and press Run
RECORD = [20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95]
FRESH = [25, 40, 55, 70, 85, 100] # the commands of the fresh drive
CENTRES = [20, 35, 50, 65, 80, 95] # the bumpy model has one bump at each
WIDTH = 8.0 # how wide each bump is, in command units
STEP = 0.1 # how far each pass moves the weights
PASSES = 300 # how many passes over the recording
DT = 0.1
def wrap(a):
return (a + 180) % 360 - 180
def turn_rate(cmd):
# spin at this command, let it settle, then time the turn with the gyro
for i in range(4):
drive(0, 0, cmd)
wait(DT)
start = imu()[0]
for i in range(8):
drive(0, 0, cmd)
wait(DT)
return wrap(imu()[0] - start) / 0.8
def fit_flat(data):
# one parameter: the average of every rate in the recording
return sum(v for c, v in data) / len(data)
def fit_line(data):
# two parameters, by least squares: the best slope and intercept
n = len(data)
sc = sum(c for c, v in data)
sv = sum(v for c, v in data)
scc = sum(c * c for c, v in data)
scv = sum(c * v for c, v in data)
slope = (n * scv - sc * sv) / (n * scc - sc * sc)
return slope, (sv - slope * sc) / n
def bumps(c):
# how strongly each bump answers to this command
return [math.exp(-((c - m) / WIDTH) ** 2) for m in CENTRES]
def bumpy(w, c):
return sum(wi * b for wi, b in zip(w, bumps(c)))
def fit_bumps(data):
# one weight per bump, nudged down its error again and again
w = [0.0] * len(CENTRES)
for p in range(PASSES):
for c, v in data:
b = bumps(c)
wrong = bumpy(w, c) - v
for j in range(len(w)):
w[j] = w[j] - STEP * wrong * b[j]
return w
def rms(model, data):
return math.sqrt(sum((model(c) - v) ** 2 for c, v in data) / len(data))
# the recording
for i in range(6):
drive(0, 0, 60)
wait(DT)
recording = []
for c in RECORD:
v = turn_rate(c)
recording.append((c, v))
plot("measured", v)
stop()
flat = fit_flat(recording)
slope, intercept = fit_line(recording)
w = fit_bumps(recording)
models = [("flat", lambda c: flat), ("line", lambda c: slope * c + intercept),
("bumps", lambda c: bumpy(w, c))]
print("recording:", len(recording), "points")
print("flat: ", round(flat, 1), " line:", round(slope, 3), "x command +", round(intercept, 2))
print("bumps:", [round(x) for x in w])
for name, f in models:
print("on the recording,", name, "is out by", round(rms(f, recording), 2))
# the fresh drive
fresh = []
for c in FRESH:
v = turn_rate(c)
fresh.append((c, v))
plot("measured", v)
plot("line", slope * c + intercept)
plot("bumps", bumpy(w, c))
print("command", c, "measured", round(v, 1),
" flat", round(flat, 1), " line", round(slope * c + intercept, 1),
" bumps", round(bumpy(w, c), 1))
stop()
for name, f in models:
print("on the fresh drive,", name, "is out by", round(rms(f, fresh), 2))
print("clock", round(clock(), 1))
The training error is no longer zero. It cannot be: sixteen measurements will not sit on a curve with six knobs, and the noise in them now has to be averaged rather than copied. That is the good news. Read down the printed rows and the model is out by between 2 and 9 on the fresh commands from 25 to 85, where before it was out by between 11 and 26.
Command 100 is the exception, and it is a different problem. It is past the end of the recording, so every bump has faded and the model is guessing about a place no data has ever reached. Extrapolation is not something more data inside the range can fix. Record the range you mean to use, and write down the range a model is good for.
Fewer parameters
The other lever is the model. The line has two parameters and cannot chase noise with them, which is why it is out by only 4.2 on fresh commands while the bumpy model is out by 25.1. Fewer parameters means less freedom to be wrong in an interesting way.
You can see the same lever inside the bumpy model. WIDTH is not a parameter that the fit chooses, it is a choice you make before fitting, and making the bumps wider ties neighbouring weights together so that the model cannot bend so sharply. Set WIDTH = 20 in the first demo and its fresh drive error falls from 25.1 to 9.4, with the training error rising only to 1.4. Choices like that are called hyperparameters, and the number of neighbours k in k-nearest neighbours is another: a small k follows every odd example, a large one smooths them together. The lesson Nearest neighbour has a classifier to try it on.
Keeping a model small on purpose is called regularisation. Fewer parameters, wider bumps, a rule that pushes weights towards zero unless the data insists: they are all the same idea, which is to make it harder for the model to say something the data does not support.
Stopping early
The third lever costs nothing. The bumpy model is trained by repeated passes over the recording, and the damage is not done at once. This demo measures a recording and a held-out set first, fits the line, and then trains six bumps on what the line missed, checking both errors after every pass.
The program
from bugbot import *
import math
connect()
# change these and press Run
RECORD = [20, 35, 50, 65, 80, 95] # the recording
HELD_OUT = [25, 40, 55, 70, 85, 100] # kept back, and never fitted to
CENTRES = [20, 35, 50, 65, 80, 95]
WIDTH = 8.0
STEP = 0.05 # how far each pass moves the weights
PASSES = 60
DT = 0.1
def wrap(a):
return (a + 180) % 360 - 180
def turn_rate(cmd):
for i in range(4):
drive(0, 0, cmd)
wait(DT)
start = imu()[0]
for i in range(8):
drive(0, 0, cmd)
wait(DT)
return wrap(imu()[0] - start) / 0.8
def fit_line(data):
n = len(data)
sc = sum(c for c, v in data)
sv = sum(v for c, v in data)
scc = sum(c * c for c, v in data)
scv = sum(c * v for c, v in data)
slope = (n * scv - sc * sv) / (n * scc - sc * sc)
return slope, (sv - slope * sc) / n
def bumps(c):
return [math.exp(-((c - m) / WIDTH) ** 2) for m in CENTRES]
def guess(w, slope, intercept, c):
# the line, plus whatever the bumps have learned on top of it
return slope * c + intercept + sum(wi * b for wi, b in zip(w, bumps(c)))
def rms(w, slope, intercept, data):
return math.sqrt(sum((guess(w, slope, intercept, c) - v) ** 2 for c, v in data) / len(data))
for i in range(6):
drive(0, 0, 60)
wait(DT)
recording = [(c, turn_rate(c)) for c in RECORD]
held = [(c, turn_rate(c)) for c in HELD_OUT]
stop()
slope, intercept = fit_line(recording)
w = [0.0] * len(CENTRES)
best, best_pass = 1e9, 0
for p in range(PASSES):
for c, v in recording:
b = bumps(c)
wrong = guess(w, slope, intercept, c) - v
for j in range(len(w)):
w[j] = w[j] - STEP * wrong * b[j]
on_recording = rms(w, slope, intercept, recording)
on_held = rms(w, slope, intercept, held)
plot("recording", on_recording)
plot("held out", on_held)
if on_held < best:
best, best_pass = on_held, p + 1
if p % 10 == 0 or p == PASSES - 1:
print("pass", p + 1, " recording", round(on_recording, 2), " held out", round(on_held, 2))
wait(0.2)
print("best held-out error", round(best, 2), "after", best_pass, "passes")
print("clock", round(clock(), 1))
The two lines part company. The error on the recording keeps falling, all the way from 2.57 to 0.15, which looks like a model getting better and is a model copying six particular measurements. The held-out error reaches its lowest point, 4.13, after nine passes, and from there it creeps up. Every pass after the ninth made the robot's model better at the recording and no better at turning.
Stopping at the bottom of the held-out line is early stopping, and it is standard practice when a model is trained in steps. The rise here is small, because a single fresh measurement carries a few degrees a second of noise of its own and that noise sets a floor no model can get under. The shape is the point: watch the held-out error, not the training error, and stop when it stops falling.
Testing honestly
- Never judge a model on the data it was fitted to. Split the measurements before you start.
- Do not choose with your test set either. If you try five models, look at the test error and keep the winner, you have fitted the test set by hand. The usual arrangement is three sets: train to fit, validation to choose between models, and a test set opened once, at the end.
- With very little data, use cross validation. Split the recording into
kparts, fitktimes leaving each part out, and average thekheld-out errors. Every measurement helps fit and gets tested, and nothing is contaminated. The lesson Generalisation works through this on a robot. - Test where you will use it. A model tested only inside the range it was fitted to tells you nothing about the edges.
- On a robot, test on a fresh run. Measurements taken seconds apart share a battery level, a mat and a warm motor. A test set cut out of one recording is easier than the world will be, which is the reality gap.
Where this is taught
- Collecting data: recording examples on the robot, and what makes a set of them good.
- Nearest neighbour: the simplest model that keeps its data, and what happens where it has none.
- A tiny network: weights nudged down their error, the fit used here.
- Fitting a model to data: least squares on a robot, and the range a model is good for.
- Generalisation: training error against held-out error, model choice and cross validation.
- The reality gap, and honest evaluation: why a model that passes in one place fails in another.
Questions
What is overfitting in simple terms?
A model has overfitted when it has learned the particular measurements it was given, including their noise, instead of the pattern behind them. It scores well on those measurements and badly on new ones. On this page a six parameter model fitted six measurements exactly and was then out by 25 degrees a second on six fresh commands, where a two parameter model was out by 4.
How do you know if a model is overfitting?
Compare two numbers: the error on the data it was fitted to and the error on data it has never seen. Overfitting is a small training error beside a much larger test error. If both are large the model is underfitting instead.
What is the difference between overfitting and underfitting?
Overfitting is too much freedom: the model follows the noise, so it is excellent on the training set and poor on new data. Underfitting is too little: the model cannot follow the pattern at all, so it is poor on both. The flat model on this page is out by about 30 degrees a second on the recording and on the fresh drive alike, which is the signature of underfitting.
How do you prevent overfitting?
Four levers, in the order most people should try them. Get more data, so each parameter has several measurements pulling on it. Use fewer parameters, or tie them together with a regulariser. Stop training when the held-out error stops falling. And keep a test set the model never touches, so you find out either way.
Does more data always fix overfitting?
More data inside the range you measured, yes, mostly: it leaves the model less freedom to invent. More data does not fix a question outside that range. In the second demo a recording of sixteen commands from 20 to 95 cut the error between those commands by about two thirds and changed nothing at command 100, which no measurement had ever reached.
What is the bias variance trade-off?
The error on new data splits into two parts. Bias is being wrong the same way every time, because the model is too simple to represent the truth. Variance is being wrong a different way every time, because the model is chasing whatever noise this particular data set had. Adding parameters cuts bias and raises variance, and the best model is at the bottom of the sum of the two.
What is early stopping?
A model trained in repeated passes fits the broad pattern first and the noise later. Early stopping means checking the error on held-out data after each pass and keeping the weights from the pass where that error was lowest, rather than the weights from the last pass. On this page the best held-out error came after nine passes out of sixty.
What is a training set and a test set?
The training set is the data the model is fitted to. The test set is data kept back, which the fit is never allowed to see, and which is used once to say how good the model is. If you also need to choose between models, use a third set, a validation set, for the choosing, so the test set stays clean.
Why does my model work in testing and fail on the robot?
Usually because the test was not as fresh as it looked. Measurements taken in one recording share a battery, a mat, a temperature and a warm motor, so a test set cut out of that recording is not new data in any useful sense. Take a second recording later, on a different day if you can, and test on that.
Can a simple model overfit?
Yes, if the data is small enough. Overfitting is about parameters compared with measurements, not about how clever the model is. Two parameters fitted to two points pass exactly through both and can be nonsense in between. What protects you is measurements per parameter, and a test set.
Is overfitting on the GCSE or A level specification?
Not by name in the GCSE Computer Science specifications, although Pearson Edexcel GCSE asks about the issues raised by artificial intelligence and machine learning, and a model that scores perfectly on its own data is a concrete example. Training and test data appear in A level Computer Science courses that cover machine learning, and in every A level Maths and Psychology treatment of fitting and sampling. It also makes a good A level programming project, because a fit, a held-out set and a chart of both errors are a complete piece of work.
Learn it step by step
These lessons build the same ideas one at a time, each with tasks the simulator marks.
- 8.2 Collecting data Learning, Robot club
- 8.3 Nearest neighbour Learning, Robot club
- 8.4 A tiny network Learning, Robot club
- U12.2 Fitting a model to data Learning, and the capstone, University
- U12.3 Generalisation Learning, and the capstone, University
- U12.6 The reality gap, and honest evaluation Learning, and the capstone, University