A reading is a distribution

Mean, standard deviation, and what one number from a sensor is actually telling you.

U4.1Noise and filteringUniversity25 min

Do this lesson in the simulator

distance() returns 57.2. The wall is not at 57.2. The wall is somewhere near 57, and 57.2 is one draw from a distribution centred on wherever it really is.

Treating a reading as a fact is the assumption that quietly breaks every robot. Treating it as a sample is the start of everything in this module and the next three.

Two numbers describe it

For a well behaved sensor, two numbers are enough:

  • the mean, which is what it says on average, and
  • the standard deviation, sigma, which is how far a single reading usually is from that.

For a Gaussian distribution, about 68 percent of readings land within one sigma, 95 percent within two, and 99.7 percent within three. That last one is where "three sigma" comes from as an engineer's word for "practically never".

Measuring it

Stand still, take a hundred readings, and look at them.

from bugbot import *
connect()

readings = []
for i in range(100):
    readings.append(distance())
    wait(0.1)

mean = sum(readings) / len(readings)
var = sum((r - mean) ** 2 for r in readings) / len(readings)
sigma = var ** 0.5
print("mean", round(mean, 2), "sigma", round(sigma, 2))
print("min", min(readings), "max", max(readings))
print("within 1 sigma:", sum(1 for r in readings if abs(r - mean) < sigma), "of", len(readings))

Run this in the simulator

That last line should come out near 68. When it does not, the distribution is not Gaussian and the two numbers are not enough, which is the case with outliers in U4.4.

Bias against noise, again

A sensor can be precise and wrong: small sigma, mean in the wrong place. That is bias, and no amount of averaging touches it.

  • Noise is fixed by taking more readings.
  • Bias is fixed by calibration against something you trust.

The same distinction as U3.3, and it is worth having reflexively. When a measurement is disappointing, the first question is always: is it scattered, or is it shifted?

A new reading only arrives so often

The depth sensor produces a fresh measurement about every tenth of a second. Read it twice in the same millisecond and you get the same number twice, because it is the same measurement.

That matters for averaging: a hundred reads in a tight loop is not a hundred independent samples, it is the same few samples repeated, and its mean is no better than one of them. Independence is what makes averaging work, and time is what buys independence.

Task: measure the noise

Standing still, print mean: and sigma: for this sensor, in centimetres.

from bugbot import *
connect()

readings = []

Challenges

  1. Print how many readings fall within one, two and three sigma. Is it 68, 95, 99.7?
  2. Change the robot's distance from the wall and measure sigma again. Does the noise grow with range?
  3. Read the sensor a hundred times with no wait at all. What is sigma now, and why is it a lie?