Tuning itself

Learning a controller gain by trial: measure, change, keep the better.

8.6LearningRobot club20 min

Do this lesson in the simulator

Module 3 had you tune a gain by hand: try a number, watch the run, try another. A robot can do that itself. Measure how well a number works, change it, measure again, keep whichever was better. This lesson's robot leaks sideways badly, and it finds its own steering gain.

This robot leaks

Drive up the lane with no steering and watch x:

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

# drive forward at 40 (keeps going until the next command)
forward(40)
# do this 5 times (second counts from 0)
for second in range(5):
    # pause 1 s (the robot keeps doing what it was told)
    wait(1)
    # where am I? (cm from where I started)
    x, y = position()
    print(f"after {second + 1} s: x = {x:.1f}")
# all motors off
stop()

Run this in the simulator

About 2 cm sideways for every second of driving. The lane is 8 cm wide. On its own it is out of the lane in two seconds.

Steering by the error

The controller from lesson 3.2: sideways speed proportional to how far off the centre line the robot is. One trial is a leg up the lane and a leg back, and its score is the mean of abs(x) over the trial. Lower is better.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

def trial(gain):
    errors = []
    # do this 2 times (leg counts from 0)
    for leg in range(2):
        # do this 50 times (tick counts from 0)
        for tick in range(50):
            # where am I? (cm from where I started)
            x, y = position()
            # up 5 s, back 5 s, steering from x
            drive(40 if leg == 0 else -40, -x * gain, 0)
            errors.append(abs(x))
            # pause 0.1 s (the robot keeps doing what it was told)
            wait(0.1)
    return sum(errors) / len(errors)

print("gain 6: mean error", round(trial(6), 1))
print("gain 12: mean error", round(trial(12), 1))
# all motors off
stop()

Run this in the simulator

One thing to know about the motors: a sideways command under about 15 percent does nothing at all, they need a minimum push to move. So a gain of 1 never steers until the robot is 15 cm out, which is why useful gains here are bigger than Module 3's.

The search

Start with a gain. After each trial, compare the score with the best so far. If it improved, keep it and step further the same way. If it got worse, turn round and step the other way from the best. This is hill climbing, and it is the simplest optimiser there is.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

def trial(gain):
    errors = []
    # do this 2 times (leg counts from 0)
    for leg in range(2):
        # do this 50 times (tick counts from 0)
        for tick in range(50):
            # where am I? (cm from where I started)
            x, y = position()
            # forward, sideways, rotation: -100 to 100 each, until the next command
            drive(40 if leg == 0 else -40, -x * gain, 0)
            errors.append(abs(x))
            # pause 0.1 s (the robot keeps doing what it was told)
            wait(0.1)
    return sum(errors) / len(errors)

gain, direction = 2.0, 1
best_gain, best_error = gain, 1e9
# do this 5 times (t counts from 0)
for t in range(5):
    error = trial(gain)
    print(f"trial {t}: gain {gain:.1f}, mean error {error:.1f}")
    if error < best_error:
        best_gain, best_error = gain, error
    else:
        direction = -direction
    gain = max(1.0, best_gain + direction * 4.0)
# all motors off
stop()
print(f"best gain: {best_gain:.1f}")

Run this in the simulator

Each trial is a measurement, and the loop is the engineer. The same shape tunes anything with a number in it: a speed, a threshold, a timeout.

Task: tune yourself

Drive up and down the lane, trying gains and keeping the best. After 30 seconds the robot must stay inside the lane. Print best gain: <value> at the end.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()
gain = 0.3
# do this 10 times (leg counts from 0)
for leg in range(10):
    # do this 50 times (tick counts from 0)
    for tick in range(50):
        # where am I? (cm from where I started)
        x, y = position()
        # forward, sideways, rotation: -100 to 100 each, until the next command
        drive(40 if leg % 2 == 0 else -40, -x * gain, 0)
        # pause 0.1 s (the robot keeps doing what it was told)
        wait(0.1)
# all motors off
stop()
print(f'best gain: {gain:.1f}')

Challenges

  1. Halve the step each time the direction flips, so the search homes in.
  2. Score a trial by its worst error instead of its mean. Does it choose a different gain?
  3. Tune the forward speed as well, keeping the gain: what is the fastest speed that still holds the lane?