Project: find the one that is green
Three identical shapes, one job, and a task the depth sensor cannot begin.
Do this lesson in the simulatorThree balls are on the mat. They are the same size, the same shape and made of the same stuff. To the depth sensor they are three identical bumps at three bearings, and no amount of cleverness with the time-of-flight data will tell you which is which.
The camera can tell you in one frame. That is the whole argument for putting a camera on a robot, and this project is built around it.
The job
- Survey. Find all three balls, and for each one report a range and a bearing. The camera runs one detector at a time, so this is three passes: set the colour, let a frame come through, read the blobs.
- Choose. Go to the green one.
- Approach. Servo onto it as in U11.6 and stop about 20 cm short, without touching it.
Print the three ranges as red:, green: and blue: before you set off.
What is worth thinking about
Identity is not geometry. The depth sensor measures a property of space. The camera measures a property of surfaces, and identity lives in the second. Every task of the form "fetch the X", "avoid the Y", "count the Z" needs the second kind of sensor, and no improvement in the first kind substitutes for it.
Switching detectors costs time. One detector at a time is not an artificial restriction: it is how a small camera pipeline really works, because the processor has one frame buffer and one configuration. The consequence is that you cannot watch for red while servoing on green. Plan the survey as a separate phase and commit.
Your range is worse than your bearing. By a lot, and by a known amount, from U11.3. So do not plan a path to computed coordinates and drive it open loop. Turn to the bearing, which you trust, and close the loop on the apparent size, which is exactly what U11.6 does. The approach corrects itself as you get closer and the measurement gets better.
Do not touch it. At 20 cm the ball is about 28 pixels wide. Overshoot happens when the forward gain is high and the loop is slow, so use the dead band trick rather than a large gain, and stop on a band rather than a point.
A shape to start from
from bugbot import *
import math
connect()
F, R = 92.4, 3.0
def survey(colour):
set_cv("blob", colour)
wait(0.3)
seen = blobs()
if not seen:
return None
cx, cy, area, x0, y0, x1, y1, aspect = seen[0]
return 2 * R * F / (x1 - x0), math.degrees(math.atan((cx - 160) / F))
for colour in ("red", "green", "blue"):
got = survey(colour)
print(colour, "not in view" if got is None else
"%.0f cm at %.0f degrees" % got)
Task: find the one that is green
Survey all three balls and print red:, green: and blue:, the range in centimetres to each. Then drive to the green ball and stop within reach of it without touching it. position() is not allowed: the robot has to find it by looking.
from bugbot import *
import math
connect()
F, R = 92.4, 3.0
TARGET = "green"
Challenges
- Run the survey again from where you finished. Which ranges improved, and by how much compared with the
d^2prediction? - Make the choice of colour come from a tag: read a marker first and let its id decide which ball to fetch. What has to change in the program, and what does not?
- Replace the colour test with size: fetch the largest ball rather than the green one. Which measurement is doing the work now, and which of the two would you rather rely on in a room with a window?
What comes next
Everything in this module was a model written down by hand. The pinhole equations, the size of the ball, the threshold that separates red from white: a person chose each one, and each one is a place where the world can disagree with the description.
U12 is the other approach. Instead of writing the model, collect examples and fit it, which is what makes a vision system cope with objects nobody measured and lighting nobody anticipated. It is also where the capstone project lives: a robot that is given a job rather than a program, and has to work out the rest for itself.