What the camera sees

set_cv, the tag list, pixels and distance, empty lists.

4.1VisionRobot club15 min

Do this lesson in the simulator

BugBot has a camera on its nose and a processor behind it that does the hard part of vision for you. You never get a picture in your program. You get a list of what was found: tags with numbers on them, blobs of a colour, faces. This module is about using those lists.

One detector at a time

The camera runs one detector at a time. Until you pick one, the vision functions return nothing:

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

print("before:", apriltags())
# camera: the tag detector
set_cv("apriltag")
print("after:", apriltags())

Run this in the simulator

set_cv("apriltag") switches on the tag detector. An AprilTag is a black and white square with a pattern that encodes a number, like a simplified QR code. They are cheap to print and robots love them, because a tag says exactly what it is and how far away it is.

The list

apriltags() gives a list with one entry per tag it can read, nearest first. Each entry is itself a list of four numbers:

[id, cx, cy, distance]
  • id: the number printed in the tag.
  • cx, cy: where the tag is in the picture, in pixels. The picture is 320 wide and 240 high, so cx = 160 is dead centre.
  • distance: how far away it is, in cm. The detector knows how big a tag really is, so it can work that out from how big it looks.
# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

# camera: the tag detector
set_cv("apriltag")
# the nearest one
tag = apriltags()[0]
print("id:", tag[0])
print("cx:", tag[1], "cy:", tag[2])
print("distance:", tag[3], "cm")

Run this in the simulator

Marker 7 is straight ahead, so its cx is 160, and it is 45 cm away.

Watch the distance fall

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

# camera: the tag detector
set_cv("apriltag")
# do this 4 times (step counts from 0)
for step in range(4):
    tag = apriltags()[0]
    print(f"marker {tag[0]} at {tag[3]} cm")
    # drive forward at 50 for 8 cm, then stop
    forward(50, distance=8)

Run this in the simulator

The distance drops by about 8 each time. The camera and the distance sensor agree with each other: try printing distance() on the same line.

When there is nothing to see

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

# camera: the tag detector
set_cv("apriltag")
# turn 90 degrees clockwise, then stop
turn_right(30, angle=90)
print("looking right:", apriltags())
# turn 90 degrees anticlockwise, then stop
turn_left(30, angle=90)
print("looking ahead:", apriltags())

Run this in the simulator

An empty list means "no tag in view". Your programs must expect that: apriltags()[0] on an empty list is an error. Lesson 4.5 is about searching.

Task: find the marker

Switch on the tag detector and print marker 7 at 45.0 cm using the id and distance from the list. Do not drive.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

print(apriltags())

Challenges

  1. Print cx and cy as well, in the form marker 7 at 45.0 cm, pixel (160, 120).
  2. Drive forward 5 cm at a time and print the distance until it is under 20.
  3. What happens to the list when the robot is 2 cm from the tag? Why?