AprilTags explained
What is printed on an AprilTag, how a camera turns one into a range and a bearing, and how a tag in a known place corrects a robot that has drifted. Three demos read tags on a 1 metre mat, and you can change them and press Run.
An AprilTag is a small black and white square, printed on paper and stuck to a wall, a box or a robot. A camera that sees one gets back a number, and the four corners of the square. From those, a program works out how far away the tag is and which way it lies, and because somebody wrote down where that tag was stuck, the robot then knows where it is. Tags like this are called fiducial markers: things put in a scene on purpose, to be measured. On this page a small robot reads three tags on a 1 metre mat, and each demo below is a real program you can change and run.
The mat is the same in every demo. The robot starts in the bottom left corner at (20, 15) in centimetres, facing up the mat. Three tags, 4 cm across, sit on cubes at (20, 82), (82, 80) and (80, 20), with the numbers 1, 2 and 3 printed on them. The camera looks forward, sees 120 degrees across, and reads the tags with set_cv("apriltag").
What is actually on the tag
The black border is there to be found. A detector looks for quadrilaterals of dark pixels in the picture, which is a much more reliable thing to look for than a colour, and the white margin round the outside makes the border stand out from whatever it is stuck to.
Inside the border is a grid of black and white cells, and each cell is one bit. The family used most often, tag36h11, has a 6 by 6 grid, so a tag carries 36 bits. Those 36 bits are not just a number written out in binary. They are a code word chosen so that any two tags in the family differ in at least 11 of their 36 bits, which is what the 11 in the name means. Turn a tag upside down, and no rotation of one valid code word is another valid code word.
That is the whole trick, and it is the reason to prefer a tag to a coloured blob:
- A few bits read wrongly, because of blur or a poor angle, still leave the answer nearer its own code word than any other, so the detector corrects them.
- More bits wrong than it can correct, and the pattern matches no code word at all, so the tag is not reported. It does not turn into a different tag.
A colour test fails by being quietly wrong. A tag detector fails by saying nothing, and a robot can be written to cope with nothing.
AprilTags came out of the APRIL laboratory at the University of Michigan in 2011, and the idea is older than that: ARToolKit markers and the QR-code-like ARTag came first, and OpenCV ships a very similar family called ArUco. They are all the same idea with different code words.
What the detector gives you
apriltags() reports one row per tag it can read, nearest first:
[id, cx, cy, range] id 7 at column 160, row 120, 45.0 cm away
cx is the column of the middle of the tag in the camera's picture, which is 320 pixels wide, so 160 is straight ahead. range is how far away it is in centimetres. marker_tags() is the same list with the tags worn by other robots left out.
A column becomes an angle through the camera's focal length, which for this camera is 92.4 pixels:
bearing = atan((cx - 160) / 92.4)
The arctangent matters. A pixel near the edge of the picture covers a smaller angle than one in the middle, because the picture is flat and angles are not, so (cx - 160) × some constant is only right near the centre.
In this demo the robot stands still and turns on the spot, reading every tag as it comes into view. Because it knows where it is standing, a range and a bearing put each tag on the mat.
The program
from bugbot import *
import math
connect()
F = 92.4 # the camera's focal length, in pixels
HERE = (20, 15) # where the robot is standing, in cm on the mat
set_cv("apriltag")
found = {}
for step in range(24):
for tag_id, cx, cy, dist in marker_tags():
bearing = math.degrees(math.atan((cx - 160) / F))
way = math.radians(heading() + bearing)
found[tag_id] = (HERE[0] + dist * math.sin(way),
HERE[1] + dist * math.cos(way))
print("tag %d: %5.1f cm away, %6.1f degrees off the nose, so it is at (%.0f, %.0f)"
% (tag_id, dist, bearing, found[tag_id][0], found[tag_id][1]))
plot("tag %d bearing" % tag_id, bearing)
draw("where the tags are", list(found.values()), "yellow", "squares", 6)
turn_right(40, angle=15)
print("tags found:", sorted(found))
Each tag's line on the chart slopes down by 15 degrees a step, because that is how far the robot turns between reads, and stops when the tag leaves the 120 degree view at about -60. Tag 1 starts again near +57 when the robot comes all the way round to it. The yellow squares on the mat are where the camera says the tags are: (20, 82), (82, 80) and (80, 20), which is exactly where they were put.
Read that the other way round and it is the whole point of fiducials. Standing still, a robot that knows where it is can work out where things are. Knowing where the tags are, a robot that is lost can work out where it is.
Where the range comes from
Nothing in the picture measures distance. What the camera measures is how big the tag looks, and a real detector turns that into a range using the pinhole rule:
width in pixels = real width × focal length / range
Turn it round and range = real width × focal length / width in pixels. A 4 cm tag through a 92.4 pixel lens gives 4 × 92.4 = 370, so range times apparent width is always 370 for these tags. At 67 cm the tag is 5.5 pixels across. At 16.5 cm it is 22.4.
The simulator's detector hands you the range directly, so this demo runs the sum the other way: it drives towards tag 1 and works out, from the range, how wide the tag must be looking.
The program
from bugbot import *
connect()
F = 92.4 # focal length, pixels
SIZE = 4 # the tags on this mat are 4 cm across
set_cv("apriltag")
print("standing still:", marker_tags())
backward(40, distance=6)
print("6 cm further back:", marker_tags())
forward(40, distance=6)
for step in range(11):
for tag_id, cx, cy, dist in marker_tags():
if tag_id == 1:
width = SIZE * F / dist
print("range %5.1f cm the tag looks %5.1f pixels wide range x width = %3.0f"
% (dist, width, dist * width))
plot("range, cm", dist)
plot("how wide it looks, pixels", width)
forward(60, distance=5)
The two lines on the chart cross: range falls from 67 to 16.5 while the apparent width climbs from 5.5 to 22.4. It is not a straight line. Doubling the distance halves the width, so the far half of the graph is nearly flat, and that is why a range worked out from apparent size is much less accurate far away than close up. One pixel of error at 5 pixels wide is a fifth of the range. One pixel at 22 is a twentieth.
The same rule sets how far away a tag can be read at all. A detector needs the cells inside the border to be a few pixels each, and this simulator's rule is that a tag under 4 pixels wide is not decoded. For a 4 cm tag that is 92 cm. Look at the top of the output: standing at the start, the robot reads tag 1 at 67 cm and tag 2 at 89.8 cm. Six centimetres further back, tag 2 is 95.8 cm away and is not in the list at all. It was not read wrongly. It was not read.
This is the number to plan a room around. Double the tag size and you double the range it can be read from. On a real detector the same rule is why tags are printed large on warehouse walls and small on the objects a robot picks up.
What a tag does for a lost robot
A robot that adds up its own movement drifts. The further it goes, the further out its guess is, and nothing in the adding up will ever tell it so. That is dead reckoning, and the only cure is something outside the robot whose position is known.
A tag is exactly that. The robot reads a range and a bearing to a tag whose place is written down, so:
x = tag x - range × sin(heading + bearing)
y = tag y - range × cos(heading + bearing)
The heading in that sum is the robot's own, from its gyro, so a fix from one tag is only as good as the heading. A bearing that is 5 degrees out swings the answer sideways by 5 degrees of the range: about 2 cm at 25 cm, and 6 cm at 70. That is why this demo only trusts a tag that is closer than NEAR.
The robot drives a lap of the mat on odometry(), its own guess, with the sensors turned up to a bad day. The chart is how far that guess is from the truth, which the simulator knows and the robot does not.
The program
from bugbot import *
import math
connect()
# change these two and press Run
FIX = True # take a fix when a tag is in view
NEAR = 50 # only trust a tag closer than this, in cm
F = 92.4
START = (20, 15) # where the robot starts, in cm on the mat
TAGS = {1: (20, 82), 2: (82, 80), 3: (80, 20)}
set_cv("apriltag")
set_noise(3) # a bad day for the sensors
reset_odometry(START[0], START[1], 0)
def truly_at():
# the lab's overhead camera: for the chart only
px, py = position()
return (START[0] + px, START[1] + py)
def how_far_out():
x, y, h = odometry()
out = math.dist((x, y), truly_at())
plot("how far out the guess is, cm", out)
return out
def take_fix():
seen = marker_tags()
if not seen or seen[0][3] > NEAR or seen[0][0] not in TAGS:
return None
tag_id, cx, cy, dist = seen[0]
x, y, h = odometry()
way = math.radians(h + math.degrees(math.atan((cx - 160) / F)))
reset_odometry(TAGS[tag_id][0] - dist * math.sin(way),
TAGS[tag_id][1] - dist * math.cos(way), h)
return tag_id, dist
for leg in range(4):
for i in range(3):
forward(80, distance=20)
out = how_far_out()
got = take_fix() if FIX else None
if got:
print("out by %4.1f cm, fix from tag %d at %2.0f cm, now out by %4.1f cm"
% (out, got[0], got[1], how_far_out()))
else:
print("out by %4.1f cm, no tag near enough" % out)
turn_right(50, angle=90)
how_far_out()
x, y, h = odometry()
print("it thinks it is at (%.0f, %.0f) facing %.0f; it is at (%.0f, %.0f)"
% (x, y, h, truly_at()[0], truly_at()[1]))
print("out by %.1f cm after %d s" % (math.dist((x, y), truly_at()), clock()))
Watch the chart go up and down. On the first three legs there is always a tag ahead, the fixes come every 20 cm, and the error never gets past 2.8 cm. The last leg runs along the bottom of the mat with tag 3 behind the robot and tag 1 out to the side, outside the camera's view, so there are no fixes at all and the line climbs steadily to 5.7 cm. Set FIX = False and the whole lap is that climb: 8.2 cm at its worst.
Two things are worth trying.
- Set
NEAR = 100. On the last leg the robot now takes a fix from tag 1, which is 76 cm away, and its heading is 7 degrees out by then. The fix moves the guess from 2.4 cm out to 6.0 cm out. A far tag with a doubtful heading is worse than no fix. - Note that the fix never corrects the heading. It cannot: one tag gives one range and one bearing, which is two numbers, and the robot needs three. Two tags at once give enough, and then a fix is a proper position and heading. The dead reckoning guide does it that way, with the two circles crossing.
Tags in the real world
- A robot vacuum finds its dock by the pattern on the front of it, which is a fiducial in all but name.
- Warehouse robots drive over floor markers laid on a grid, and read one every few metres to stay square with the aisles.
- Drones land on a pad with a large tag in the middle, and a smaller tag inside it for the last few centimetres, because the big one goes out of the frame as the drone comes down.
- Film and television put markers on a set so the camera's own path can be worked out afterwards.
- Robot competitions put tags on the field and on the robots, which is what
robot_tags()is for here.
A real detector does more than this simulator's. From the four corners of a tag whose real size you know, it works out the full pose: three numbers for where the tag is and three for how it is turned, all in one go. That is enough to drive a robot arm to a box, or to tell that a tag on a wall is being seen from 40 degrees off to the left.
The two things to get right when you use them are size and light. Print the tag big enough for the range you need, keep the white margin round it, mount it flat, and light it evenly. A tag seen almost edge-on, or one with a reflection across it, is a tag that does not get read.
Where this is taught
- What the camera sees reads the first tag, and Where is it turns what the camera reports into a distance and a direction.
- Searching spins until a tag comes into view, and Project: the marker trail drives the mat used on this page.
- The pinhole camera is where the arctangent comes from, Intrinsics measures the focal length, and Range from size is the rule in the second demo.
- A tag as a fix and A fix from a landmark are the fix in the third demo, done properly.
- Fusing a fix blends a fix with the estimate instead of replacing it, which is what a Kalman filter is for.
Questions
What is an AprilTag?
A square black and white marker, printed on paper, that a camera can find and read. Inside its black border is a grid of cells, each one a bit, which spell out a number chosen from a family of code words. A detector finds the square, reads the bits, checks them against the family, and reports the tag's number and where its four corners are in the picture. AprilTags came from the APRIL laboratory at the University of Michigan in 2011.
How does an AprilTag work?
In three steps. The detector finds edges in the picture and joins them into quadrilaterals, which is how it finds candidate tags without knowing anything about colour. It then samples the grid of cells inside each quadrilateral and reads the bits. Finally it compares those bits with every code word in the family: a close match is corrected and reported, and anything that is not close to one code word is thrown away.
What is the difference between an AprilTag and a QR code?
A QR code carries a message, often hundreds of characters, and is meant to be read once by a phone held still. An AprilTag carries only a small number, often fewer than a thousand possibilities, and is meant to be found and measured accurately in every frame of a moving camera, from a long way off. Fewer bits means bigger cells for the same printed size, which means it is readable further away and at worse angles.
How does a robot work out distance from an AprilTag?
From how big the tag looks. The pinhole rule says the apparent width in pixels is the real width times the focal length divided by the range, so the range is the real width times the focal length divided by the apparent width. You need to know the tag's printed size and the camera's focal length in pixels, which comes from calibrating the camera. On this page a 4 cm tag through a 92.4 pixel lens gives a range times width that is always 370.
How accurate is an AprilTag?
The bearing is good, because a corner can be located to a fraction of a pixel. The range is worse, and gets worse with distance, because it comes from an apparent size that shrinks as one over the range: a pixel of error costs you a fifth of the range at 5 pixels wide, and a twentieth at 22. The angle a tag is turned through is the least reliable number of the lot when the tag is seen nearly face on, and it is worth reading only the numbers you actually need.
What is tag36h11?
The AprilTag family used by default: a 6 by 6 grid of bits, so 36 of them, with the code words chosen so that any two differ in at least 11 bits. That gap is what lets a detector correct a few misread bits and reject anything that is not a real tag. Smaller families, with fewer bits, give bigger cells and longer range for the same printed size, but fewer ids and less protection against a false reading.
Why can my robot not see the tag?
Work through the list. It may be too far away: a tag has to be a few pixels per cell, which for a 4 cm tag and this camera means under about 92 cm. It may be outside the camera's view, which is 120 degrees wide here. Something may be in the way. It may be at too sharp an angle, or blurred by the robot moving, or have a reflection across it. Or the white margin round the border may have been cropped off when it was printed.
What is a fiducial marker?
Anything put into a scene on purpose so that a camera has something known to measure. Tags are the usual kind in robotics, but the dots on a motion capture suit, the chequerboard used to calibrate a camera and the pattern on a vacuum's charging dock are all fiducials. The word means a thing taken as a reference.
AprilTag or ArUco?
Both are square marker families and both work. ArUco is built into OpenCV, which makes it the easy choice in Python, and its dictionaries are easy to make smaller for longer range. AprilTag's detector was designed for exactly this job and tends to hold up better at sharp angles and in poor light. Whichever you pick, use one family, keep a written list of which id is stuck where, and print them larger than you think you need.
Learn it step by step
These lessons build the same ideas one at a time, each with tasks the simulator marks.
- 4.1 What the camera sees Vision, Robot club
- 4.2 Where is it? Vision, Robot club
- 4.5 Searching Vision, Robot club
- 4.6 Project: the marker trail Vision, Robot club
- U3.5 A fix from a landmark Odometry and drift, University
- U11.1 The pinhole camera Vision, University
- U11.2 Intrinsics and calibration Vision, University
- U11.3 Range from apparent size Vision, University
- U11.4 A tag as a fix Vision, University
- U6.6 Fusing a fix State estimation, University