HSV colour detection explained
Why a red > 150 test finds the whole room, what hue, saturation and value each measure, and how a ratio survives a change of light. Four demos run a real camera on a mat with a red ball and a blue one, and you can change the thresholds and press Run.
A robot that has to pick up the red ball first has to find it. The camera hands the program a grid of pixels, each one three numbers, and somewhere in that grid is the ball. Deciding which pixels belong to it is colour detection, and the first test everybody writes, red > 150, finds the floor, the walls and the ceiling as well. On this page a small robot looks at a mat with a red ball and a blue ball on it, and each demo below is a real program you can change and run.
The scene is the same in every demo. The robot stands still at the bottom of a 1 metre mat. A red ball 6 cm across sits 22 cm ahead and to the left, a blue ball the same size 30 cm ahead and to the right, and a low wall runs across the back of the mat. The camera sees the mat, the wall, the edge of the mat and the pale background above it.
The chart under each demo counts pixels. The picture is 64 wide and 48 high, so there are 3,072 of them in total, and a good test finds the 25 that are ball.
A pixel is three numbers
camera_image(64, 48) gives back 48 rows of 64 pixels, top row first, and each pixel is (red, green, blue), three numbers from 0 to 255. That is the whole of what the program gets. There is no "ball" in it and no "mat", only numbers.
In this simulator every surface is a flat colour, so the picture holds exactly six different pixels:
(248, 246, 238) 1472 pixels the background above the mat
(230, 238, 226) 1421 pixels the mat
(176, 164, 140) 97 pixels the edge of the mat
( 91, 107, 115) 44 pixels the wall at the back
(220, 50, 40) 25 pixels the red ball
( 50, 90, 210) 13 pixels the blue ball
A real camera gives hundreds of slightly different pixels for each surface, because of noise, shading and the lens. This one is the kindest possible case. If a test cannot separate the ball here, it has no chance in a classroom.
Why "red is high" is not a red detector
The red ball is (220, 50, 40). So red > 150 sounds right. The trouble is that pale things have a lot of red in them too: the mat is (230, 238, 226) and the background is (248, 246, 238). Both are whiter than the ball, and white contains all the red there is.
This demo sweeps the level from 0 to 250 and counts how many pixels pass at each one.
The program
from bugbot import *
connect()
picture = camera_image(64, 48) # 48 rows of 64 pixels
print("pixels in the picture:", 64 * 48)
for level in range(0, 251, 10):
passed = 0
for row in picture:
for (r, g, b) in row:
if r > level:
passed += 1
print("red >", level, "passes", passed, "pixels")
plot("pixels passing red > level", passed)
wait(0.1)
Read the chart as a staircase. Every step is one surface leaving the test. The blue ball goes at 50, the wall at 91, the edge of the mat at 176, the ball at 220, the mat at 230 and the background at 248. The ball leaves before the mat does, so there is no level anywhere that keeps the ball and throws the mat away. At the usual red > 150 the test passes 3,015 pixels out of 3,072, which is 98 percent of the picture.
The test is not measuring colour at all. It is measuring brightness, with a slight lean towards red.
Hue, saturation and value
The fix is to stop storing a colour as three amounts of light and start storing it as three things a person would say about it. That is HSV, and it is the same colour written a different way.
- Hue is which colour it is, as an angle round a wheel: 0 is red, 120 is green, 240 is blue, and 360 is back to red.
- Saturation is how strong the colour is, from 0 (grey, white or black) to 1 (as pure as the screen can make it).
- Value is how bright it is, from 0 (black) to 1 (as bright as the camera can read).
The arithmetic is short. Take the largest of the three channels and the smallest:
value = largest / 255
saturation = (largest - smallest) / largest
hue = which channel is largest, plus how far the other two lean
Value is the size of the numbers. Saturation and hue are ratios between them, which is the whole point: turn the light down and all three channels shrink together, so the ratios stay where they were and only the value moves.
Here are the six colours of the picture, converted.
The program
from bugbot import *
connect()
def hsv(pixel):
r, g, b = pixel
high = max(r, g, b)
low = min(r, g, b)
if high == 0:
return 0.0, 0.0, 0.0 # black: no hue, no saturation
saturation = (high - low) / high
value = high / 255
if high == low:
hue = 0.0 # grey: the hue means nothing
elif high == r:
hue = 60 * (g - b) / (high - low) % 360
elif high == g:
hue = 120 + 60 * (b - r) / (high - low)
else:
hue = 240 + 60 * (r - g) / (high - low)
return hue, saturation, value
picture = camera_image(64, 48)
counts = {}
for row in picture:
for pixel in row:
counts[pixel] = counts.get(pixel, 0) + 1
for pixel, n in sorted(counts.items(), key=lambda item: -item[1]):
hue, saturation, value = hsv(pixel)
print("%-16s %5d pixels hue %3.0f saturation %.2f value %.2f"
% (str(pixel), n, hue, saturation, value))
plot("saturation", saturation)
plot("value", value)
wait(0.2)
The chart has one point per colour, in order of how much of the picture it covers. Look at what separates the balls from everything else. The mat, the background, the edge and the wall all have a saturation of 0.21 or less: they are pale or grey, whatever their brightness. The red ball is 0.82 and the blue ball is 0.76. Value does not separate them at all: the ball's value is 0.86 and the mat's is 0.93, which is the same fact as the failed brightness test, in different clothes.
Tightening the test until only the ball passes
A colour test in HSV is a window: a range of hue, and a floor under saturation. The hue window says which colour, the saturation floor says "and actually coloured, not just a pale thing leaning that way".
This demo widens the hue window from 0 to 90 degrees either side of red, twice: once on hue alone, and once with saturation > 0.4 as well.
The program
from bugbot import *
connect()
def hsv(pixel):
r, g, b = pixel
high = max(r, g, b)
low = min(r, g, b)
if high == 0:
return 0.0, 0.0, 0.0
saturation = (high - low) / high
value = high / 255
if high == low:
hue = 0.0
elif high == r:
hue = 60 * (g - b) / (high - low) % 360
elif high == g:
hue = 120 + 60 * (b - r) / (high - low)
else:
hue = 240 + 60 * (r - g) / (high - low)
return hue, saturation, value
picture = camera_image(64, 48)
colours = [hsv(pixel) for row in picture for pixel in row]
# change this floor and press Run
FLOOR = 0.4
for window in range(0, 91, 5):
hue_only = 0
with_saturation = 0
for (hue, saturation, value) in colours:
near_red = hue < window or hue > 360 - window
if near_red:
hue_only += 1
if saturation > FLOOR:
with_saturation += 1
print("window +/-%2d degrees: hue alone %4d, hue and saturation %4d"
% (window, hue_only, with_saturation))
plot("hue alone", hue_only)
plot("hue and saturation", with_saturation)
wait(0.1)
The two lines tell the story. Hue alone holds at 25 while the window is narrow, jumps to 122 at 45 degrees when the tan edge of the mat comes in at hue 40, and to 1,594 at 50 degrees when the background arrives at hue 48. With the saturation floor it is 25 at every window from 5 degrees to 90: the pale surfaces are thrown out before their hue is ever looked at.
That is the right way round. A narrow hue window is brittle, because the colour of a real object moves when the light changes. A saturation floor is not, because pale things stay pale. Set FLOOR = 0.05 and watch the second line climb to meet the first.
Only the red ball passes now. Change hue < window or hue > 360 - window to a window round 225 and the 13 blue pixels pass instead.
When the light changes
The simulated camera has no lighting at all. Nothing casts a shadow and nothing is brighter on one side. So to see what a shadow does, this demo dims the picture itself: take the same picture, multiply every channel by a fraction, and run three tests on it at each level of light.
The program
from bugbot import *
connect()
def hsv(pixel):
r, g, b = pixel
high = max(r, g, b)
low = min(r, g, b)
if high == 0:
return 0.0, 0.0, 0.0
saturation = (high - low) / high
value = high / 255
if high == low:
hue = 0.0
elif high == r:
hue = 60 * (g - b) / (high - low) % 360
elif high == g:
hue = 120 + 60 * (b - r) / (high - low)
else:
hue = 240 + 60 * (r - g) / (high - low)
return hue, saturation, value
picture = camera_image(64, 48)
for step in range(20):
light = 1.0 - step * 0.05 # 100 percent down to 5
bright = 0
beats = 0
colour = 0
for row in picture:
for (r, g, b) in row:
r, g, b = int(r * light), int(g * light), int(b * light)
if r > 150:
bright += 1
if r - max(g, b) > 60:
beats += 1
hue, saturation, value = hsv((r, g, b))
if (hue < 20 or hue > 340) and saturation > 0.4:
colour += 1
ball = (int(220 * light), int(50 * light), int(40 * light))
print("light %3d%% ball %-16s bright %4d beats %3d colour %3d"
% (round(light * 100), str(ball), bright, beats, colour))
plot("r > 150", bright)
plot("r beats g and b by 60", beats)
plot("hue and saturation", colour)
wait(0.1)
Three lines, three fates. r > 150 starts at 3,015 and falls off a cliff: it loses the edge of the mat at 85 percent light, the mat and the ball together at 65, and the background at 60, after which it finds nothing at all. r - max(g, b) > 60 is much better, and finds exactly the 25 ball pixels, but it is still a difference between raw numbers, so it gives the ball up below 39 percent light. The hue and saturation test reports 25 at every level down to 5 percent, where the ball reads (10, 2, 1).
That is what dividing does. Dimming multiplies all three channels by the same fraction, and a ratio does not notice a common factor. Hue and saturation are ratios. Value is not, and that is exactly why the colour is kept apart from the light in the first place.
From a mask to a direction
A test that says yes or no for each pixel gives a mask: the set of pixels that passed. On its own that is not much use to a robot. What a robot wants is a direction to turn.
The average column of the mask is the middle of the object in the picture. A column converts to an angle through the camera's focal length, which for this camera is 92.4 pixels in its own 320 wide frame. The picture here is 64 wide, so each of its columns covers five of the camera's, and column x of the small picture sits at x * 5 + 2.5 in the big one.
The program
from bugbot import *
import math
connect()
F = 92.4 # the camera's focal length, in pixels
HERE = (50, 20) # where the robot is standing, in cm on the mat
def hsv(pixel):
r, g, b = pixel
high = max(r, g, b)
low = min(r, g, b)
if high == 0:
return 0.0, 0.0, 0.0
saturation = (high - low) / high
value = high / 255
if high == low:
hue = 0.0
elif high == r:
hue = 60 * (g - b) / (high - low) % 360
elif high == g:
hue = 120 + 60 * (b - r) / (high - low)
else:
hue = 240 + 60 * (r - g) / (high - low)
return hue, saturation, value
def look():
# the bearing of the red pixels, in degrees, positive to the right
picture = camera_image(64, 48)
columns = []
for row in picture:
for x, pixel in enumerate(row):
hue, saturation, value = hsv(pixel)
if (hue < 20 or hue > 340) and saturation > 0.4:
columns.append(x * 5 + 2.5) # in the camera's own 320 wide frame
if not columns:
return None, 0
middle = sum(columns) / len(columns)
return math.degrees(math.atan((middle - 160) / F)), len(columns)
for step in range(8):
bearing, found = look()
if bearing is None:
print("no red in view")
break
print("%d pixels of red, bearing %.1f degrees" % (found, bearing))
plot("bearing to the red ball, degrees", bearing)
# draw the direction the camera is pointing the robot in
way = math.radians(heading() + bearing)
draw("what the mask says", [HERE, (HERE[0] + 30 * math.sin(way), HERE[1] + 30 * math.cos(way))], "red", "line", 2)
turn_right(30, angle=bearing * 0.6)
wait(0.2)
stop()
The red line on the mat is where the camera says the ball is, drawn from where the robot is standing. It lands on the ball at every step, because a bearing plus the robot's own position is a direction in the room. The count of red pixels moves between 25 and 20 as the robot turns, because the ball lands on the grid of pixels differently each time, and that is normal.
The chart is a bearing being driven to zero, which is a proportional controller with a camera for a sensor. It stops at 0.8 degrees rather than 0, because a turn of less than half a degree is smaller than the smallest turn the robot will make. Turning until a target is in the middle of the picture, then driving at it, is called visual servoing.
What still goes wrong in a real room
HSV removes the largest single failure, which is the light level. It does not make colour detection reliable. The things that break it next, roughly in order:
- Automatic exposure. The camera changes its own gain to suit the scene. Point it at something dark and everything else brightens. Your value threshold no longer means what it meant. Lock the exposure if the driver lets you.
- Automatic white balance. The camera decides what counts as white and shifts the channels to suit, which moves hue, the one number you were relying on. Lock this too.
- Hue is meaningless when saturation is low. For a grey pixel,
high - lowis nearly zero, and the hue you get out is decided by one count of noise. This is why the floor under saturation is not optional. - Red wraps around. Red sits at 0, so a red test is two tests,
hue < 20 or hue > 340. Forgetting the second half finds half a ball. - Highlights clip. A shiny spot adds white light to all three channels, which pulls the saturation towards 0, and once a channel hits 255 it is cut off and what it was cannot be worked out again.
- The object is not one colour. The lit side and the shaded side of a ball have different values and slightly different hues, and a deep shadow has so few counts left that a single count of noise moves the hue a long way.
For anything that has to work every time, use something with structure instead of colour. A printed AprilTag is read from its pattern, so it is either decoded correctly or not reported at all, where a colour test is quietly wrong.
Where this is taught
- What the camera sees is the first look at the camera, and Colour finds coloured balls with the built-in detector.
- Images works with the raw picture as numbers, which is where
camera_image()comes in, on the same mat as the demos above. - Colour, and why it breaks is the full University-level lesson: what a pixel actually measures, chromaticity, and the four ways a colour threshold fails outside the lab.
- The pinhole camera turns a column into a bearing, and Visual servoing drives a robot on what the camera reports.
- Project: find the one that is green puts the whole thing together on a mat with three balls.
Questions
What is HSV colour detection?
It is finding an object by converting each pixel from red, green and blue into hue, saturation and value, and then testing hue and saturation instead of the raw channels. Hue says which colour the pixel is, saturation says how strongly coloured it is, and value says how bright it is. Because the light level mostly moves value and leaves the other two alone, a test written in hue and saturation survives a change of lighting that a test in RGB does not.
Why is HSV better than RGB for detecting colour?
In RGB, changing the light changes all three numbers at once, so every threshold you wrote has moved. HSV splits that change off into one number, value, and leaves hue and saturation nearly where they were. On this page, dimming the picture to 39 percent of full light breaks an RGB test that had been finding the ball perfectly, while the HSV test still finds every pixel of it at 5 percent.
How do you convert RGB to HSV?
Take the largest channel and the smallest. Value is the largest divided by 255. Saturation is (largest - smallest) / largest, or 0 when the largest is 0. Hue depends on which channel is largest: if it is red, 60 × (g - b) / (largest - smallest), taken modulo 360; if green, 120 + 60 × (b - r) / (largest - smallest); if blue, 240 + 60 × (r - g) / (largest - smallest). The second demo on this page is that code, eight lines of it.
What are good HSV values for detecting red?
Start with hue within about 20 degrees of 0, remembering that red wraps, so the test is hue < 20 or hue > 340. Put the saturation floor at about 0.4 and a low floor under value, around 0.15, to throw out near-black pixels whose hue is noise. Then tune them by looking at the actual numbers from your own camera, in the room the robot will work in, rather than copying numbers from a page. Show the robot the object and print what it reads.
Why does my colour detection find the wall as well as the ball?
Almost always because the test is a brightness test in disguise. r > 150 passes anything pale, since white has as much red in it as red does. On this page it passes 98 percent of the picture. Ask instead whether the pixel is strongly coloured (saturation) and which colour it is (hue). If the wall is close to the same colour, raise the saturation floor, because a painted wall is usually much paler than a coloured ball.
What is the difference between hue and saturation?
Hue is which colour, as an angle: 0 red, 120 green, 240 blue. Saturation is how much of that colour there is against grey. Pink and deep red have nearly the same hue and very different saturations. A white wall has a hue too, but it means nothing, because there is almost no difference between its three channels for the hue to be worked out from.
Is HSV the same as HSL or Lab?
They are all ways of splitting a colour into "which colour" and "how bright". HSL uses lightness instead of value, so its middle is the pure colour rather than the brightest one. Lab is built from measurements of human vision, so equal steps in it look like equal steps to a person, and it separates light from colour better than HSV does. HSV is the usual choice on a small robot because it costs a handful of integer operations per pixel.
How does a robot turn the pixels it found into a direction?
Average the columns of the pixels that passed, which gives the middle of the object in the picture. Subtract the middle column of the frame and divide by the camera's focal length in pixels, then take the arctangent: that is the bearing in degrees. The last demo on this page does it and turns the robot until the bearing is zero. To get a range as well you need something more, such as the object's real size, or a depth sensor.
Should I use colour detection at all?
For a game with coloured balls, yes, and it is fast and easy to explain. For anything that has to work in a strange room under unknown light, prefer something with structure: a tag, a line, a known shape. Colour fails quietly and gradually, which is the worst way for a robot to fail. A tag detector either reads the tag or says nothing.
Learn it step by step
These lessons build the same ideas one at a time, each with tasks the simulator marks.
- 4.1 What the camera sees Vision, Robot club
- 4.4 Colour Vision, Robot club
- F8.7 Images Data representation, GCSE
- U11.1 The pinhole camera Vision, University
- U11.5 Colour, and why it breaks Vision, University
- U11.6 Visual servoing Vision, University
- U11.7 Project: find the one that is green Vision, University