A14.5 Testing strategies worksheet

Black box and white box testing, alpha, beta and acceptance testing, destructive testing, and a black box test plan that finds two bugs.

Questions 6 marks in all

[1 mark]What is black box testing?

ADesigning tests from the specification, without looking at the code
BTesting every path through the code
CTesting by the client before they accept the system
DTesting that deliberately tries to break the hardware

[1 mark]A game is released to a limited group of players outside the company to find problems before the full launch. What is this?

ABeta testing
BAlpha testing
CAcceptance testing
DWhite box testing

[1 mark]A speed is allowed from 10 up to but not including 40. Which set is boundary data?

A9, 10, 39 and 40
B25 and 30
C"far" and -5
D0 and 100

[1 mark]What is the purpose of acceptance testing?

AFor the client to check the system meets the agreed requirements before accepting it
BFor programmers to test each module as it is written
CFor testers to try to crash the system
DTo check the code compiles

[1 mark]What does this white box coverage check print?

ran = set()
def speed_for(gap):
    if gap < 10:
        ran.add("stop")
        return 0
    elif gap < 40:
        ran.add("slow")
        return 30
    ran.add("fast")
    return 60
for gap in [5, 20, 30]:
    speed_for(gap)
print(len(ran), sorted(ran))

[1 mark]Why is testing described as showing the presence of errors but not their absence?

ANo set of tests can try every possible input, so an untested input could still fail
BTests always contain errors themselves
CErrors cannot be found by testing
DOnly syntax errors can be found by testing

The task: test the black box

A teammate has written speed_for(gap). Its specification is: - gap is a distance in cm, and should be an int or a float. - If gap is not a number, or is less than 0, it must raise ValueError. - Otherwise it returns 0 if gap is less than 10, 30 if gap is from 10 up to but not including 40, and 60 if gap is 40 or more. Do not change speed_for: your job is to test it from the specification. Make a list tests of tuples (value, expected, kind), where expected is the number the specification says, or the string "error" if it should raise ValueError, and kind is "normal", "boundary" or "erroneous". Include at least two normal tests, the four boundary values 9, 10, 39 and 40, and the erroneous values "far" and -5. Run each test, treating a ValueError as the result "error". For each, print <value!r> <kind>: pass if the result matches, or <value!r> <kind>: FAIL (got <result>) if it does not, where <value!r> is the value as repr shows it (so the string appears as 'far'). At the end print failed: <n>. The robot stays still.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

def speed_for(gap):
    if not isinstance(gap, (int, float)):
        raise ValueError("gap must be a number")
    if gap < 10:
        return 0
    elif gap <= 40:
        return 30
    else:
        return 60

tests = []

Plan your program here, then type it in and press Run.

Do it on the robot
www.bugbotlab.com/learn/a14-5-testing-strategies/
The simulator checks it and tells you when it passes. Nothing to install, no account.

Challenges

Your tests found two bugs. Fix speed_for, run the tests again, and check they all pass. What is this kind of re-testing called?

Write the extra white box tests needed to run every branch of the fixed function, including the one that raises the error.

Plan three destructive tests for the delivery robot, and say what a safe failure looks like for each.

What this lesson is about

Questions 6 marks in all

The task: test the black box

Challenges