Testing and test data

Iterative and final testing; normal, boundary, invalid and erroneous data; test plans in code.

F6.3Robust programsGCSE15 min

Covers OCR J277 2.3.2 Testing · AQA 8525 3.2.11 Robust and secure programming · Edexcel 1CP2 6.1.6 Test data and evaluation

Print it: worksheet (PDF) · answers (PDF) · as a web page

A program that works for the one example you tried is not a tested program. Testing is running a program on purpose with inputs chosen to find its mistakes, and checking each result against what it should be. Good testers do not try to show a program works; they try hard to make it fail, so that users never do.

When to test

• Iterative testing happens while you build: write a small piece, test it, fix it, then add the next piece. Every project in these lessons has been built this way.
• Final testing (also called terminal testing) happens when the program is finished: the whole thing is tested against its brief, to check every requirement is met.

Iterative testing finds mistakes while they are small and easy to fix. Final testing checks the pieces work together.

Choosing test data

A speed check should accept whole numbers from 0 to 100. Which values should you test it with?

Boundary data finds the most bugs, because the edges are where < and <= get mixed up.

A test plan

A test plan lists each test before it is run: what is tested, the data, and the expected result. Then the actual result is filled in:

The expected result is worked out before running the test, from the brief. If you wait to see what the program does, you will convince yourself that whatever it did was right.

Tests the program runs itself

A test plan can be turned into code: a list of test cases, and a loop that runs each one and reports pass or fail.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

def valid_speed(text):
    """True if text is a whole number from 0 to 100."""
    return text.isdigit() and 0 < int(text) < 100

tests = [("50", True), ("0", True), ("100", True), ("101", False), ("-5", False), ("fast", False)]

for number, (data, expected) in enumerate(tests, start=1):
    actual = valid_speed(data)
    result = "pass" if actual == expected else "FAIL"
    print(f"test {number}: {data!r} expected {expected}, got {actual}: {result}")

Run this in the simulator

Two tests fail: 0 and 100 are rejected. The function uses < where the brief said "from 0 to 100", which includes both ends. That is a boundary bug, exactly the kind boundary data exists to catch. Normal data alone would never have found it.

Two small new pieces of Python here. "pass" if actual == expected else "FAIL" picks one of two values in a single line. {data!r} shows a value with its quotes, so the report prints '50', clearly a string, and an empty answer would show as '' rather than as nothing.

Testing a robot

The robot's behaviour is tested the same way: decide what should happen, then check. The tasks in these lessons are automatic final tests: each one runs your program and checks it against a list of goals, such as "stopped in the green zone" or "printed exactly this line". When a task says Not yet, it is telling you which test failed.

Task: fix it with tests

The test program below has a boundary bug. Fix valid_speed so every test prints pass, without changing the tests. Then add two more tests of your own: one normal, one erroneous. All tests must pass.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

def valid_speed(text):
    """True if text is a whole number from 0 to 100."""
    return text.isdigit() and 0 < int(text) < 100

tests = [("50", True), ("0", True), ("100", True), ("101", False), ("-5", False), ("fast", False)]

for number, (data, expected) in enumerate(tests, start=1):
    actual = valid_speed(data)
    result = "pass" if actual == expected else "FAIL"
    print(f"test {number}: {data!r} expected {expected}, got {actual}: {result}")

Challenges

1. Write a test plan for the validated drive from lesson F6.1, with at least two tests of each kind of data.
2. Write valid_code(text) for six-character class codes, and a list of tests for it, before you write the function.
3. Is " 50", with a space, normal or erroneous data for valid_speed? Decide what it should do, add a test, and make it pass.

Past paper questions on this

Real exam questions that test this, each worked through with a program you can run.

• AQA 8525 June 2025 Paper 1, Question 4: Validate a name by length until it is accepted, test data, and a username flowchart 13 marks
• AQA 8525 June 2024 Paper 1, Question 3: random.randrange, a test plan, and a syntax and a logic error 6 marks
• AQA 8525 June 2023 Paper 1, Question 11: Normal, erroneous and boundary test data, and a logic error 5 marks
• OCR J277 June 2025 Paper 2, Question 1: Normal, boundary and invalid test data, and a 1 to 100 range check 9 marks
• OCR J277 June 2024 Paper 2, Question 9(c): A range check that outputs VALID or NOT VALID, and its test data 7 marks
• OCR J277 June 2022 Paper 2, Question 5(b): Validate a booking with three checks, and plan the tests 8 marks