Command words and levels of response

What each command word asks for, how extended answers are marked in levels, and a marker that levels answers with regular expressions.

A15.2Exam preparationA level45 min

The first word of a question tells you what kind of answer earns the marks. At A level the longest questions, often worth 9 or 12 marks, are not marked by counting facts but by levels of response: the examiner judges the quality of the whole answer. This lesson is how to read the question and how to write for the level you want.

Command words at A level

The commonest way to lose marks you know is to describe when the question says explain. "A stack is last in, first out" describes. "A stack suits undoing the robot's moves, because the last move made is the first that must be reversed" explains.

How short answers are marked

Short questions have a points-based mark scheme: one mark per valid point, up to the total. So a 4-mark "explain" question wants four separate marking points, or two points each with its reason. Read the marks as an instruction about how much to write.

• Use the scenario. If the question is about a delivery robot in a school, every point should mention the robot or the school. Generic answers often get no application marks.
• Use the vocabulary. "It is faster" is weak; "the cache holds recently used instructions, so fewer slow fetches from main memory are needed" is precise.
• Do not contradict yourself. Giving two answers, one right and one wrong, usually earns nothing.

How extended answers are marked

For extended questions the mark scheme has levels. Each level has a descriptor, and there is a list of indicative content: points a good answer might include, but not a checklist. The examiner reads the whole answer, decides which level's descriptor it fits best, then places it high or low within that level.

A typical 9-mark scheme looks like this:

What moves an answer up a level is almost never another fact. It is:

1. Development: each point followed by why it matters and what follows from it.
2. Application: each point tied to the scenario in the question.
3. Balance: benefits and risks, or both options, where the question has two sides.
4. Judgement: a conclusion that decides, and says why, especially for evaluate and discuss.

Planning a 9-mark answer

Take this question: A school is considering letting its delivery robot recognise teachers' faces so it delivers parcels to the right person. Discuss the issues the school should consider. (9 marks)

Spend two minutes planning. Aim for three or four developed points across both sides, then a conclusion:

That plan, written out in full sentences, is a level 3 answer. A list of ten issues in single lines, with no conclusion, stays in level 1 or 2 however many facts it holds.

A model of a marker

Real examiners read for meaning, and no program can do that. But the features that separate the levels, development, balance and a conclusion, are visible enough to check your own drafts. A regular expression (lesson A6.4) can find them:

import re

answer = "A benefit is speed, so staff save time. A risk is privacy. On balance it is worth it."
sentences = re.split(r"(?<=[.!?])\s+", answer)
for s in sentences:
    developed = bool(re.search(r"\b(because|so)\b", s, re.I))
    conclusion = bool(re.match(r"(on balance|overall|in conclusion)\b", s, re.I))
    print(f"{s!r}: developed={developed}, conclusion={conclusion}")

Run this in the simulator

(?<=[.!?])\s+ splits at spaces that come just after a full stop, question mark or exclamation mark. \b(because|so)\b finds either word as a whole word, so "also" and "sometimes" do not count.

Task: level the answers

Four students answered the face recognition question. Write a program that levels each answer using these rules. It is a crude model of a real examiner, but the rules are exactly defined.

• Split an answer into sentences with re.split(r"(?<=[.!?])\s+", answer.strip()).
• A sentence is the conclusion if it begins with On balance, Overall or In conclusion, ignoring capitals. A conclusion sentence is not counted as a point.
• Any other sentence is a point if it contains a word from FOR_WORDS or AGAINST_WORDS, ignoring capitals (a word counts if it appears anywhere in the sentence).
• A point is developed if it also contains because or so as a whole word, ignoring capitals. A developed point is on the for side if it contains a word from FOR_WORDS, and on the against side if it contains one from AGAINST_WORDS (it can be both).

Write analyse(answer), returning a tuple (points, developed, both_sides, conclusion): two whole numbers, then True if the developed points include both sides, then True if there is a conclusion.

Write level_and_mark(points, developed, both_sides, conclusion), returning a tuple (level, mark):

• level 3 if developed is at least 3, both_sides is true and there is a conclusion; the mark is 7 + min(2, developed - 3);
• otherwise level 2 if developed is at least 2; the mark is 4 + min(2, developed - 2);
• otherwise level 1 if points is at least 1; the mark is min(3, points);
• otherwise level 0 and 0 marks.

For each answer in order, print <name>: level <level>, <mark> marks (<points> points, <developed> developed). The robot stays still.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

import re

FOR_WORDS = ["benefit", "advantage", "helps"]
AGAINST_WORDS = ["risk", "drawback", "harm"]
answers = [
    ("Ada", "A benefit is that parcels reach the right teacher, because the robot recognises them. "
            "A risk is that faces are biometric data, so the school must have a lawful basis and keep the data secure. "
            "Another risk is bias, because a detector trained mostly on adults may fail for younger students. "
            "On balance the school should use a PIN instead, because it gives the benefit without storing faces."),
    ("Bolt", "One benefit is speed, so staff save time. Another benefit is accuracy, because no parcel goes to the wrong room. "
             "There is a risk to privacy."),
    ("Cog", "Face recognition is a benefit. It is also a risk."),
    ("Dot", "Robots are interesting and schools are busy places."),
]

Challenges

1. Rewrite Bolt's answer so your marker puts it in level 3, and check that a human examiner would agree.
2. Find a sentence your marker counts as developed that a human would not credit. What does that tell you about marking by keywords?
3. Write a plan for: Evaluate the use of an agile methodology for the school's robot booking system. (9 marks)