Performance, pipelining and parallel processors

Cores, cache, clock speed, word length and bus widths; pipelining; CISC and RISC; multicore systems and GPUs.

A9.8Computer architectureA level25 min

At GCSE (F9.5) you learned that clock speed, cores and cache affect performance. At A level you need the full list of factors with the reason each one matters, and three ideas that modern processors are built on: pipelining, the choice between CISC and RISC instruction sets, and parallel processing with multicore processors and GPUs.

Factors affecting performance

Factor	Why it matters	The catch
Clock speed	each tick moves the processor on a step, so more ticks a second means more instructions a second	faster switching makes more heat and uses more power; the chip must be cooled, and a battery drains faster
Number of cores	each core is a complete processor that can run its own fetch-decode-execute cycle, so several tasks, or parts of one task, run at the same time	only helps if the software is written to split work between cores, and some work cannot be split; coordinating the cores has a cost
Cache memory	a small amount of very fast memory, on or next to the processor, holds copies of instructions and data used recently or often; each time the processor finds what it needs there (a hit), it avoids a slower trip to main memory	cache is expensive; a bigger cache takes longer to search, which is why there are levels (L1 small and fastest, then L2, then L3)
Word length	the number of bits the processor handles as one unit; a longer word does more in one operation, holds larger numbers and more precise values, and can address more memory directly	software must be written for that word length
Address bus width	more address lines means more memory locations can be addressed, so more programs and data fit in main memory at once	it does not make any one instruction faster
Data bus width	a wider bus moves more bits in each transfer, so large values move in fewer transfers	wider buses need more wires and connections

Clock speed alone does not decide performance. A processor that does more work per tick, or has more cores, or misses the cache less often, can beat one with a faster clock.

Pipelining

Without pipelining, the processor finishes one instruction before starting the next. But fetch, decode and execute are done by different parts of the processor, and while one instruction is being executed, the fetch hardware sits idle.

Pipelining keeps every stage busy. As soon as an instruction moves from fetch to decode, the next instruction is fetched. Like a production line, each stage passes its work on and takes the next item.

Tick	Fetch	Decode	Execute
1	I1
2	I2	I1
3	I3	I2	I1
4	I4	I3	I2
5		I4	I3
6			I4

Four instructions take 6 ticks instead of 4 × 3 = 12. In general, n instructions through s stages take n + s - 1 ticks. Pipelining does not make any single instruction faster: it increases throughput, the number of instructions completed per second.

It has a problem: branches. The processor has already fetched and decoded the instructions after a branch before it executes the branch. If the branch is taken, those instructions are the wrong ones, and they must be flushed from the pipeline, wasting those ticks. Processors reduce the damage with branch prediction: guessing which way a branch will go, based on what it did last time.

CISC and RISC

	CISC: complex instruction set computer	RISC: reduced instruction set computer
Instructions	many, some doing complex jobs in one instruction	a small set of simple instructions
Instruction length	variable	fixed, often 32 bits
Time per instruction	many take several clock cycles	each takes about one clock cycle
Programs	fewer instructions per program, so less memory for code	more instructions per program
Where the work is	in the hardware, which is more complex	in the compiler, which must build complex operations from simple ones
Pipelining	harder, because instructions vary in length and time	easy, because every instruction is the same shape
Registers and memory	fewer general-purpose registers; many instructions work on memory directly	many general-purpose registers; only load and store touch memory
Power	more transistors, more power	fewer transistors, less power and heat
Examples	Intel and AMD x86 in most desktop PCs and servers	ARM in almost every phone, RISC-V in BugBot's processor

RISC's low power and easy pipelining are why it dominates phones, tablets and embedded systems. CISC kept the desktop because decades of software were written for x86. The line has blurred: modern x86 processors translate their complex instructions into simple RISC-like steps inside the chip, and ARM processors now run laptops and servers.

BugBot's ESP32-P4 has two main RISC-V cores, running at up to 400 MHz, and a separate low-power core. Every RISC-V instruction in its basic set is 32 bits long.

Multicore and parallel systems

A multicore processor has two or more cores on one chip, sharing main memory. A parallel system more generally uses several processors, or several computers, working on one problem at once: from a phone with eight cores to a supercomputer with thousands of processors.

Parallel processing only speeds up the part of a job that can be split. If each step needs the result of the step before (such as a program that works out each position of a robot from its last position), it must run one step at a time on one core. The part that cannot be split limits the speedup however many cores are added:

def speedup(parallel_fraction, cores):
    serial = 1 - parallel_fraction
    return 1 / (serial + parallel_fraction / cores)

for cores in [1, 2, 4, 8, 64, 1024]:
    print(f"{cores:5} cores: {speedup(0.9, cores):.2f} times faster")

Run this in the simulator

With 90% of a job parallel, even 1,024 cores give less than a tenfold speedup, because the 10% that must run serially is still there. (This is known as Amdahl's law. It is why doubling the cores rarely halves the time.)

GPUs

A graphics processing unit has hundreds or thousands of small, simple cores. Each is much less capable than a CPU core, but together they do the same operation on huge amounts of data at once: single instruction, multiple data (SIMD). Graphics needs exactly that: the same calculation for every pixel on the screen, or every corner of every shape in a 3D scene.

Many other problems have the same shape, so GPUs are also used for:

training and running machine learning models, which are mostly huge matrix multiplications;
scientific simulation: weather, fluids, molecules, and sound or light waves in a grid;
video encoding and image processing;
cryptocurrency mining and password cracking, which try enormous numbers of hashes.

A GPU is poor at work full of decisions and branches, or where each step depends on the last. That stays on the CPU, which hands the parallel parts to the GPU. This is why BugBot keeps heavy AI work, such as recognising objects in camera images, on your laptop rather than on the robot: a laptop's GPU can do it far faster than the robot's small processor.

In the exam

Questions ask you to explain how a named factor affects performance, to explain why doubling cores does not always double speed, to describe pipelining and the effect of a branch, to compare CISC and RISC, and to give uses of GPUs beyond graphics. Always link the factor to what the processor does faster: "more cache" alone earns little; "fewer slow fetches from main memory" earns the mark.

AQA A level 4.7.3.7

AQA asks you to explain the effect on processor performance of multiple cores, cache memory, clock speed, word length, address bus width and data bus width.

OCR A level 1.1.1 1.1.2

OCR asks for the factors affecting CPU performance (clock speed, number of cores and cache), the use of pipelining to improve efficiency, the differences between and uses of CISC and RISC processors, GPUs and their uses including those not related to graphics, and multicore and parallel systems.

Eduqas A level 2.1

Eduqas asks about contemporary processor architectures and parallel processing in its hardware and communication topic. Be ready to describe how processors run instructions in parallel and why it helps.

Task: a pipeline, tick by tick

Simulate a pipeline. instructions is the list ["I1", "I2", "I3", "I4"] and stages is the list ["F", "D", "E"], in pipeline order. At each tick, starting from tick 1, every stage holds the instruction that entered the pipeline that many ticks after it, or nothing.

For every tick until the last instruction leaves the last stage, print one line in the form tick 2: F=I2 D=I1 E=-, with each stage's name, =, and the instruction it holds or - for an empty stage, separated by single spaces.
Then print pipelined: 6 ticks and not pipelined: 12 ticks, working both numbers out from the lengths of the two lists.

The robot does not move.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

instructions = ["I1", "I2", "I3", "I4"]
stages = ["F", "D", "E"]

Challenges

Change the pipeline to five stages. How many ticks does it take now for four instructions, and for 100?
Instruction I2 is a branch that is taken, so I3 and I4 were the wrong instructions. Print the pipeline with those two flushed and the correct instructions I7 and I8 fetched instead. How many ticks were wasted?
A job is 60% parallel. What is the most it can ever be sped up by adding cores?