Reading and writing files

Open, read, write, append and close: a run log in a CSV file.

F7.1Files and databasesGCSE15 min

Covers OCR J277 2.2.3 Additional programming techniques · Edexcel 1CP2 6.4.2 Reading and writing CSV files

Do this lesson in the simulator

Print it: worksheet (PDF) · answers (PDF) · as a web page

Everything a program stores in variables disappears when it stops. To keep data between runs, a program writes it to a file, and reads it back next time. Robots keep logs this way: every run, what happened, and how it went. In this lesson BugBot reads its run log from a file and adds to it.

The files on this page

Here is a text file called runs.csv. It is a real file as far as the programs on this page are concerned: they can open it, read it, and write to it, and when a program changes it you will see the change appear here.

run,task,cm,seconds
1,wall,42,3.1
2,wall,38,2.7
3,square,80,9.4

It is a CSV file: comma-separated values. Each line is a record, and the commas separate its fields. The first line is a header naming the fields.

Reading a file

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

f = open("runs.csv", "r")      # open the file for reading
text = f.read()                # read the whole file into one string
f.close()                      # always close a file when you have finished
print(text)

Run this in the simulator

Three steps, the same in every language: open the file, read from it, close it. "r" is the mode: read. Closing tells the computer you have finished, so the file is not left locked or half written.

The program opens runs.csv, reads the whole file into one string called text, and closes itruns.csv, on the diskrun,task,cm,seconds\n1,wall,42,3.1\n2,wall,38,2.7\n3,square,80,9.4\nthe programf = open("runs.csv", "r")text = f.read()f.close()1 f links to it2 copies it all3 link endstext is now one string, with a \n where each line ended:"run,task,cm,seconds\n1,wall,42,3.1\n2,wall,38,2.7\n3,square,80,9.4\n"
Open links the variable f to the file, read copies every character into text, and close ends the link so the file is not left locked. The 4 lines arrive as one string: the green \n marks are where the lines broke.

Line by line

Most of the time a program wants one line at a time:

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

with open("runs.csv", "r") as f:
    header = f.readline()
    for line in f:
        fields = line.strip().split(",")
        print("run", fields[0], "drove", fields[2], "cm in", fields[3], "s")

Run this in the simulator

with open(...) as f: opens the file and closes it automatically at the end of the block, even if something goes wrong inside. It is the usual way to open files in Python. readline() reads one line; a for loop over the file reads the rest, a line at a time. strip() removes the newline at the end, and split(",") breaks the line into its fields, as strings.

readline takes the header, the for loop takes one line each turn, and turn 1's line is stripped and split into four fieldsruns.csvrun,task,cm,seconds\n1,wall,42,3.1\n2,wall,38,2.7\n3,square,80,9.4\nreadline(): the headerfor loop, turn 1for loop, turn 2for loop, turn 3turn 1, step by stepline = "1,wall,42,3.1\n"strip() removes the \n "1,wall,42,3.1"split(",") cuts at each comma"1"fields[0]"wall"fields[1]"42"fields[2]"3.1"fields[3]print:run 1 drove 42 cmin 3.1 s
readline() uses up the header, so the loop starts at the first run and takes one line per turn, 3 turns in all. Each field comes out as a string in quotes, such as "42": it is text until int() or float() makes it a number.

Writing a file

Mode "w" writes a new file, and empties the file first if it already exists. Mode "a" appends: it adds to the end and keeps what was there.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

start = clock()
forward(50, distance=30)
seconds = round(clock() - start, 1)

with open("runs.csv", "a") as f:
    f.write("4,wall,30," + str(seconds) + "\n")
print("logged a run of 30 cm in", seconds, "s")

Run this in the simulator

Run it, then look at runs.csv above: the new line is there. Run it again, and another line is added. write does not add a newline, so each line ends with "\n" yourself, and numbers must be turned into strings first.

Mode "a" adds the new line after the last one and keeps the rest; without "\n" the next write would join onto the same linebeforerun,task,cm,seconds\n1,wall,42,3.1\n2,wall,38,2.7\n3,square,80,9.4\nafter one runrun,task,cm,seconds\n1,wall,42,3.1\n2,wall,38,2.7\n3,square,80,9.4\n4,wall,30,time\nopen "a"writeadded at the endIf the "\n" is left out, the line never ends, so asecond run carries on along the same line:4,wall,30,time4,wall,30,timeone line, not two
Mode "a" leaves the four lines alone and writes after the last \n. Here "time" stands for the seconds the run took, which change every run. write() adds only what you give it, so the "\n" at the end is what lets the next record start on a new line.

Press Reset file on runs.csv to put it back.

The whole file as a list of records

Reading a CSV into a list of records combines this module with F3:

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

runs = []
with open("runs.csv") as f:
    f.readline()                          # skip the header
    for line in f:
        run, task, cm, seconds = line.strip().split(",")
        runs.append({"run": int(run), "task": task, "cm": float(cm), "seconds": float(seconds)})

best = runs[0]
for r in runs:
    if r["cm"] / r["seconds"] > best["cm"] / best["seconds"]:
        best = r
print("fastest run:", best["run"], "at", round(best["cm"] / best["seconds"], 1), "cm per second")

Run this in the simulator

open("runs.csv") with no mode reads. Four names on the left of = take the four fields of the line in order.

When files go wrong

Opening a file that does not exist in read mode is a runtime error, FileNotFoundError. A robust program expects it, for example by creating the file first with mode "a", which makes the file if it is not there. Try changing "runs.csv" to "run.csv" in the first cell to see the error.

Task: log the run

Drive forward 25 cm, timing it with clock(). Then append a line to runs.csv for this run: run number 4, task wall, 25 cm, and the seconds rounded to one decimal place, in the same format as the other lines. Finally read the file and print 4 runs logged, counting the lines after the header rather than typing 4.

The task starts from the original four-line runs.csv every time it runs, whatever earlier runs added to the copy above.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

start = clock()
forward(50, distance=25)

Challenges

  1. Write a program that prints the total distance of all the runs in the file.
  2. Log a whole session: drive three different distances, appending a line for each.
  3. What happens to runs.csv if you use mode "w" instead of "a"? Try it, then reset the file.

Past paper questions on this

Real exam questions that test this, each worked through with a program you can run.