Reading and writing files

Open, read, write, append and close: a run log in a CSV file.

F7.1Files and databasesGCSE15 min

Everything a program stores in variables disappears when it stops. To keep data between runs, a program writes it to a file, and reads it back next time. Robots keep logs this way: every run, what happened, and how it went. In this lesson BugBot reads its run log from a file and adds to it.

The files on this page

Here is a text file called runs.csv. It is a real file as far as the programs on this page are concerned: they can open it, read it, and write to it, and when a program changes it you will see the change appear here.

run,task,cm,seconds
1,wall,42,3.1
2,wall,38,2.7
3,square,80,9.4

It is a CSV file: comma-separated values. Each line is a record, and the commas separate its fields. The first line is a header naming the fields.

Reading a file

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

f = open("runs.csv", "r")      # open the file for reading
text = f.read()                # read the whole file into one string
f.close()                      # always close a file when you have finished
print(text)

Run this in the simulator

Three steps, the same in every language: open the file, read from it, close it. "r" is the mode: read. Closing tells the computer you have finished, so the file is not left locked or half written.

Line by line

Most of the time a program wants one line at a time:

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

with open("runs.csv", "r") as f:
    header = f.readline()
    for line in f:
        fields = line.strip().split(",")
        print("run", fields[0], "drove", fields[2], "cm in", fields[3], "s")

Run this in the simulator

with open(...) as f: opens the file and closes it automatically at the end of the block, even if something goes wrong inside. It is the usual way to open files in Python. readline() reads one line; a for loop over the file reads the rest, a line at a time. strip() removes the newline at the end, and split(",") breaks the line into its fields, as strings.

Writing a file

Mode "w" writes a new file, and empties the file first if it already exists. Mode "a" appends: it adds to the end and keeps what was there.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

start = clock()
forward(50, distance=30)
seconds = round(clock() - start, 1)

with open("runs.csv", "a") as f:
    f.write("4,wall,30," + str(seconds) + "\n")
print("logged a run of 30 cm in", seconds, "s")

Run this in the simulator

Run it, then look at runs.csv above: the new line is there. Run it again, and another line is added. write does not add a newline, so each line ends with "\n" yourself, and numbers must be turned into strings first.

Press Reset file on runs.csv to put it back.

The whole file as a list of records

Reading a CSV into a list of records combines this module with F3:

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

runs = []
with open("runs.csv") as f:
    f.readline()                          # skip the header
    for line in f:
        run, task, cm, seconds = line.strip().split(",")
        runs.append({"run": int(run), "task": task, "cm": float(cm), "seconds": float(seconds)})

best = runs[0]
for r in runs:
    if r["cm"] / r["seconds"] > best["cm"] / best["seconds"]:
        best = r
print("fastest run:", best["run"], "at", round(best["cm"] / best["seconds"], 1), "cm per second")

Run this in the simulator

open("runs.csv") with no mode reads. Four names on the left of = take the four fields of the line in order.

When files go wrong

Opening a file that does not exist in read mode is a runtime error, FileNotFoundError. A robust program expects it, for example by creating the file first with mode "a", which makes the file if it is not there. Try changing "runs.csv" to "run.csv" in the first cell to see the error.

In the exam

Questions ask you to write code that opens a file, reads or writes it, and closes it; to read a file line by line until the end; and to explain why data is stored in files.

OCR 2.2.3

The Exam Reference Language opens a file, reads and writes lines, and closes it:

myFile = open("runs.txt")
while NOT myFile.endOfFile()
    print(myFile.readLine())
endwhile
myFile.close()

newFile("log.txt")
logFile = open("log.txt")
logFile.writeLine("run 4")
logFile.close()

@aqa Reading and writing files is not on AQA's 8525 specification. Storing data between runs is still how the databases in the next lessons keep their tables.

Edexcel 6.4.2

Edexcel asks you to write code that reads from and writes to text files, including CSV files, in Python, using modes such as "r", "w" and "a", and splitting lines into fields.

Task: log the run

Drive forward 25 cm, timing it with clock(). Then append a line to runs.csv for this run: run number 4, task wall, 25 cm, and the seconds rounded to one decimal place, in the same format as the other lines. Finally read the file and print 4 runs logged, counting the lines after the header rather than typing 4.

The task starts from the original four-line runs.csv every time it runs, whatever earlier runs added to the copy above.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

start = clock()
forward(50, distance=25)

Challenges

Write a program that prints the total distance of all the runs in the file.
Log a whole session: drive three different distances, appending a line for each.
What happens to runs.csv if you use mode "w" instead of "a"? Try it, then reset the file.