Models as text

Saving and loading a model with JSON; models trained elsewhere.

8.7LearningRobot club15 min

A model is data. For nearest neighbour it is the samples; for the network it is the weights. Data can be written down, and written-down data can be kept, shared, and loaded into a different program, on a different machine, on a different day. This lesson does that with JSON.

JSON

JSON is a way of writing lists, dictionaries, numbers, strings, True, False and None as text. Python's json module turns things into text with dumps and back with loads.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

import json
DATA = [
    ([54, 52, 51, 51, 51, 51, 52, 54, 54, 52, 51, 51, 51, 51, 52, 54], 'open'),
    ([16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16], 'wall-ahead'),
    ([36, 47, 46, 46, 46, 16, 16, 16, 36, 47, 46, 46, 46, 16, 16, 16], 'gap-left'),
    ([16, 16, 16, 46, 46, 46, 47, 36, 16, 16, 16, 46, 46, 46, 47, 36], 'gap-right'),
]

model = {"kind": "nearest-neighbour", "features": "depth grid rows 2 and 3",
         "samples": [{"x": feats, "label": label} for feats, label in DATA]}
text = json.dumps(model)
print(len(text), "characters, starting", text[:80])
back = json.loads(text)
print("same after the round trip:", back == model)

Run this in the simulator

That text is the whole model. Print it, copy it, paste it into a file, send it to a friend. Pairs became lists on the way (JSON has no tuples), which is why the samples are stored as small dictionaries with named fields instead: the names travel with the data.

A model from somewhere else

MODEL below is a model saved as text, just as if you had pasted it from a file. Your program does not need to know who made it or how. It needs the kind to know what to do with it, and the samples.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

import json
MODEL = '{"kind": "nearest-neighbour", "features": "depth grid rows 2 and 3", "samples": [{"x": [54, 52, 51, 51, 51, 51, 52, 54, 54, 52, 51, 51, 51, 51, 52, 54], "label": "open"}, {"x": [16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16], "label": "wall-ahead"}, {"x": [36, 47, 46, 46, 46, 16, 16, 16, 36, 47, 46, 46, 46, 16, 16, 16], "label": "gap-left"}, {"x": [16, 16, 16, 46, 46, 46, 47, 36, 16, 16, 16, 46, 46, 46, 47, 36], "label": "gap-right"}]}'

model = json.loads(MODEL)
print(model["kind"], "using", model["features"])
print(len(model["samples"]), "samples, labels:", sorted(set(s["label"] for s in model["samples"])))

Run this in the simulator

Using it

Turn the samples back into (features, label) pairs and hand them to nearest:

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

import json
def level_rows():
    # rows 2 and 3: the 16 readings that look straight ahead
    return tof_grid()[16:32]

def nearest(sample, data):
    # the label of the recorded sample most like this one (smallest sum of squared differences)
    best_label, best_d = None, 1e18
    for feats, label in data:
        d = sum((a - b) ** 2 for a, b in zip(feats, sample))
        if d < best_d:
            best_label, best_d = label, d
    return best_label
MODEL = '{"kind": "nearest-neighbour", "features": "depth grid rows 2 and 3", "samples": [{"x": [54, 52, 51, 51, 51, 51, 52, 54, 54, 52, 51, 51, 51, 51, 52, 54], "label": "open"}, {"x": [16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16], "label": "wall-ahead"}, {"x": [36, 47, 46, 46, 46, 16, 16, 16, 36, 47, 46, 46, 46, 16, 16, 16], "label": "gap-left"}, {"x": [16, 16, 16, 46, 46, 46, 47, 36, 16, 16, 16, 46, 46, 46, 47, 36], "label": "gap-right"}]}'

model = json.loads(MODEL)
data = [(s["x"], s["label"]) for s in model["samples"]]
print("model says:", nearest(level_rows(), data))

Run this in the simulator

The network's weights save the same way: W1.tolist() turns a numpy table into plain lists that JSON can write, and np.array(...) turns them back.

Task: a model from text

Load the model in MODEL, classify the current view with it, and print model says: <label>. Do not drive.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()
import json
MODEL = '{"kind": "nearest-neighbour", "features": "depth grid rows 2 and 3", "samples": [{"x": [54, 52, 51, 51, 51, 51, 52, 54, 54, 52, 51, 51, 51, 51, 52, 54], "label": "open"}, {"x": [16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16], "label": "wall-ahead"}, {"x": [36, 47, 46, 46, 46, 16, 16, 16, 36, 47, 46, 46, 46, 16, 16, 16], "label": "gap-left"}, {"x": [16, 16, 16, 46, 46, 46, 47, 36, 16, 16, 16, 46, 46, 46, 47, 36], "label": "gap-right"}]}'
model = json.loads(MODEL)
print(model['kind'], len(model['samples']), 'samples')

Challenges

Save the trained network from lesson 8.4 as JSON with kind set to network, and write predict(model, view) that works for both kinds.
Add a trained_on field with today's date and print it when the model loads.
Store k in the model and vote among the k nearest samples.