How the Internet works

Packet switching, routers and gateways, URLs, fully qualified domain names and DNS.

A12.3Networks and the webA level25 min

At GCSE (F10.5 and F10.6) you learned that data is split into packets, that the Internet is a network of networks, and that DNS turns names into IP addresses. At A level you need the detail: what exactly is in a packet, what a router decides and how, how a gateway differs from a router, the parts of a URL, and the chain of servers a DNS lookup really visits.

Packet switching

The Internet is a packet switched network. A message is split into packets, each sent independently. Each packet may take a different route across the network, and they may arrive out of order; the receiver uses sequence numbers to put them back together.

A packet has three parts:

The alternative is circuit switching, used by the old telephone network: a dedicated path is set up between the two ends before any data is sent, and held for the whole conversation.

Routers and gateways

A router connects networks and forwards packets between them. When a packet arrives, the router reads the destination IP address in its header, looks it up in its routing table, and sends the packet on to the next router (the next hop) on the best route it knows towards that destination. It also decrements the TTL. Routers exchange information with their neighbours so that their tables reflect links that are busy or have failed. No router knows the whole route: each only knows the best next hop.

A router joins networks that use the same protocols. A gateway is needed where networks use different protocols: it translates, removing the header of one network's packets and building the header the other network expects, so data can pass between them.

The Internet's addresses

A URL (uniform resource locator) is the full address of a resource on the Internet:

https://lessons.bugbot.example/a12/index.html
\___/   \____________________/ \____________/
scheme   fully qualified         path to the
         domain name             resource

• The scheme (or protocol) says how to fetch the resource: https, http, ftp.
• A domain name identifies an organisation or group on the Internet, such as bugbot.example or bbc.co.uk. It is a hierarchy read from the right: uk is the top-level domain, co.uk is below it, bbc.co.uk below that.
• A fully qualified domain name (FQDN) is a domain name that includes the host name as well, so it names one particular host: lessons.bugbot.example, www.bbc.co.uk. It is the complete name, all the way from the host to the top-level domain.
• An IP address identifies a device on the Internet. Routers need IP addresses, not names, so every name must be turned into an address before any packet can be sent.

Names and addresses must be unique, so someone must hand them out. Internet registries hold the records of which organisations own which blocks of IP addresses: the Internet Assigned Numbers Authority (IANA) allocates blocks to five regional Internet registries, which allocate them within their regions. Domain names are bought through domain name registrars, companies that record who owns each name in the registry for its top-level domain.

DNS

The domain name system (DNS) is a hierarchy of domain name servers that turns a domain name into an IP address. No single server holds every name. When your computer looks up lessons.bugbot.example:

1. It asks its resolver (usually run by the ISP or the local network). If the answer is in the resolver's cache from a recent lookup, it replies at once.
2. Otherwise the resolver asks a root name server. The root does not know the address, but knows which servers handle the top-level domain .example, and replies with their address.
3. The resolver asks the top-level domain server, which replies with the address of the server responsible for bugbot.example.
4. The resolver asks that authoritative name server, which holds the actual record, and gets the IP address.
5. The resolver caches the answer for a time and passes it back. Your computer can now send packets to that address.

# each name server knows only the next step down the hierarchy
servers = {
    "root": {"example": "tld-example", "uk": "tld-uk"},
    "tld-example": {"bugbot.example": "ns1.bugbot.example"},
    "tld-uk": {"co.uk": "ns.co.uk"},
    "ns1.bugbot.example": {"lessons.bugbot.example": "203.0.113.25", "shop.bugbot.example": "203.0.113.26"},
}
cache = {}

def resolve(fqdn):
    if fqdn in cache:
        print("  cache hit")
        return cache[fqdn]
    labels = fqdn.split(".")
    server = "root"
    for size in range(1, len(labels) + 1):
        name = ".".join(labels[-size:])        # "example", then "bugbot.example", then the whole name
        if name in servers[server]:
            answer = servers[server][name]
            print(f"  asked {server} about {name}: {answer}")
            if size == len(labels):
                cache[fqdn] = answer
                return answer
            server = answer
    return None

for name in ["lessons.bugbot.example", "lessons.bugbot.example"]:
    print("looking up", name)
    print("address:", resolve(name))

Run this in the simulator

The second lookup never leaves the resolver. Caching is why DNS copes with billions of lookups a day, and also why a changed record can take hours to reach everyone.

Task: put the packets back together

The Server on this mat sends one short message as four packets, one every 0.4 seconds, but not in order. Each packet is text in the form <number>/<total>:<payload>, for example 2/4:an take d. The number counts from 1, the total is how many packets make the message, and the payload is everything after the first colon (it may contain spaces).

Listen with messages(), checking every 0.1 s, until you hold every packet. Then print two lines:

• arrived in order: <numbers>, the packet numbers in the order they arrived, separated by single spaces;
• message: <text>, the payloads joined in number order.

The robot does not drive.

# the two lines every program starts with: the commands, then the robot
from bugbot import *
connect()

wait(2)
for sender, text in messages():
    print(text)

Challenges

1. Add a TTL to the DNS cell: stop a lookup after 3 servers and print too many hops.
2. Drop one packet: have your program give up after 3 seconds and print which packet numbers are missing.
3. Split https://www.bbc.co.uk/news/technology into its scheme, FQDN, domain name and path with string methods.