BGP · Islamabad, 2008 · Menlo Park, 2021

II

To understand why that is possible you have to look at the protocol, and the protocol is no mystery. It is written down, it is public and it has worked without fundamental changes since 1989.

The internet is not a network. It is some seventy thousand independent networks [autonomous systems] that have agreed to speak to each other. Each has a number, and between them they use the Border Gateway Protocol, which in 1989 Kirk Lougheed and Yakov Rekhter sketched during an IETF meeting on the back of three napkins.

What it does is simple; each network tells its neighbours which blocks of addresses it knows how to reach and by which route. The neighbours choose the route they prefer and tell their own. When there are two routes to the same place the more specific one wins — an announcement of a thousand addresses beats one of four thousand that contains them — and that is the rule Islamabad used.

That table is not small. A router that wants to speak to the whole internet keeps close to a million routes today, and recalculates them every time any of the seventy thousand networks changes its mind about something. It is a data structure. It fits in the memory of a rack unit, and with it the traffic of a continent is decided.

There is in the base protocol no mechanism that checks that whoever announces a block has the right to announce it. An autonomous system says what it knows how to reach. The others believe it.

That is not a fault that escaped them, it is the design. In 1989 the networks that spoke to each other were a few hundred, their administrators knew each other by name and trust was cheaper than verification. The mechanism was never changed, because replacing the routing system of the whole world while it is switched on is not an operation anybody wants to lead.

III

On 4 October 2021, at 15:39 UTC, Facebook did the same as Pakistan Telecom but in reverse. Instead of announcing other people's routes, it withdrew its own.

What lay behind it was a routine maintenance task: a command that assessed the capacity of the backbone network. The command ran badly and brought down all the backbone connections at once, leaving the data centres isolated from one another.

At that point a piece that was working exactly as it had been designed came into play. Facebook's name servers continually check whether they can speak to the data centres, and if they cannot, they stop announcing themselves to the world; the idea is that a broken server should not go on receiving queries it cannot serve [a health check, written by somebody prudent]. Since none of them could speak to anything, they all withdrew at once.

Within the space of a few minutes, the names of Facebook, Instagram and WhatsApp stopped resolving. It is not that the servers were down; it is that they had erased from the world's tables the indication of where to find them.

It lasted about six hours. What lengthened it was not the diagnosis. It was that remote access to the equipment went through the same network that had disconnected itself, so somebody had to go physically to the data centres; and the door readers consulted systems that had also disappeared.

IV

The two stories have the same shape and it is as well not to soften it with the word error.

In 2008, an operator announced a false route and the rest of the world accepted it because the protocol accepts. In 2021, an operator withdrew its true routes and the rest of the world forgot them because the protocol forgets. Neither time was anything broken: it was enough to use the mechanism exactly as it is written.

And what is more uncomfortable, in both cases the system worked as foreseen. The Pakistani network was not attacked; Facebook's name server did not fail. Each piece did its job and the result was that a country switched a site off for everybody and that a company erased itself from the map.

It is as well besides to undo a sentence that gets repeated a great deal — the one about the network interpreting censorship as a fault and routing around it. In Islamabad it did not route around it: it propagated it, and propagated it fast and to everybody, because the mechanism described as resilience is in reality obedience to whatever the neighbours say. What makes a network robust against a cut cable makes it fragile against a neighbour who gets it wrong, and they are the same property seen from the two sides.

There is a version with money in it, in case the technical reading is not enough. In April 2018 somebody announced routes towards Amazon's name servers, diverted the users of a cryptocurrency wallet to a copy of the site and took away the equivalent of some hundred and fifty thousand dollars in two hours. The procedure was Islamabad's, with another purpose.

V

The remedy exists and has existed for years. It is called route origin validation [RPKI]. The holders of address blocks cryptographically sign a certificate that says which autonomous system has the right to announce them, and the networks that check it discard the announcements that do not add up.

It is deployed in a large part of the world, not in all of it, and it protects against the announcement of the wrong origin — not against a route falsified further along the chain. The extensions that would cover that have been under discussion for more than a decade.

Meanwhile, what decides where the traffic of this text, of a payment, of a medical record or of a purchase order travels is still what some seventy thousand networks tell one another about themselves.

Baran had designed a routing in which each node learned the paths from the traffic that passed over it, with no central authority and with no need to trust anybody. What was built learns from what the neighbours tell it, and trusts.