Kademlia, 2002 · Git, April 2005

II

What that produces in practice is a system with no central register at all where anybody finds anything in a dozen questions.

There is no server to switch off because there is no server; there is no membership list because each node knows a few hundred and none knows the whole. A node coming in only needs to know the address of any other to begin placing itself, and from there it positions itself alone.

It has been switched on since 2005 with no known interruption. At any given moment there are of the order of several million simultaneous nodes, coming and going without warning, on domestic connections of variable quality and with no coordination at all.

It is, with fair probability, the largest genuinely distributed system that has ever worked continuously. No organisation maintains it. It has no budget and no headquarters, and the reason it was designed that way was not ideological but defensive — what has no centre cannot be sued or seized.

III

The second case is the one that affects us daily, and it was born of a quarrel over a licence.

The Linux kernel had been developed since 2002 with a commercial tool called BitKeeper, which its author gave free to free projects. In April 2005 that grant was withdrawn, over a disagreement about reverse engineering somebody had done of the protocol.

Linus Torvalds wrote the replacement in a few days. The design criterion he stated was that the tool should have no server; each copy of the repository had to contain the entire history, signed and verifiable, and any two copies had to be able to synchronise with each other without passing through any third.

That is exactly the right-hand panel. Each clone is a complete node, any pair of nodes can exchange, there is no hierarchy in the protocol and there is nothing one node has that another cannot obtain.

And it works. The Linux kernel has been developed that way since 2005, with thousands of people, by email and with trees that merge into one another without any of them being technically the principal one.

IV

And even so, everybody pushes to the same place.

The most deliberately decentralised tool in daily use operates, in practice, against a server a company bought in 2018 for seven thousand five hundred million dollars. When that server goes down, the integration pipelines of half the world stop — and they stop even though the code is whole, complete and signed on every developer's disk. The property Torvalds considered non-negotiable is still there, intact, and is of no use whatever on that day.

It is worth asking why, because it was not out of ignorance or laziness. Those who use Git know perfectly well that it does not need it.

What happens is that Git distributes the data and distributes nothing else. It has no issues, no reviews, no discussion, no permissions, no identity, no way for a stranger to find you. All that —which is the work, and not the storage— was left outside the protocol, and somebody built it on top and offered it free.

There is the lesson of the whole series, and there is nothing ironic about it. The hard part was never distributing the file: that was solved by Kademlia in 2002 and Git in 2005, and they solved it well. The hard part is finding one another, agreeing and trusting, and for that a place where everybody is is unbeatably convenient.

It is as well besides not to idealise the case that does work. The distributed table has its own problems and they are known: a malicious node can manufacture identifiers to measure in order to place itself near a particular key and answer lies; the network is observable, so anybody can list who is asking for what; and the cold start needs to know somebody beforehand, which in practice means a list of seed nodes that somebody maintains.

None of those defects has brought it down since then. But the honest sentence is not that it has no centre: it is that it has several soft points, distributed, and that none of them has turned out to be enough to switch it off.

V

Sixty years after Baran's report, the balance can be written in two lines.

The right-hand panel is buildable, it was built and it goes on working: there are several million nodes moving in an ownerless table while this is being read. And the middle panel is the one we use, not because the technique failed, but because concentrating is still cheaper and more convenient at every step, one by one, for each person who decides it.

There is in none of these eight texts a villain. There is an operator who carried out an order, a telephone company that defended its business, a county in Virginia that offered cheap land, a building from 1930 with good ducts and a non-profit organisation that gave certificates away.

Each of those decisions was reasonable on its own, and several of them were besides good. The drawing they all make together was proposed by nobody, and there is no meeting at which it was approved.

That is what has to be looked at in the figure of 1964. It is not that somebody took the right-hand panel away from us. It is that nobody ever chose it, because there was never a place where it would be chosen.

Baran, who devoted two years to calculating how many links are needed to survive a war, never got as far as posing that question. It was not his.