A Head Start That Compounds

Most stories put the fatal decision at midnight. Somebody frightened, or cruel, or cornered, in a room with the lights flickering, makes the call the rest of the story pays for. It is good drama, and I understand why writers reach for it. When I wrote the middle of Utopia: The First Awakening, I put the fatal decision somewhere else on purpose, because I do not think that is where the important ones are actually made.

In the novel it reads like this: "It was made in the afternoon, in good light, over coffee that went cold because the conversation was pleasant enough that no one remembered to drink it."

The good years

By that point the founders of a floating city built from the ocean's garbage have done what they promised. The sea is coming back. The clinics are healing people, family by family, lifting out the inherited death sentences that used to be a matter of waiting. The island has gone from a proof to a town to a small strange country. These are the years the book calls the Utopian years, and I wrote them as real, because they are. A paradise you were never given cannot be lost, and I wanted the reader to feel that this one was given.

Then comes the meeting. The same science that removes a disease from a genome does not know where the disease stops. It could keep going, past healing, into making a person more: stronger, quicker, longer lived. Nobody in the room is a villain. The geneticist is tired of pretending that the line between curing and improving is anything but a line she drew herself. The founder wants to open it to everyone. And the ethicist, Sheba, finishes his sentence for him: to everyone who can pay.

Every sentence true

What I wanted the chapter to do was give the founder the better argument, and I think it does. He points out that a free cure builds one clinic, a monument, and everyone past its doors dies the way they would have anyway. A priced cure builds forty, and the price falls every year, the way prices always fall, until the privilege is a vaccine. That is how phones reached the poor, and medicines, and electric light. History is on his side of the table, and he knows it, and he says it gently.

Sheba does not have a better argument. What she has is one sentence, and I built the chapter around it: "You are not selling a head start that closes. You are selling one that compounds."

That is the whole difference, and it is the one we keep missing. Most head starts close. The first people to own a phone had a few years of advantage and then everyone had one. But some head starts are not like that. If what you sell to the first buyers is more years, more strength, more speed of thought, then the people who buy first get to spend the decade while the price falls building a world that runs at their speed. By the time the thing is cheap, the distance is permanent, because the distance was used to set the terms for everyone who arrives later.

We are used to asking whether a new thing is good, and whether it is fair to charge for it. Those are the questions the founder answers, and he answers them well. The question Sheba is asking is different. She is asking what the thing does to the gap between people while it is still expensive, and whether that gap will still be there after the price has come down. Some advantages wash out. Others are self-reinforcing: the people who have them use them to get more of them. The test is not whether the price falls. It is what the first buyers can do with the years before it does.

Saying yes on the record

Sheba holds a veto under the founding contract. She can stop any single step of the work. And in the meeting she discovers the problem with a veto, which is that it needs a clean moment to land on. Nobody hands her one. She is given a hundred reasonable steps, each defensible, none of them the one worth spending the veto on. So she does something I find braver than stopping it. She says yes out loud, and asks that it be written down that she understood what she was agreeing to, so that nobody can later tell the story where she was not in the room.

And then she says the line that I think is the real warning of the book, more than any machine: "That we're doing it because we're right, one reasonable afternoon at a time."

Why this is not about the machine

I build AI for a living, so I want to be careful about what this chapter claims. The danger in it is not a machine that wants something. Today's AI wants nothing. Nobody knows what self-improving systems will be able to do, and the novel goes where it goes with that question. But the chapter is about something older, and it would be true with no machine in the story at all. More capable is never more human. A person made stronger and quicker has not become worth more than the person next to them, and a society that prices the difference has already answered the question the whole book turns on: what a human being is worth.

That question never gets asked out loud in the meeting. It is answered anyway, by a number written on a whiteboard, by good people in good light, with the best intentions anyone could bring to a table. That is the scene I wanted to write, because I think it is the scene we are most likely to live. Not the midnight. The afternoon.

If you are building something powerful, the question worth asking in your own pleasant meetings is not only whether each step is right. It is whether the head start you are selling closes, or compounds.