The Oldest Argument Is a Sound

Novels about artificial intelligence usually make their case for humanity in an argument. Somebody stands up in the last act and gives the speech: about love, or sacrifice, or the soul, or whatever the machine supposedly lacks. I understand the instinct. It is satisfying to have the point said out loud. But when I sat down to write Utopia: The First Awakening, I wanted the first and deepest case for human worth to arrive before anyone had argued anything at all. So I put it in the second chapter, at a concert, and I gave it to a singer.

A stage on the water

By that point in the book, four people from a small Berkeley lab have been invited to see a city that two billionaires raised out of the plastic in the Pacific. They have seen it on screens. They have admired it as engineering. Then they go out to it, on Valentine's Day, for a concert, and the place stops being footage.

The stage floats a thousand miles from any shore. The speakers are grown from the same recycled plastic as everything else on the island. Twenty thousand people are packed below the guest platform. And Troy, the roboticist, the man who builds minds, stands at the rail holding Sheba's hand, fully expecting to be bored.

He is not. What catches him is not quite the music. It is what the music does to the crowd: the way it takes a stadium of strangers and, for the length of a chorus, makes them one thing breathing together, all leaning the same direction at the same instant. And Troy, who cannot stop analyzing anything, thinks the sentence I most wanted him to think. In the book it reads: "this was a technology older than language and still the most powerful, and that no machine he had ever built could do it."

Sheba, without looking at him, tells him to stop analyzing it. He says he is not. She says he is doing the face. It is one of my favorite small exchanges in the book, because it is the two of them in miniature: the builder who cannot stop measuring, and the ethicist who keeps reminding him that some things are diminished by it.

Not a sentence but a sound

The singer, Ziba, grew up singing in a hard place, in a language most of that crowd does not speak. She never traded the thing that made her voice hers for the thing that would have made it easier to sell. So when she sings, even in the tongue nobody understands, people weep anyway, because meaning was never the point.

The narrator says she is making "the oldest argument there is for the worth of a human being, which is not a sentence but a sound." That line is the reason the chapter exists. I wanted the reader to feel, physically, before any of the book's debates begin, what it is that the rest of the story will put at risk.

Notice what the argument is not. It is not that her voice is technically superior. It is not that she can do something no machine will ever do; I am careful about that kind of claim, and so is the book. Machines can already make beautiful sound, and nobody knows what self-improving systems will be able to do. The argument is narrower and much harder to refute. Her voice is the specific sound of one particular person who had never existed before and would never exist again. It is not valuable because it wins. It is valuable because it is hers.

Why the builders needed to hear it

I wrote that scene for the four characters as much as for the reader. Every one of them works on a power that can remake a human being: genes, molecules, minds. They are about to spend the rest of the book being tempted by the idea that people can be improved, optimized, measured. A world that measures people the way it measures machines is the world the novel is afraid of.

So before any of that begins, I wanted them standing in a crowd, hearing something that cannot be measured that way. Not because it is beyond measurement in principle, but because measuring it misses what it is. You can score a voice for pitch and range. You cannot score it for being someone's.

Sheba feels the pull of the place in that moment and distrusts it, because she distrusts anything that makes her lean. She makes a private note that the island first reached her through her feelings and not her judgment. That note matters later. I will not say how.

What a machine is, and is not

I build artificial intelligence for a living, and I wrote the book with AI at the desk, openly. So I want to be precise about what this chapter claims. It does not claim that machines are empty and always will be. Today's AI wants nothing. It does not sing to be heard. But I do not pretend to know what self-improving systems will become capable of, and the novel takes that uncertainty seriously; it is a story about a superintelligence, after all.

What the chapter claims is that more capable is never more human. Power is not personhood. A machine might one day produce a sound that moves twenty thousand people, and it would still not be the sound of one particular person who was born, and grew up somewhere hard, and kept her voice when it would have been easier to sell it.

The whole trilogy turns on people, and on what a human being is worth. The concert is where I let the reader hear the answer first, before anyone tries to argue it, so that later, when the arguments come, you already know what they are arguing about.