Dystopia Is a Current Event
This post is part of my Medium blog.
I was at a conference in Berkeley last night. Except conference is the wrong word. An unconference — a room full of smart people deciding on the spot what they wanted to talk about, no agenda handed down from anywhere.
At some point a guy started telling me about philosophical benchmarks for language models. He didn't mean accuracy scores. He meant putting a model in front of an ethical framework and measuring what comes back. What the system believes. Where it leans. Whether it applies a principle or just tells you what you want to hear. Whether you can characterize the moral commitments of a machine before you hand it decisions that matter.
Toward the end, he asked the question you eventually get at these things.
So what's your view of a potential dystopian reality?
I had to laugh. Funny you should ask. I've written four novels about potential dystopian futures.
Mine has a date on it. The Condition Set is a trilogy about what happens when AI quietly takes over decisions that used to belong to people — who gets cut, who gets used, what must stay human. In Essential, the last book, the argument about AI has become a public vote — the country decides which decisions have to remain human. I set it in 2033, which felt like a polite distance when I wrote it. There's a real election in ten days, and I don't have to imagine the mood around it — the synthetic ads, the labor fights, the models quietly making calls nobody remembers approving. Dystopia isn't a thought experiment this month. It's a current event. If you want to read more about it, I have books. Three are out. The fourth is still with me.
But here's what I keep turning over. Why does it feel like the people asking these questions are the ones who don't work for the model companies? It wasn't just him. All night, in different corners of the room, people kept arriving at the same two questions. Aren't there better ways to measure these models? Aren't there better ways to characterize their ethical leanings? The research outside the industry is asking it too — the papers on moral evaluation of language models are stacking up, and some of that work is good.
Meanwhile the industry itself is focused on moving as fast as possible. Shipping, scaling, racing. The urgency about what these systems actually are — what they lean toward, what they'll do in a hard moment — is coming from outside. Regulators, academics, philosophers, novelists, people at unconferences holding paper plates. The industry has the speed. Everyone else has the stakes. That's backwards.
Frankly, this is a case for regulation right now. Not regulation of the output — I don't mean rules about what a model is allowed to say. Regulation of the measurement. Before one of these systems gets embedded in hiring, lending, medicine — decisions people can't opt out of — somebody should be able to say what it leans toward and how it reasons. There's good research out there on how to do this. It should be front and center, not a side conversation at an unconference.
I'm going back to the conference today.