Who Decides Who Decides?
This post is part of my Medium blog.
The transcript answers it
This is a quick one. The New York Times Magazine put five people who think about AI rules for a living on one video call, moderated by Emily Bazelon, and published the transcript under the headline question: can the companies be trusted to write the rules for their own technology? The transcript answers it without meaning to.
First the setup, because it's a lot. Safety researchers have been resigning from OpenAI and Anthropic, on the record, warning the race has outrun the guardrails. The companies are reportedly gaming out what to do after a catastrophic cyberattack in the next six to twelve months. When the people building the thing start rehearsing the day after, the argument about whether this is serious is over. What's left is who's in charge when it happens.
The room
Alex Bores sponsored New York's AI safety law and helped found a group literally called Who Decides. Kate Brennan left Google and now runs policy work on AI and corporate power at a research institute. Tyler Cowen is an economist who has worked with Anthropic and OpenAI himself. Daniel Kokotajlo quit OpenAI and now runs a project that publishes forecasts of where this all ends up. Sneha Revanur's advocacy group organized a letter this summer, signed by more than a thousand AI company employees, asking to slow down.
Bazelon barely has to steer. Kokotajlo argues the catastrophe is the thing to regulate, now, before it's too late. Brennan argues catastrophe talk is the industry's favorite move: the more supernatural the technology sounds, the more natural it sounds for its makers to be the only ones qualified to oversee it. He asks her, human to human, whether that makes him a useful idiot for the companies. Twice. She won't say yes.
Cowen wants America to stay ahead of China and thinks Congress is in no shape to regulate something it doesn't use. Bores points out that Congress regulates air travel without a pilot's license in either chamber. It's the best two-sentence exchange in the piece.
What I'd call the main point
Here's where I land. The fear is no longer the fringe position — it's the premise. Researchers are gone, companies are rehearsing, a state legislator and a godfather of the field both treat disaster as a live possibility. And the room still can't agree on which fear counts. Is it the machine that slips the leash, the defective product on the market, or the Chinese model that gets there first? Depending on which one you pick, you end up with a completely different set of rules, so the room never gets to the rules. Five experts, one shared premise, zero shared conclusions.
A deadlocked room doesn't stay empty. Someone is always positioned with a draft ready. The companies are already courting Congress for the day after — that reporting came out earlier this month — working the members they'll need when something breaks and everyone reaches for a law off the shelf. The people with the most money on the outcome are quietly writing the response to their own failure. The headline asks whether they can be trusted, but I don't think trust is the issue here. Nobody has to trust you to let you write the rules. They just have to run out of other options, and a room that can't agree on anything tends to reach for whoever already has a draft.
The quiet part
One more detail, the quietest line in the piece. Bores says the industry is pushing to pre-empt state AI laws in the lame-duck session, and people are sleeping on it. The states are the only level of government that has passed actual AI safety law: California first, then his own New York. If Congress clears those off the board this winter, in a session nobody watches, the only governments that ever said no to this industry get overruled before the new members are even sworn in. You want to know who decides? Watch that vote.
In Essential, the country votes on which decisions have to stay with people. It took me a whole novel to stage that argument, and no spoilers, but the arguing doesn't stop when the ballots close. The roundtable got there faster. Same standing fight. Whoever you think should hold the pen — that's your politics now.
Yesterday I wrote about the number people in this world carry around for the end of everything. The transcript is what it sounds like when the people holding that number can't agree what it means.