You Can't Outsource Public Safety
Two weeks ago, I was talking with someone who considers frontier models a military-grade munition. You don't have to accept that description literally to see why the companies building them shouldn't be the only ones deciding how they are controlled.
That conversation came back to me when Carnegie Mellon professor Costa Samaras posted on Bluesky: "AI safety is too important to leave to the AI safety community." In a follow-up, he called for people who can keep infrastructure and government running, protect public safety, and help regulation keep pace with industry.
Samaras is calling for broader expertise. My conclusion goes further: government needs the authority to enforce limits on these systems. That's my argument, not a policy program he laid out in that post.
Read the safety report
Anthropic's May 2025 deployment safeguards report describes protections against models helping people develop chemical, biological, radiological, or nuclear weapons. It acknowledges gaps and uncertainty. This is the supplier's assessment, not an independent certification.
Outside assurance isn't automatically better. In METR's November report, Anthropic redacted most of the evidence the researchers found most informative. METR says it takes no AI-company funding; the money problem is a separate example. Anthropic calls its Accenture partnership "independent evaluation" while directly funding Accenture's work. We're asked to trust conclusions without seeing the evidence, or accept "independent" without financial independence. As public accountability, it doesn't work.
Government needs limits too
The military connection is already concrete. An October briefing by Kelley Sayler at the Congressional Research Service, Congress's nonpartisan research staff, documents the Pentagon's July 2025 contracts with Anthropic, Google, OpenAI, and xAI, each with a ceiling of $200 million.
The same briefing describes Anthropic's refusal to permit mass domestic surveillance and fully autonomous weapons. It says the Pentagon is not publicly known to be using frontier models in autonomous weapon systems.
My position is that AI should not be used for lethal applications, including selecting targets or deciding when to fire. Having a person approve a machine's recommendation doesn't settle the question of who actually made the decision to kill.
Government involvement must mean more than purchasing. A defense department wants useful capabilities; protecting the public requires scrutiny of the department as well as its suppliers.
That's a concern I write about in Contingent: official approval can help create a disaster when the people granting it waive the protections they're supposed to enforce.
I want public agencies with their own technical expertise, mandatory reporting of serious failures, and enforceable limits that bind public buyers too. Before a high-risk system goes into service, someone outside the purchase should have inspected it and have the legal power to stop it.