California Governor Gavin Newsom has ordered the state to build a “kill switch” for artificial intelligence, directing agencies to move fast on two recent laws and convening experts to recommend new safety measures within two months.
Newsom’s executive order, issued Friday, mandates a kill switch for frontier AI models. The state is told to bring together a group of experts who will deliver recommendations on strengthening AI safety measures in state law. The group’s task includes considering how the state could require AI companies to put independent verification groups onsite for regular audits, and making transparency reports and risk assessments subject to independent auditor standards. The group must also create a kill switch that is routinely verified as effective, and companies would be required to report “loss-of-control incidents” like the OpenAI attack on Hugging Face as critical safety incidents.
The Kill Switch Mandate
The order directs a state agency to speed up implementation of two recent laws Newsom signed: a framework for independent verifiers assessing AI safety and a state registry of AI auditors. The timeline is tight. The expert group must produce its recommendations within two months.
The kill switch requirement is notable because it targets the worst-case scenario directly. The OpenAI attack on Hugging Face is cited as an example of what this kind of incident looks like.
Why California Is Moving First
Congress is unlikely to act before the midterms, leaving California to position itself as the leader on AI oversight. The state has already moved faster than Washington on this front, signing the two recent laws that now need to be implemented.
Newsom’s push is partly about leadership. He wants the state to set the standard that the federal government has failed to meet. His order notes that the federal government has produced “no form of meaningful AI oversight or accountability,” and he is calling on Congress and President Donald Trump “to review and adopt the state’s framework, or use its nation-leading regulation as a floor, not a ceiling, for the benefit and safety of all Americans.”
Trump Responds With Skepticism
President Donald Trump responded to the order, saying “The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” That response suggests the federal government sees the problem differently. Trump’s framing puts the responsibility for AI safety on the presidency itself rather than on regulatory frameworks.
Newsom has a different view. In a statement, he said “the federal government’s abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation.” He added: “We’re going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action.”
The Expert Group’s Job
The expert group is tasked with considering how the state could require AI companies to embed independent verification groups onsite for regular audits. Those audits would check whether the kill switch actually works as advertised.
The group must also consider making transparency reports and risk assessments subject to independent auditor standards. The goal is to ensure that companies are not simply reporting what they want others to hear.
What This Means for Companies
Companies operating in California will now face new reporting requirements. Loss-of-control incidents must be reported as critical safety incidents, and the state will be watching closely. The order also directs the state to convene a group of experts delivering recommendations within two months on strengthening AI safety measures in state law.
The order directs a state agency to speed up implementation of two recent laws Newsom signed: a framework for independent verifiers assessing AI safety and a state registry of AI auditors. The timeline is tight. The expert group must produce its recommendations within two months.
The Bigger Picture
Newsom’s order is a response to a growing sense of urgency about AI safety. The former Anthropic researcher Jacob Coxon posted warning of more than a 10 percent chance that AI “could kill all humans” within a decade. That kind of warning is not abstract.
California is moving first because the federal government is not. Newsom is positioning the state as a model for federal action.
The Key Facts
- Executive order: Issued Friday by California Gov. Gavin Newsom (D)
- Kill switch: Mandated for frontier AI models
- Expert group: Required to deliver recommendations within two months
- Two recent laws: A framework for independent verifiers assessing AI safety and a state registry of AI auditors
- Loss-of-control incident examples: The OpenAI attack on Hugging Face
- Trump’s response: “The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT”
The order is a statement of intent. It tells companies that California is serious about AI oversight, and it tells the federal government that the state is willing to lead. Whether the federal government follows remains to be seen. For now, California is setting its own course.
Where the paper stands
The paper backs narrow rules against direct harm, like forcing companies to disclose safety failures they hid, and is against a kill switch that hands the market to the incumbents. The order asks state agencies to implement two recent laws quickly and to build a kill switch for frontier AI models, with companies reporting loss-of-control incidents as critical safety incidents. These are reasonable guardrails. But the kill switch requirement and the quick timeline risk locking in the current leaders rather than opening the field to challengers.
The danger is big tech dominance, not the technology itself. Licensing regimes and compliance costs only giants can afford are a moat, not a safeguard. The expert group’s task of requiring independent verification groups onsite for regular audits, and making transparency reports and risk assessments subject to independent auditor standards, will raise the cost of entry for smaller firms. That is the pattern the paper warns against: regulation that protects the biggest players and raises the cost of entry for small ones.
The paper would want the rules narrow and aimed at actual harm, with disclosure requirements attached to safety failures companies have hidden. It would oppose a broad rulebook that freezes today’s leaders in place and locks out whoever would challenge them. The reader should watch for signs that the order’s requirements are being written by the biggest firms themselves, and that the costs of compliance fall hardest on the small players.
Source material: “Gavin Newsom is pushing for an AI kill switch,” The Verge.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.