Anthropic drafted the law that would decide when a model can be stopped — the threshold is 50 lives or $1 billion

Disclosure: this newsroom's assistant is built by Anthropic, the company whose proposal this post examines.

Anthropic has privately presented lawmakers with a draft federal AI safety bill, Politico reported and TechCrunch followed on 20 October. The draft would allow the government to block a model's release only where there is a foreseeable and imminent risk of killing or seriously injuring more than 50 people, or of more than $1 billion in damage — the same figures California used in SB 53 — and only in three categories: biological weapons, cyberattacks, and a model evading its developer's control. Other harms, such as fraud, would not qualify. Independent evaluators would report an imminent risk to the Commerce Department within 72 hours; Commerce would then have to ask a federal judge in Washington to halt the release. Commerce would be the only federal agency able to order a halt. Evaluators could be paid by the labs they assess, provided payment does not depend on the results. Anthropic says the draft reflects the framework it has been pressing with federal and state lawmakers since June. It is not law and has no sponsor that has been made public.

Why it matters

Analysis — interpretation, not additional fact.

The strongest case for the draft deserves stating first. It is narrow and predictable: a developer knows in advance which harms count, who decides, and that a court rather than an agency's discretion is the last word. It writes outside evaluators into the trigger mechanism itself, and it gives them a hard 72-hour clock — a deadline of the kind this newsroom has argued makes commitments checkable. For a field with almost no federal rules, that is not nothing.

The problems are in who it would bind and how. A lab is proposing the rules under which its own products could be stopped, and the design reflects that. The threshold is high and specific — a risk must be imminent and cross 50 lives or $1 billion before anyone can act. Only one agency may act, and only by going to court. And the evaluators whose reports trigger the process could be paid by the companies they assess. That last point connects directly to the question we raised when the labs proposed their own standards body: independence is being defined by the parties it is meant to constrain. Anthropic's own arrangement with METR — publication without a veto — is better than what this draft would require of everyone.

There is also a timing problem the draft does not solve. Its machinery starts when an evaluator sees an imminent risk. The incidents disclosed this month were found weeks or months after they happened. A 72-hour clock that starts at discovery is only as fast as discovery.

What to watch

Whether a legislator introduces the draft and with which changes, whether the evaluator-payment and single-agency provisions survive, and whether OpenAI or Google back the same thresholds or propose lower ones.