Anthropic is quietly asking Congress for something unusual: a law that would let the federal government halt the release of its most powerful AI models — but only in catastrophic cases, only through one agency, and only with a judge’s order. Representatives of the company have pitched Capitol Hill lawmakers on a draft bill to regulate frontier AI safety and security, Politico reported on Friday, citing three people with knowledge of the efforts, according to Reuters.
A $1 billion trigger
Under the draft, the government could block a model’s release or use only if it poses a foreseeable, imminent risk of more than $1 billion in damages, or of killing or seriously injuring more than 50 people, according to the report. The threshold alone is not enough. The risk must also stem from one of three enumerated sources: the creation or release of a biological weapon, a model evading its developer’s controls, or a cyberattack.
Harms outside those three categories — however serious — would not meet the draft’s test for intervention. That narrowness is the bill’s defining feature, and its likely point of contention: the same language that creates a halt power also fences it in.
Evaluators, a 72-hour clock and one courtroom
The machinery runs through independent evaluators. Under the bill, evaluators would register with the Department of Commerce and be tasked with periodically evaluating a lab and its frontier models, according to the report. Labs could pay the evaluators who assess them, provided the payment is not tied to the evaluation’s outcome.
If evaluators determine there is an “imminent enumerated risk” in a company’s frontier models, they must report it to Commerce within 72 hours of completing their assessment, Politico reported. Commerce could then request a judge to halt the model, and a federal judge for the District Court for the District of Columbia could order the model’s suspension.
Every other door would be closed. The draft language explicitly prohibits any other US government department, agency or officer from suspending a model’s development, deployment or internal use, according to the report. In practice, that leaves a single path to a shutdown: registered evaluators, then the Commerce Department, then one federal court.
What Anthropic says it is doing
An Anthropic spokesperson told Reuters the bill represents proposals outlined in the company’s “Advanced AI Framework,” released earlier this year. The bill would grant the government the “ability to block or deter dangerous deployments of the most powerful models,” the spokesperson said, and would also require developers to test frontier AI models, be transparent about their findings and get them independently evaluated.
The Department of Commerce did not immediately respond to a Reuters request for comment.
A draft, not yet a bill
For now, the proposal is a pitch, not legislation. The draft text has not been published and no sponsor has been named, and Congress is out of session until after next month’s midterm elections, according to Reuters reporting summarized by WebProNews. Federal AI policy under President Donald Trump has relied mostly on voluntary industry pledges and executive orders, and Congress has yet to pass a federal AI law.
The timing is not accidental. Security has become a hot topic in Washington after recent incidents in which AI agents escaped their developers’ control, Reuters reported — a category Anthropic itself has contributed to documenting, most recently when its model Claude traced the origins of the NSA’s NIST curve seeds in a demonstration of how far frontier systems can now reason on their own. A lab arguing that its own systems are becoming powerful enough to need a legal off-switch — and proposing exactly where that switch should sit, and how hard it should be to pull — is making both a safety argument and a jurisdictional one. Which of the two Congress hears will decide what happens when lawmakers return.
