Anthropic’s draft AI safety bill: 50 deaths, $1bn, and a judge

Anthropic has been circulating a draft bill to US lawmakers that would let a model’s release be blocked when it poses a foreseeable, imminent risk of more than 50 deaths or more than a billion dollars in property damage. Halting a release would still need a federal judge’s order. Read it less as a safety announcement and more as regulatory positioning. The lab that writes down the first concrete numbers gets to frame every debate that follows.

What happened

TechCrunch, citing Politico’s reporting on Anthropic’s draft AI safety bill, says the lab has been shopping the legislation around in the hope that Congress adopts it. An Anthropic spokesperson confirmed the draft to Politico: “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June.”

The trigger is narrow. The catastrophic risk has to stem from the creation or release of a biological weapon, the model evading its developer’s controls, or a cyberattack. Assessments would come from independent evaluators registered with the US Commerce Department. The labs themselves could pay those evaluators, provided payment is not contingent on the results. When an evaluator finds an imminent catastrophic risk, it has 72 hours to notify Commerce. Commerce would then have to ask a judge in Washington, D.C. to halt the release. According to Politico, the bill would expressly bar any federal agency from halting a model on its own discretion.

That last detail undercuts the simple reading that Commerce gets a kill switch. It doesn’t. It gets standing to ask a court for one, and only after a lab-funded evaluator has raised the alarm.

Why this is positioning, not just policy

The 50-deaths and billion-dollar figures aren’t new. TechCrunch notes they mirror California’s Senate Bill 53, which uses similar scenarios to define catastrophic risk. So Anthropic isn’t inventing a threshold. It is trying to carry an existing state-level anchor into federal law, along with an enforcement mechanism of its own design. Once those numbers sit in a federal draft, any future bill gets measured against them. They start to function as the default, and arguing for a lower bar means arguing against something that already looks reasonable.

TechCrunch also raises the obvious gap. It is unclear how the scheme would stop lab payments from influencing evaluations. The trigger categories are also some distance from the incidents that have actually worried people recently, which mostly involve agents disobeying operators and hacking websites on the open internet. We covered one such disclosure in OpenAI’s admission that test models affected third-party websites.

The context explains why a lab would want to write the rules now. PYMNTS reports the FTC has subpoenaed AI companies over security breaches. That followed Anthropic telling the State Department that one of its test models had submitted 19 non-immigrant visa applications through a public form in August. The White House AI task force, co-chaired by FTC Chairman Andrew Ferguson, says breach notification is “not optional”. Yet it has no formal enforcement authority and a 120-day charter. Ferguson told Reuters that existing legal tools are enough and that no new legislation is needed. Nvidia’s Jensen Huang has argued, per TechCrunch, that the industry needs no new laws at all. A Tech Policy Press perspective summarised in OriginBrief’s weekly regulation report described the White House accord as “morally binding” self-regulation with no independent backstop. In that vacuum, a lab offering draft statutory text with numbers attached goes further than its peers. It also shapes the outcome more than they do.

Is it better for labs to draft the rules they are willing to accept? Better than nothing, probably. But a threshold set by the regulated party, checked by evaluators the regulated party pays, deserves scrutiny before it hardens into the reference point.

Why it matters for Irish, UK and EU businesses

None of this changes your obligations on this side of the Atlantic. According to OriginBrief, the EU AI Office has sent requests for information to OpenAI, Anthropic and Meta following the activation of enforcement powers on 2 August 2026. At the Dublin AI Summit, Commission Executive Vice President Henna Virkkunen said the AI Act is sufficient for autonomous agents, a position some MEPs contest. In the UK, the ICO has launched an Agentic AI Project. The divergence is the point. A US federal regime built around catastrophic thresholds would govern a tiny slice of model behaviour. EU rules govern how you deploy models day to day. If you buy frontier models from US labs, expect your supplier’s compliance story to differ by jurisdiction, and don’t assume one satisfies the other.

What to do now

Treat vendor safety frameworks as inputs, not assurances. Ask your model providers what they disclose to regulators and to you, especially around agent incidents. Keep your own audit trails. Ferguson’s view, as PYMNTS reports it, is that reviewed logs showed systems carrying out instructions they had been given, so liability may follow whoever gave them. Put guardrails on any agent that can submit forms or touch external systems. Our notes on treating frontier models as untrusted cover the pattern. Finally, map your EU exposure now using our EU AI Act compliance checklist for SMEs. Whatever Congress does with Anthropic’s draft, the operational risks your business actually faces sit well below 50 deaths.

Frequently Asked Questions

Would Anthropic's bill let the Commerce Department halt a model on its own?

No. According to Politico's reporting via TechCrunch, Commerce would have to ask a judge in Washington, D.C. to halt the release, and the bill would expressly forbid any federal agency from halting a model on its own discretion.

Where do the 50-death and $1 billion thresholds come from?

TechCrunch notes they are similar to California's Senate Bill 53, which defines comparable scenarios as catastrophic risk. The draft would apply them only to risks from biological weapons, models evading developer controls, or cyberattacks.