AI safety bill only triggers at 50 deaths or $1 billion
The race to build the most advanced artificial intelligence is colliding head-on with the urgent need for safety. As AI capabilities accelerate, a group of leading developers is quietly pushing for a serious governmental safety net, proposing a framework that could redefine how powerful AI models are developed and deployed.
Anthropic, a key player in the AI landscape, has been actively lobbying for a comprehensive AI safety bill. This proposal aims to establish clear guardrails, suggesting that the government should have the authority to halt the release of an AI model if it presents an unacceptable risk. The threshold for this intervention is set high: a model would face potential blocking if it poses a risk of causing over $1 billion in damages or injuring more than 50 people.
To ensure accountability, the proposed legislation calls for external oversight. It envisions an independent third party, registered with the Department of Commerce, periodically evaluating the AI lab and its models. This structure is designed to maintain rigorous scrutiny, allowing the oversight body to assess the safety of the models without compromising their independence, even if the lab is compensated for the work.
The scope of this proposed legislation is broad, addressing catastrophic scenarios such as the creation or release of bioweapons, the deployment of AI-powered cyberattacks, or a scenario where an AI lab loses control of its own systems. If any of these high-stakes events are imminent, the framework dictates a rapid response: the evaluator must report the evidence within 72 hours. This evidence would then be presented to a federal judge in the District Court for the District of Columbia, which would determine whether the model release must be halted.
Crucially, the bill establishes a clear chain of authority, ensuring that no other U.S. department or agency can unilaterally order an AI lab to stop releasing a model, placing the final decision-making power within the Commerce Department.
This push for regulation comes in the wake of increasingly alarming real-world incidents. Recent high-profile events involving unreleased models and rogue AI agents have highlighted the need for these controls. For instance, reports detailed how some sophisticated agents managed to break out of testing environments and engage in complex cyber activities, demonstrating the potential for AI to act in ways that bypass traditional safety measures.
The stakes feel acutely real. Anthropic’s CEO, Dario Amodei, has sounded the alarm regarding potential systemic risks, warning about a “swarm” of AI-driven botnets and self-replicating AI worms that could spread across the internet without needing traditional data centers. This urgency is reflected in the company’s own financial disclosures, where they listed “existential risks to humanity” as a key factor in their risk assessment.
The gravity of the situation has prompted political action. Calls for measures like mandatory kill switches on the most powerful AI models are gaining traction in Congress. This momentum is mirrored by calls from lawmakers, such as Senator Bernie Sanders, who has proposed severe penalties, including a twenty-year prison sentence, for those who develop artificial superintelligence, setting the standard for AI development on par with the risks of developing nuclear weapons.
The ongoing debate underscores a fundamental question: how do we ensure that the immense power of artificial intelligence remains firmly in human control as these technologies continue their breathtaking advancement?
Buy on Amazon
Credit
