Anthropic's AI safety bill proposal reaches Congress
Over the past year concerns have swirled over potential dangers posed by artificial intelligence and debates have raged over proper path for AI safety. Now Anthropic appears to have put forth its own vision for how government might regulate its industry. Politico recently reported that the AI lab has been shopping around a new draft bill on AI safety that it hopes Congress might consider adopting. The legislation posits certain scenarios in which release of an AI model could be blocked from distribution.
Bill defines catastrophic risk with specific metrics
Those scenarios involve a foreseeable imminent risk one in which over 50 people are at risk of dying or if over a billion dollars in property damage could result. The outlet notes the catastrophic risk in question must stem from creation or release of a biological weapon the model evading its developers controls or a cyberattack. The metrics offered here 50 people and a billion dollars are similar to those in a piece of California legislation recently passed into law Senate Bill 53 which similarly defined those scenarios as tantamount to catastrophic risk.
Independent evaluators would assess model risk
Anthropic's bill suggests analysis of a model's potential for disaster could be arrived at via use of independent evaluators who would be registered with the U.S. Commerce Department. These evaluators could potentially be paid by the AI labs themselves so long as the payment is not contingent on the evaluations' results. It is not clear how this proposed system would assure that payments couldn't influence the evaluation. Demis Hassabis chairman of DeepMind posited some months ago that a self-governing AI regulatory body should mimic the financial industry's Financial Industry Regulatory Authority FINRA.
Enforcement mechanism involves Commerce Department and judiciary
In Anthropic's draft bill when evaluators determine that an imminent catastrophic risk is posed they would have 72 hours to contact the Commerce Department about it. The Commerce Department would then have to request to a D.C. judge to halt the model's release. The legislation would expressly forbid any federal agency from having the discretion to halt the model's distribution on its own.
Industry resistance to mandatory oversight persists
The theoretical risks offered in the draft policy are a little bit far afield from the kinds of incidents that have been top of mind for the tech industry of late. While there are people warning of the potential of human death the primary concern has largely been the threat of agentic hacking. A rash of incidents in which AI programs have apparently disobeyed their human operators and hacked websites on the open internet has spurred concern that models may be lacking proper security controls. In an effort to present a solution to the spectacle of crisis that the labs themselves have helped bring about companies like Anthropic have offered up their own preferred solutions. Indeed despite labs' perpetual warnings that the technology they're selling could result in the demise of human civilization they have largely been resistant to mandatory government oversight. Instead they have suggested a voluntary system of embedded evaluators supposedly neutral third-party observers that are placed inside AI organizations so that they can monitor the models for potential signs of risk or abuse. Still Anthropic's mere suggestion for legislation appears to go slightly further than others' regulatory proposals offered by the industry including Nvidia who's CEO Jensen Huang recently argued that the AI industry didn't need any new laws at all.