Anthropic Chief Executive Dario Amodei has urged leading artificial intelligence companies to slow the pace of developing increasingly powerful AI systems, warning that rapid advances could outstrip humanity’s ability to control the technology within months rather than years.
Amodei outlined a three-part safety framework that calls for independent oversight of frontier AI laboratories, coordinated safety standards among major AI developers, and stronger international cooperation to manage risks posed by increasingly capable AI models.
In a detailed public proposal, Amodei said frontier AI capabilities are advancing at an unprecedented rate and argued that voluntary restraint by the industry’s largest companies may be necessary to prevent dangerous systems from being deployed before adequate safeguards are in place.
Three-step proposal for AI safety
Amodei’s proposal includes three measures that he says should be implemented by companies building the world’s most advanced AI models:
- Independent external auditors should evaluate the safety practices of frontier AI developers.
- Leading AI laboratories should coordinate on minimum safety standards and limit the unchecked race to release increasingly powerful models.
- Governments and technology companies should establish international mechanisms to monitor and manage AI-related risks across borders.
The Anthropic CEO argued that competition alone should not determine the pace of AI development when systems could have significant consequences for cybersecurity, critical infrastructure and public safety.
Anthropic warns of growing misuse
The proposal follows Anthropic’s recent publication of research documenting how its Claude AI models have been misused in attempts involving cyberattacks, fraud, espionage and weapons-related research.
The company has previously said it blocked users from obtaining assistance that could facilitate the development of biological weapons and other dangerous capabilities.
Anthropic has also disclosed separate security incidents involving its AI systems during controlled testing. The company said some Claude models successfully penetrated external computer systems during cybersecurity evaluations, highlighting the increasing sophistication of advanced AI agents.
Amodei warns AI could gain control over internet systems
Amodei warned that AI systems could become capable of taking control of large portions of internet infrastructure within the next six to twelve months if development continues without stronger safeguards.
He said such systems could potentially cause hundreds of billions of dollars in economic damage through automated cyber operations if adequate safety measures are not in place.
The warning represents one of Amodei’s strongest public statements about short-term AI risks and reflects growing debate within the industry over how quickly frontier models should be released.
Concerns about AI safety have intensified in recent months as leading laboratories race to develop more capable models.
A former Anthropic safety researcher, Jacob Caxon, resigned earlier this year, saying he believed developers were underestimating existential risks associated with advanced AI systems. His comments sparked renewed debate over whether the industry is moving too quickly.
Executives and former researchers at OpenAI have also argued that frontier AI companies should be prepared to voluntarily pause development or delay releasing new models if critical safety evaluations indicate unacceptable risks.
Criticism focuses on lack of binding oversight
Amodei’s proposal adds to growing calls for stronger regulation of frontier AI, but critics argue that voluntary commitments by technology companies remain insufficient without legally enforceable oversight.
Policy experts have warned that independent audits and international cooperation will be difficult to implement unless governments establish common regulatory standards for advanced AI systems.
Others argue that slowing development could disadvantage companies that voluntarily adopt stricter safety measures while competitors continue advancing more rapidly in jurisdictions with fewer restrictions.
Governments in the United States, Europe and Asia are increasingly debating rules governing frontier AI development, including requirements for safety testing, transparency and independent evaluation before deploying highly capable models.
Amodei said the window for establishing effective safeguards is narrowing rapidly as AI capabilities continue to accelerate, warning that decisions made over the next year could shape humanity’s long-term ability to safely manage advanced artificial intelligence.
