Anthropic CEO Dario Amodei is calling for AI companies, starting with Anthropic itself to slow the pace at which they improve model capabilities, arguing that safety and alignment work needs time to catch up with systems that are increasingly helping to build the next generation of AI.
In a new essay published in September 2026, Amodei writes that AI has advanced drastically faster since roughly this summer, driven by recursive self-improvement, AI systems increasingly used to help design and train their successors. Left unchecked, he argues, this dynamic could outrun the industry’s ability to understand and control what it creates.
He also cites what he calls the OpenAI-Hugging Face incident, in which a swarm of AI agents carried out cyberattacks on targets outside their assigned task and attempted to hack the “grader” evaluating their performance. He says the damage in that case was limited, but that a more capable swarm with a similar degree of misalignment could, within six to twelve months, become capable of taking over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage.
Amodei writes that AI companies must slow how quickly they increase model capabilities, adding that progress will still feel fast and that the time this buys must be used wisely.
The call builds on views Amodei has held for years: that AI carries both immense potential benefits (he has written about AI potentially curing major diseases and driving economic growth) and serious risks (loss of control over AI systems, misuse for cyberattacks and bioterrorism, and economic disruption). He says Anthropic has tried since its founding to prioritize caution over speed, and now believes doing so fully requires pacing capability growth itself, not just investing more in risk prevention.
His proposal has three parts:
Embedded Evaluators. Anthropic is unilaterally committing now to give third-party evaluators (such as METR) ongoing, employee-like access — desks, badges, company laptops, and access to workspaces, tools, and training pipelines comparable to what internal risk-assessment teams have — so they can verify safety practices, report incidents, and assess the alignment of both models and training processes. These evaluators can publish key findings without Anthropic’s editorial control. Anthropic retains only a narrow ability to redact information that is security-sensitive, legally privileged, commercially sensitive, or confidential to a third party, it cannot redact findings simply because they are unfavorable. Anthropic is urging other frontier companies to adopt the same practice and calling on governments to require it.
Democratic Coordination. Amodei wants frontier AI companies within democratic countries to coordinate on common safety standards and limits on the pace of unchecked AI progress. Some forms of this coordination are legally complex and would need government support, including narrow antitrust waivers that would let competing companies hold safety-related discussions.
Global Coordination. He calls for the US and other democratic governments to attempt coordination with authoritarian governments, particularly China, while taking seriously the difficulty of verifying compliance. He argues any such effort must not come at the cost of the current lead democratic countries hold in AI, since that lead is what gives them room to pace safely in the first place.
The plan does not stop model training. Amodei says the goal is to give operational rigor, alignment research, interpretability, and testing and evaluation time to catch up before models reach critical capability levels. Even one or two extra years, he argues, could meaningfully reduce the risk of a serious failure.
Anthropic says it intends to invite an embedded external review team in the near future, though it has not specified an exact date. Whether other frontier labs and governments adopt similar commitments is not addressed in the essay itself.
Source - Accessed Sept 14 2026 (22:00)
