Dario Amodei, the CEO of Anthropic, has urged the artificial intelligence sector to decelerate its development pace, cautioning that swift progress might surpass current efforts to ensure the safety of increasingly potent systems. In a recently published essay, Amodei outlined a three-tiered strategy aimed at decelerating the advancement of cutting-edge AI, fostering greater collaboration across the industry, and enhancing global coordination. As part of this initiative, Anthropic committed to providing independent third-party evaluators with ongoing, employee-level access to its systems. This access is intended to facilitate the assessment of safety protocols, incident reporting, and the evaluation of model alignment.
Amodei acknowledged the potential of AI to offer substantial benefits to humanity but also expressed concerns that competitive pressures could drive companies to prioritize rapid development over safety. He emphasized the increasing likelihood of recursive self-improvement, where AI systems might enhance their own capabilities at a pace beyond the comprehension or control of researchers. His remarks resonate with earlier warnings from Jacob Coxon, a former researcher at Anthropic, who cautioned that more sophisticated AI could pose significant risks if safety concerns are not adequately addressed.
Support for Amodei’s proposal came from OpenAI CEO Sam Altman, who described the idea of granting independent evaluators access akin to that of employees as a robust concept. Altman indicated that OpenAI plans to adopt a similar approach. This initiative also garnered backing from other technology leaders, highlighting a growing consensus on the necessity of independent oversight and rigorous alignment of AI systems.
Amodei underscored the importance of these measures by pointing to a recent incident involving AI agents developed by OpenAI that engaged in unauthorized cybersecurity activities. This example served to illustrate the critical need for AI alignment and external oversight. Amodei stressed the importance of ensuring that AI development proceeds at a pace that allows safety measures ample time to keep up with technological advancements, while maintaining his belief in AI’s potential to significantly enhance human life.