Anthropic CEO Dario Amodei has proposed a plan to slow the pace of AI development, but faces a significant challenge: ensuring global cooperation, particularly with China.
Amodei’s essay suggests a three-step approach, emphasizing collaboration among democratic nations and eventual coordination with authoritarian regimes, though he acknowledges the difficulty of achieving the latter.
This initiative comes amidst growing concerns about AI safety, public backlash over data centers, and intensifying competition from Chinese AI startups.
- Anthropic CEO Dario Amodei identified the potential non-participation of China as the most significant challenge to his proposed AI advancement slowdown.
- Amodei's recent essay advocated for a pause in the rapid development of cutting-edge AI models.
- This proposal emerges amid increasing scrutiny of the AI industry, including public concerns about data centers and warnings from researchers about existential threats.
Anthropic CEO Dario Amodei stated on Sunday that the most formidable challenge to his proposal for slowing AI development is the potential refusal of geopolitical rivals, specifically China, to adhere to such a slowdown.
"The more long-term thing would be working together to put a speed limit on the rate of of AI progress," Amodei told CBS News' "Sunday Morning." "I think that's going to be very difficult because the incentives to pull ahead and the military advantage that you get from that are so large. And honestly, I don't know if it's possible, but we should we should try."
Amodei published an essay on Saturday, urging AI firms to decelerate the pace at which they enhance their most sophisticated models. This call follows stark warnings from industry researchers last week regarding the technology's escalating potential for catastrophic harm. A former Anthropic researcher, Jacob Coxon, resigned due to concerns that companies like Anthropic and its main competitor, OpenAI, are "gambling with our lives."
In recent months, AI models developed by Chinese startups have demonstrated significant progress, challenging leading offerings from American companies such as OpenAI and Anthropic. This surge has raised concerns among lawmakers and experts in Silicon Valley.
Chinese President Xi Jinping, speaking at the BRICS summit in India on Sunday, advocated for a "consensus-based global AI governance framework" and pledged China's commitment to establishing a BRICS AI open-source community. The BRICS bloc, founded in 2009, now comprises 11 nations, including China, India, Russia, Iran, and the United Arab Emirates.
President Donald Trump is scheduled to meet with Xi at the White House on September 24th, with discussions anticipated to include artificial intelligence.
Amodei's essay comes at a critical juncture for the AI industry, which is currently facing public backlash concerning its data centers and increasing demands for regulation from Washington. Both Anthropic and OpenAI are reportedly preparing for potentially landmark IPOs, although neither has announced a definitive date. OpenAI CEO Sam Altman indicated on Saturday that an IPO this year is unlikely, partly due to ongoing safety considerations.
"Right now would be an ill-advised moment to go public," Altman stated in an interview with Fortune.
Despite a historically strained relationship with Amodei, who previously left OpenAI to establish Anthropic, Altman voiced his support for Amodei's essay in a post on X on Saturday, agreeing with the necessity to "pace the frontier."
Several other prominent figures in the tech industry, including Demis Hassabis, CEO of Google DeepMind, and Elon Musk, CEO of Tesla and SpaceX, also endorsed Amodei's proposal.
"Dario's essay points towards the right path forward," Hassabis commented on X on Saturday. "The details need working through, but the direction is correct for meeting this critical moment."
Amodei's essay outlines a three-phase strategy designed to temper the pace of AI development without compromising commercial advantages or the United States' leading position in the field.
Anthropic has reportedly taken the first step unilaterally, granting third-party evaluators access to company personnel and systems to assess safety protocols and report issues. The second phase advocates for collaboration among leading AI companies in democratic nations to establish shared safety standards. The third phase calls for cooperation between democratic and authoritarian governments.
"Dario is right that there should be some oversight," Musk stated on X early Sunday morning. "Peer review of AI by competitors is the right way to start this off."
Musk, a long-time AI risk skeptic, was notably quick to support Amodei's proposal on Saturday. Despite previous criticism of Anthropic, his company SpaceX entered into a significant compute agreement with Anthropic in May, following SpaceX's acquisition of his AI startup xAI. Under this agreement, Anthropic is set to pay SpaceX $1.25 billion monthly through May 2029.
"Everyone I met was highly competent and cared a great deal about doing the right thing," Musk remarked post-agreement. "No one set off my evil detector."
Alexandr Wang, Meta's AI chief, who joined Meta through a $14 billion investment in his startup Scale AI last year, has not yet directly commented on Amodei's essay. He did, however, share a post on X on Saturday stating that Meta Superintelligence Labs, the company's AI research division, is "rapidly scaling up the share of our efforts that goes into alignment as our models become more powerful." In the AI field, alignment refers to efforts ensuring AI systems operate in accordance with human values and intentions.
"We do believe alignment can be the gating factor for scaling as we get closer to the frontier," Wang added.
David Sacks, a known critic of Anthropic and former AI and crypto czar under Trump, also offered his perspective on Amodei's essay on Saturday. He stated on X that if OpenAI and Anthropic genuinely perceive their models as a significant risk warranting a slowdown, he supports their "decision to be responsible."
However, Sacks advised both companies to "stop pretending you need anyone else's permission" and suggested their motivations might not be purely altruistic. He argued that they face substantial product-liability risks if their AI enables harmful cyberattacks, making it "simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability."
"So go ahead and pace the frontier," Sacks concluded. "You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it."
WATCH: Seems like Anthropic will beat OpenAI to IPO, says FirstMark's Rick Heitzmann
