AI rivals found rare agreement on safety. Putting it into practice is harder

Dario Amodei [AP Photo]

The leading AI industry voices rarely agree — so when they do, it’s no wonder it captures attention.

Anthropic’s Dario Amodei, OpenAI’s Sam Altman, SpaceXAI’s Elon Musk and other prominent industry leaders have said in recent days that AI’s development needs to slow down. The consensus came after an Anthropic researcher resigned with a dire warning about the threats the technology poses to humanity.

But obstacles including domestic and global competition, profit motivations and pushback from President Donald Trump all stand in the way of a coordinated effort to pace AI’s development.

Still, this moment could force the industry to wrestle with a “reframing of what we consider successful,” said Nick Reese, former director of emerging technology policy at the Department of Homeland Security and an adjunct professor at New York University.

“What if we considered success in AI the same way that we consider success for airlines? On time and haven’t had a crash in who knows how long,” Reese said. “We don’t tolerate commercial airline crashes, but we do tolerate significant AI failures.”

Here’s what tech leaders want

In theory, the leading AI startups could slow the progress of their models at any time. But leaps in the technology have partly sprung from competition to build even more powerful models, all with the blessing of a White House that doesn’t want the U.S. to lose its edge over China. Anthropic and OpenAI also are racing toward potentially record-breaking public offerings on Wall Street, although the latter said it’s holding off on going public this year while it continues its safety efforts.

Sm Altman [AP Photo]

That’s why the proposals floated by industry depend upon mutual collaboration and, to an extent, government regulation.

Amodei recently published a lengthy essay in which he outlined a plan for slowing advanced AI development.

One proposal calls for companies at the forefront of AI, often called frontier labs, to commit to giving “ongoing, employee-like access” to independent outside evaluators. Evaluator teams would have offices, access badges and company laptops so they can monitor safety practices.

Anthropic is “unilaterally committing” to this step now, Amodei wrote. Altman called it a “great idea” on social media and said OpenAI will follow suit.

Altman said he welcomed a “federal framework” setting out safety standards for advanced AI, but said the company didn’t need to wait for legislation or an antitrust exemption before addressing AI safety.

Amodei’s essay also called for government regulation and intervention, including coordination between frontier AI companies with the help of the U.S. government and the governments of other democratic countries. He said he hopes the leading labs could establish “common safety standards as well as limits on the rate of unchecked AI progress.”

Amodei’s plan also includes the U.S. and other countries working to coordinate with “authoritarian governments.” He acknowledged the difficulty, specifically citing challenges in getting cooperation from China.

Elon Musk [AP Photo]

The Anthropic head outlined several levels of agreement that global leaders could sign off on. The most feasible, he said, would be an agreement banning “certain narrow and obviously dangerous uses of AI” like producing biological weapons or allowing users to do so. More difficult would be getting the U.S. and its adversaries to agree to test models before release, possibly through a global standards body, for risks in areas like cybersecurity or biological threats.

Still harder would be to set a “speed limit” on AI models that can develop improved versions of themselves on their own, known as recursive self-improvement.

“Slowing the rate from ‘extremely fast’ to ‘only somewhat fast’ gives up relatively little strategic advantage, while potentially greatly improving safety,” Amodei wrote.

He drew an analogy to Cold War-era treaties in which “capping the number of missiles limited the potential for destruction while preserving each country’s deterrent.”

Amodei said the most difficult proposal to implement would be a “full pacing, or even ‘pause,’” whereby participating governments agree to substantially limit the overall rate of AI development.

Writing on social media, Altman said the “pacing” of AI development does not mean stopping. AI progress will continue to be rapid, Altman said on X. “But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs.”

“Pacing will be well worth this cost,” he said, adding that “no amount of American competitive pressure should justify recklessness.”

Questions swirl about AI slowdown’s feasibility

While experts said the sense of agreement across the industry is promising, there are many unanswered questions.

A slowdown is possible “in theory,” said Sandra Wachter, professor of technology and regulation at the Oxford Internet Institute.

But in practice, “it would require substantial coordination and cooperation between companies and governments across countries,” something she deems “highly unrealistic” due to “the current state of the world.”

Options for promoting a slowdown, she said, include governments doing more to hold AI companies responsible for the risks posed by their technology and pursuing regulation that limits their access to electricity, water and other resources that power data centers.

The idea of embedding external evaluators inside companies has been mostly well-received by the industry, but some critics questioned how independent the assessors could be and how the evaluation standards would be drawn up. The result could be even more concentration of power, Aidan Gomez, co-founder and CEO of the Canadian AI lab Cohere, wrote in response to Amodei’s essay.

“A handful of the most powerful labs based in one country would agree on shared standards and the limits to how fast the technology should advance,” Gomez said.

He also pushed back against Amodei’s idea to have the U.S. government give frontier labs waivers to bypass antitrust restrictions.

“If this is the most consequential technology in human history, then the rules for it cannot be written by a small group of commercially aligned companies behind an antitrust waiver,” Gomez said.

Elham Tabassi, the director of the AI and Emerging Technology Initiative and senior fellow at the Brookings Institution, said until the measures companies are committing to are solidified and made public, having independent auditors would be a “voluntarily-provided, company-controlled” move.

Tabassi said it’s crucial AI companies continue to invest in developing “scientifically valid” ways of testing and measuring their models as they aim for increased safety. Without this, regulation won’t hold as much power, she said.

Pressure to beat China stand in the way

The Trump administration has long favored a light-touch approach to the AI industry in hopes that innovation would flourish and that AI development in the U.S. would surpass China.

The competition between American and Chinese AI companies has been likened to an arms race, which experts said is not conducive to collaboration. That creates “conditions where now we have this existential adversary,” Reese said.

Still, some industry leaders remain “cautiously hopeful” about AI development moving in a more safety-minded direction, said Zahra Timsah, the co-founder and CEO of governance platform i-GENTIC AI.

“Within days, you have CEOs that are competing — all of them, all of a sudden, found common ground. You have senators from both parties that started to advance different forms of oversight,” Timsah said.

“It’s receiving attention, but attention is not the same as implementation. Will it really work? I don’t know.” KAITLYN HUAMANI & KELVIN CHAN, Technology Writers, MDT/AP