America's two leading AI companies are sounding the same alarm and, according to a news report, trying to shape how that alarm gets answered. The report, published Sunday, says the chief executives of Anthropic and OpenAI have declared that America's most advanced AI models are so powerful they are dangerous and need regulation plus independent testing before release. In a rare stretch of unity, they have laid out alarming scenarios in essays, social media posts and United Nations speeches. At stake is who writes the safety rules while the market stays largely unregulated.
Experts, analysts and former government evaluators told reporters that with those warnings the companies appear to be seeking public favor and to set the terms for their own safety protocols. Conrad Stosz, a former leader at the federal Center for AI Standards and Innovation (CAISI) who is now at Transluce and chairs the AI Evaluator Forum, told reporters the labs are not calling for more oversight from CAISI. Instead, he said, they vow to set their own audit criteria and choose particular evaluators to grade them.
PitchBook senior research analyst Harrison Rolfes said in an interview that the calls for caution look meant to win investor favor ahead of public stock offerings and the midterm elections, and that they can build a competitive moat, a defense against rivals, using safety as the reason. An Anthropic spokesperson told reporters the company has been calling for regulation for several years. OpenAI spokesperson Liz Bourgeois said safety starts with what companies like hers do themselves, noting that the company recently paused training of its most advanced models.
Sarah Shoker, who previously led OpenAI's geopolitics team, said in an interview that talk of AI threatening humanity's survival can push present harms, such as military use of AI, down the list of priorities. The report also notes that President Trump has dismissed talk of risk to humanity as a "HOAX" and shown little appetite for new AI rules, while CAISI still exists alongside independent evaluators such as METR.
It is not yet clear whether the evaluators the labs prefer will become the standard in practice, or whether Congress, CAISI or state lawmakers will take a harder line.
Update (Oct. 5): NIST now lists the Center for AI Standards and Innovation under a new name, the Center for Advancing Innovation and Standards for Super Intelligence (CAISSI).