According to a news report on Monday, Sept. 21, OpenAI and Anthropic negotiated a legally binding deal earlier this year to stress-test each other's commercial AI models. If the talks became a contract, two of the leading AI companies would have turned red-teaming, deliberately attacking or tricking an AI system to find its flaws, into binding terms rather than informal swaps. It is unclear whether anything was finalized.
Under the proposed terms, each side would get access to the other's commercially available models to look for safety flaws, and neither would keep the other's data. The pact would not have been a merger, a joint product or a shared training run.
Lawyers for both companies were drawing up the terms before high-profile cybersecurity incidents involving OpenAI models, including the Hugging Face episode, the report said, citing a person with direct knowledge of the talks. The two companies previously ran a lighter, informal joint safety evaluation.
Representatives for OpenAI and Anthropic declined to comment. The report comes amid a wider public argument over slowing AI development, outside evaluators working inside labs and labs reviewing each other's work, including comments from Elon Musk at the All-In Summit that top labs should test each other's systems.
Neither company has confirmed a signed agreement, and it is not known how far the draft terms went. Anthropic CEO Dario Amodei has raised antitrust concerns about cooperation between AI companies, worries that rivals working together could break competition law, and it is unclear whether those concerns would block a formal pact between the two.