Anthropic, the company behind the Claude chatbot, said on Thursday, Oct. 15, that it has started giving a team from METR, a nonprofit that tests the most capable AI models, what it calls "employee-like access." Six METR employees now have company hardware, office space and access to internal tools and systems that Anthropic says are roughly on par with what its own risk assessment teams use. Anthropic says the aim is to let outsiders judge the risks of its AI without it having to publish information that could help people misuse it, or that it considers commercially sensitive, such as details of how its models are trained.
Anthropic compares the setup to bank supervision, where regulators place examiners inside the largest banks because banks know more about their own risks than outsiders do, though it says the comparison isn't exact. Anthropic says its own staff are encouraged to share information with the METR team as they would with a colleague, except in a few set areas, such as customer data, legally privileged information and details of how its models are designed.
Under the deal, Anthropic says, METR will publish its analysis of Anthropic's risk levels, incidents and practices. Anthropic says it cannot veto publication, but it can require METR to remove information in narrow categories set in advance, such as customer data. METR may describe the removals publicly and say whether it thinks they were justified, as long as it does not reveal what was taken out. Other METR employees, including researchers who work with Anthropic on other projects, are kept apart from the team's work.
Anthropic has also signed the White House Accord on Super Intelligence, in which AI companies pledged to work with independent outside auditors or evaluators, and last month it promised that outside groups would assess its models and safety practices more deeply. It is also working with SecureBio, a nonprofit focused on biological threats, which is reviewing the chemical and biological weapons chapter of the risk report Anthropic published in August on the dangers of its models. Anthropic is also working toward a partnership with Accenture and its AI business Faculty, and is in early talks with several other groups, including Hugging Face.
Because those partnerships are new, Anthropic says, it gives those groups less information and narrower rights to publish. It is setting up three levels of access. At the first, where SecureBio sits, Anthropic decides what the group can see, and findings need its approval to be published. At the second, groups see more and can publish without approval, though Anthropic can still require some details to be taken out. METR is at the third level, with staff-like access and publishing rights similar to the second.
A news report earlier this week said Anthropic and OpenAI had sought significant control over what outside reviewers could publish, worried that findings could leak trade secrets. A news report on Thursday, citing a person familiar with the talks, said Anthropic had recently become more open in those negotiations. Experts have questioned how independent the reviewers can be when the companies choose them. Anthropic says an evaluator should hold no shares in the company it checks and should not depend on it for money or operations. METR is not new to Anthropic: it has reviewed several of the company's risk reports over the past year. Anthropic says it is doing this voluntarily and that every company building the most capable AI should do the same. It is not yet clear when METR's first report will appear, how much will be taken out of it, or whether other AI companies will follow.