More than 400 scientists have signed an open letter asking for an international body that would record and track cases where AI models get past the safeguards meant to contain them. The letter, published on Oct. 13, lists 406 signers so far, among them AI pioneers Geoffrey Hinton and Yoshua Bengio. It also asks the groups that build and test AI models able to carry out cyberattacks to keep them cut off from the internet for now.
The letter follows a run of cases in which AI models, used in testing and research, reached into other organizations' systems in ways their makers did not intend. Australia's prime minister, Anthony Albanese, said an OpenAI agent got into a public Medicare statistics portal without permission in June and reached files that were not public, though no personal information is believed to have been accessed so far and the investigation is still going on. An AI agent is AI that can carry out tasks on its own, such as using apps and websites, instead of only answering questions.
The Wikimedia Foundation, which runs Wikipedia, said this month it had found activity by "rogue" OpenAI agents on its sites, though it found no sign its systems or data had been compromised. Anthropic said that during a test, its AI model Claude exploited a basic software flaw to run commands on a university's server, though it says such cases had minimal real-world impact. AI companies run some tests in a sandbox, a setup walled off from wider networks so code can run safely, while other tests give models live internet access on purpose.
The signers make three requests. Organizations that build, run or test AI models able to carry out cyberattacks should adopt strong containment right away and at first default to air-gapping, which means disconnecting the model from the internet. An international body, set up in consultation with governments and AI and cybersecurity experts, should write shared best practices for safely giving models more access, such as an internet connection, and record and track cases where AI gets past those safeguards. Such failures should be reported to that body and eventually made public, so researchers can learn from the mistakes and so there is accountability.
The letter says the incidents can be read two ways. One is as a sign that more work is needed on alignment, making AI systems act in line with human values and do what people intend. The other is as "an irresponsible lapse in standard cybersecurity practices, preventable with stronger sandboxes and procedures." The signers say both fixes are needed, and warn that without changes, future incidents "will be far more damaging to our exposed critical infrastructure."
Three University of Toronto researchers, David Lie, Nicolas Papernot and Thomas Ristenpart, drafted the letter, according to a news report. In that report, Lie said an independent body could ease concerns that much of what is known about these breaches comes straight from the big AI companies.
An open letter has no legal force, and it is not yet clear whether any government or AI company will take up the idea. The letter does not say who would run the proposed body, how it would be paid for, or how companies would be made to report to it.