French President Emmanuel Macron hosted a meeting on AI safety on Monday, Nov. 9, with world leaders, AI company heads and researchers. Out of it came the Paris Call for AI Safety and Security, a pledge on handling the risks of frontier AI: the most capable AI models, ones that match or beat the best that exist today. By Nov. 11, the list of signers posted by the Élysée, the French presidency, named 16 governments: Austria, Bahrain, Canada, Croatia, Cyprus, Estonia, Finland, France, Ireland, Kenya, Latvia, Liechtenstein, Luxembourg, the Netherlands, Norway and Portugal. Macron wrote on X that AI is advancing very fast and brings real risks to collective security. The pledge says it builds on earlier efforts, including safety commitments AI companies made at a summit in Seoul and a June 17 meeting of world leaders and AI lab heads at the G7 in Évian, France.
The Élysée lists ten companies as signers: OpenAI, the maker of ChatGPT; Anthropic, the maker of Claude; Microsoft; France's Mistral AI; and Cohere, Hugging Face, Black Forest Labs, Sakana AI, Synthesia and Domyn. Seven research and policy groups signed too, including LawZero, led by the AI researcher Yoshua Bengio.
The companies that build the most advanced models make promises of their own. They commit to put in place fail-safe ways to stop models they control, such as limits on what a model can do, pauses or complete shutdowns. That covers models still being trained and models a company uses only inside its own walls, not just the products the public sees. The text doesn't say which of the signers count as such makers. OpenAI, one of the signers, already describes one such off switch: it says a system called Control Room, running across its research computers, halts a model's output within 35 seconds of a decision to stop or pause a task. The companies also promise to disclose what their models can do, where they fall short and what risks are known, and, where the risk calls for it, to support testing by qualified outside experts.
All the signers commit to build a common way to test and manage the risks of these models. It would focus on misuse in hacking and in chemical, biological, radiological and nuclear weapons, and on models acting more on their own or against their makers' intent, up to losing control of them. They agree to publish test results that aren't sensitive in a standard format, so they can be compared, and to set up a shared collection anyone can look at. They also back strengthening existing warning systems so that news of serious cyberattacks, or of an AI slipping out of its makers' control, reaches companies and governments quickly, and say it is worth exploring whether a joint way of cooperating would help, and what it would cover. Governments promise safe, confidential ways for whistleblowers, including company staff, to raise concerns about serious, widespread risks without fear of punishment at work or in court.
Not every country at the table signed. The Élysée says the leaders of Japan, Britain and Sierra Leone took part, along with the U.S. Treasury Secretary and representatives of the United Nations and the Organisation for Economic Co-operation and Development. None of those four countries is on the list of signers so far. Google, Meta and xAI, which also build leading AI models, aren't on it either. The Élysée says the pledge stays open to new signers and that its list is being updated.
Update (Nov. 11): the Élysée's list of signers has grown since the meeting. It now names 16 governments, up from eight on Nov. 10, and ten companies, with seven research and policy groups listed separately. The United States, Britain, Japan, Google and Meta are still not on it.
The pledge sets no deadlines and names no penalty for a signer that falls short. The signers commit to follow strict safety standards set or recognized by national authorities, under those authorities' oversight. The governments also recognize that laws need to make clear who is responsible when AI causes large-scale harm, including across borders. The testing rules are meant to build on existing work, including an international network for measuring and testing advanced AI, but the text doesn't say who will run the public collection of results, or when.