A federal appeals court has ruled that the Pentagon acted within the law when it labeled Anthropic, the company behind the Claude AI models, a risk to its supply chain and ordered Claude removed from its systems. In a 2-1 decision on Sept. 25, the U.S. Court of Appeals for the D.C. Circuit turned down Anthropic's challenge. On Oct. 6, a Pentagon spokesperson said the department "has ceased the use of Anthropic products," according to a news report.
The fight was over two limits. Anthropic's terms barred the use of Claude for lethal autonomous warfare and for mass surveillance of Americans, and the Pentagon, which now calls itself the Department of War, wanted permission for "all lawful uses." Anthropic's chief executive, Dario Amodei, has said fully autonomous weapons, which select and strike targets with no person involved, may one day prove critical for national defense, but that today's AI is not "reliable enough" to run them. After Anthropic refused on Feb. 26, Secretary of War Pete Hegseth turned to a 2018 law that lets an agency shut out a supplier when it finds in writing that this is needed to protect national security and that no gentler step would do. The law describes the risk as the chance that any person could sabotage the technology, maliciously plant unwanted functions in it, pull data out of it or "otherwise manipulate" it so as to spy on, block or disrupt how it works.
The case turned on that word, "manipulate." Judges Katsas and Rao, the majority, said the Pentagon had good reason to see a risk, because Anthropic builds its rules into Claude itself, through the way it trains the model, and those rules have stopped Claude from doing work government users asked for. They pointed to Claude refusing questions from the Centers for Disease Control and Prevention about research on preventing the spread of infectious disease, which Anthropic said came from safety features built for business customers and was fixed with government staff. They also pointed to a dispute over whether Anthropic's rules allowed Claude's use in an overseas military operation, which Anthropic suggested was a misunderstanding. The word needs no bad intent, the majority said: the judges had "no reason to doubt" that Anthropic acted "with noble intentions," but the law "turns on what Anthropic does, not why Anthropic does it." The court also rejected Anthropic's free speech claim, saying the Pentagon cut the company off for refusing a contract term, not for its support for more government regulation of AI.
Judge Karen LeCraft Henderson disagreed. She read "manipulate" the way its neighbors in the law suggest, as something done in "a subtle, devious, or underhand manner," which would require bad intent. She wrote that Congress passed the law after intelligence agencies warned about hostile foreign governments and other bad actors slipping compromised products into federal systems, not to cover a contractor's "honest and upfront enforcement" of limits the government dislikes. Under the majority's reading, she warned, the company that replaces Anthropic, if told to drop its own limits, will have a choice: "Agree to the Secretary's demands or risk being designated a national security threat."
Anthropic says the label covers only the use of Claude as a direct part of Pentagon contracts and subcontracts, not other federal agencies, state or local governments, or companies' work outside their Pentagon contracts. A Pentagon order on March 6 called for Anthropic's products to be removed from its systems "as soon as practical" and within 180 days at most, and barred contractors from using them in their Pentagon work. The department quickly moved to expand its work with OpenAI, the court noted. A federal court in California set aside a separate designation on Aug. 27, made under a different law that covers only an "adversary" acting with bad intent. The appeals court accepted that Anthropic acted with no bad motive, but said the 2018 law is much broader.
It is not yet clear whether Anthropic will ask the full appeals court to hear the case again or ask the Supreme Court to take it up. The court called the case one that "raises profoundly difficult questions" about how the military should use a powerful new technology, and said it is the president and the secretary of war "who must determine how best to balance the competing risks."