Back

Amazon says hackers' AI agents found a previously unknown software flaw and took over a university server

Illustration: small spider-like robots crawl over a tall server cabinet, one prying open a glowing hatch in its side, with a cable running from the hatch to a second server cabinet
Illustration: FrameSignal (AI-generated)

Amazon's threat investigators say a hacking group Amazon is moderately confident is tied to China set a free AI tool called Cairn loose on universities in East Asia in September, with goals such as "obtain shell access," meaning the ability to run commands on a computer from afar. Amazon says the tool's AI agents found a flaw no one knew about in a medical imaging app and used it to take full control of a server, and Amazon believes the group used the same flaw to get into a server at a second university. The app's makers have fixed the flaw in their code, but their advisory says the fix will reach users only in the next release.

Amazon's threat investigators say records left on a hackers' server show the hackers' AI finding and using a zero-day, a security flaw nobody knew about before it was found or used, with little human help. In a new report, Amazon Threat Intelligence said a hacking group used Cairn against universities in East Asia outside mainland China. Cairn is software whose code anyone can download, free for personal and educational use. Its maker calls it a general problem-solving engine, first tested on probing computer systems for weaknesses, and says it should be used only where the user has explicit permission; Amazon says it was built for defenders. It runs a team of AI agents, AI that can carry out tasks on its own, on AI models its users connect to it. The group gave it a list of internet addresses and goals such as "obtain shell access," meaning the ability to run commands on a computer from afar. Amazon says the agents then "planned, executed, and adjusted their approach as they learned, without the operator choosing each next step."

Amazon found the operation on Sept. 10 because the attackers had left their Cairn server open to anyone on the internet, with no login. The server held the attackers' instructions and the AI's output for a campaign of about 48 hours that began on Sept. 8. The instructions were written in Simplified Chinese, and the times recorded in its scripts were set to China Standard Time. Together with the targets, universities likely of strategic interest to China, this makes Amazon moderately confident the group is tied to China.

Against one target, a server running a free medical imaging AI app, the agents worked out what software the app was built on and mapped the ways other programs can talk to it. They found an upload feature that let them pick where a file was saved. It took them about 11 minutes to go from mapping the app to writing files anywhere on the server, Amazon says. In a later session, they used the same flaw to plant code that runs every time Python, the programming language the app uses, starts up. They then set off the app's training feature, which started a new Python process that ran the code and gave them control of the server with the highest level of access. Amazon says it is highly confident the flaw was unknown, since it found no fix, warning or earlier mention of the flaw in the project's records or in public lists of known flaws.

The attackers then stepped in themselves. Amazon says they handed the agents a tunnel into a second university's network, through a server running the same app, and Amazon infers that the group broke into that server with the same flaw. Elsewhere the agents failed. At one university, the agents spent about 16 hours and 148 steps trying to take control of a database that was already open to anyone, without succeeding. Another effort, against a university's cluster of powerful research computers, built a list of about 80 real usernames from that university's directories, lab websites and academic papers. It then tried more than 30,000 username and password pairs on the cluster and on other university systems, including a VPN server and the campus portal, over more than six hours, without finding one that worked.

The app is MONAI Label: Amazon's report links to the security advisory its makers posted on Nov. 18, which says Amazon's security team first reported the flaw and rates it critical. The makers have fixed it in their code, but the advisory says versions 0.8.5 and earlier are affected and the fix will come in the next release, so no fixed version is out yet. The login is off by default, and even with it on, any user allowed to label images can still use the flaw. Until the fix ships, the advisory tells users to turn on the login and give that role only to people they trust, to keep the server off networks untrusted people can reach, and to run it with access only to its own data folder. Amazon says the universities were not its customers, so it worked with a government partner to warn them.

Amazon says Cairn is being taken up by both spies and criminals. Gambit Security, a security company, reported on Sept. 22 that a separate, money-driven group was running Cairn and two other AI tools in attacks on hundreds of online retailers. Amazon's report does not name the universities or the country they are in, and does not say whether anything was taken from them or which AI models Cairn ran on. Amazon says an agent that finds and uses a new flaw in minutes finishes faster than many security teams can check a single alert by hand. It says defenders should let automated systems answer the clearest threats, with people overseeing the results.

More on Amazon