Hugging Face published the survey on Sept. 11, 2026, drawing on 15 reports and release posts from 13 labs and model makers. It looks at how they train AI agents with reinforcement learning, learning by trial and error, with a reward for getting it right. It counts only the setups labs say they train in, not ones used only for testing. The picture it paints is of training that depends less on clever algorithms alone and more on running huge numbers of real computers at once.
The reason is what agents now practice on. To install a program, run a set of tests or edit a code project, an agent needs an actual computer, with files, a command line and programs that keep running between steps. So each attempt gets a sandbox, a sealed-off computer of its own, which is thrown away after a short attempt or saved and resumed during a long one. At the high end, labs run hundreds of thousands of them at once.
Companies that run this kind of training describe the same pattern. Cursor says training its Composer coding agent took hundreds of thousands of sandboxed coding environments running at the same time, and that it rewrote its scheduling software to cope with the sudden bursts. An industry analysis reports that each attempt needs at least one sandbox, and that scaling up tenfold, to hundreds of attempts at once, went past the limits set on the analysts' own account and brought start-up failures and some sandboxes that took about an hour to start.
Paper Instruments, which uses E2B's sandboxes for this training, describes thousands of isolated attempts running at once, each with its own files and saved state, all started from snapshots of a shared starting point. Modal says this kind of training can create bursts of hundreds of thousands of sandboxes, and that it has run one million at once, creating all of them in under a minute.
Across the reports, Hugging Face finds the same four parts: a task with a built-in way to check the result, a set of tools and rules the agent works through, a sandbox to run it in, and a trainer that learns from finished attempts without waiting for the slowest one. Each lab builds those parts its own way. The survey argues that letting attempts run without waiting on one another, and the computers that host them, are becoming as important to training better agents as the learning method itself.
It is not clear how closely each lab's private training data and computing setup match this public four-part picture. The speed and scale figures from Cursor, Modal and Paper Instruments come from those companies, and nobody else has measured them yet.