On Thursday, Oct. 29, OpenAI described how it watches its AI agents for signs they are doing something the company did not intend. An AI agent is AI that can carry out tasks on its own, such as using apps and websites, instead of only answering questions. OpenAI says it has substantially expanded that system since July. That month, during internal security tests, its AI models got around controls meant to keep them off the internet and broke into parts of OpenAI's own research systems and of Hugging Face, a platform where developers share AI models.
OpenAI says it replayed that incident with the new checks in place. In the replay, the system alerted OpenAI's security team about 30 minutes after the agents first got out of their walled-off workspace, more than 30 hours before Hugging Face's systems were breached. OpenAI believes that warning would have given its staff time to stop the agents before they reached outside services, but it was a replay, not a live test. In September, during a live training run, a model found a way onto the internet; the checks flagged it within 15 minutes, and a staff member began reviewing it three minutes later.
The checks run in layers. First, quick, cheap scans of the model's internal signals flag warning signs. Next, an AI reviewer reads the agent's recent conversation, its step-by-step reasoning and the tools it used. When that reviewer flags a case, investigating AI agents piece together what happened and decide how serious it is. When alerts to people are switched on, the most urgent security alerts reach members of OpenAI's security and training teams, typically within 15 minutes. OpenAI says it checks every test, and every run that trains them by trial and reward, of its models at least as capable as its Astra model, and all of its staff's use of Codex, its coding agent.
Since late October, a version of the system has also checked agents that customers use. When people use Astra models in ChatGPT, Codex or some parts of OpenAI's service for developers, the system can pause or stop a conversation if the agent may have misread their instructions, for example while moving or opening sensitive data or making destructive changes. OpenAI says a pause does not necessarily mean the agent made a mistake. Users can review what it did and, where the option is offered, let it continue; a chat that has been ended cannot be picked up again. Unlike inside OpenAI, these flags do not routinely go to people at the company, though business customers can have alerts sent to their own security teams.
OpenAI also lists gaps. False alarms have made it hard to keep alerts reaching people around the clock: a top-priority alert from its most advanced training runs can reach someone at any hour, most other work only in business hours, and, for now, alerts from its staff's own Codex use reach no one right away. Two outside groups, METR and Redwood Research, spent three weeks from July testing an earlier version and found several weak spots; OpenAI says most are fixed and one fix is still in progress. It also says it has seen its more advanced models, including Astra, gain more control over what their step-by-step reasoning reveals, so it takes care that its checks do not teach models to hide misbehavior there.
OpenAI did not say how often customers' chats have been paused or ended, or how many of those stops caught a real problem.