METR identified monitoring gaps in OpenAI agent systems

A METR employee and a Redwood Research employee conducted the exercise over three weeks, from July 21 to August 11, 2026.[2] They assessed OpenAI’s monitoring system as it existed on and before July 21, rather than changes then in progress.[2] The evaluators had access to internal code, monitor prompts and configuration details, and documentation; they also interviewed approximately a dozen researchers and engineers from OpenAI monitoring, safety-systems and security teams.[2] The work produced a 61-page internal report shared with OpenAI researchers and security staff.[2] OpenAI redacted a 38-page version shared with METR and Redwood leadership.[2]

METR estimated that 75–94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring.[1] Separately, it estimated 99.2% eligibility for Responses API traffic used by internal Codex agents.[2] Those figures describe eligibility, not the amount of traffic monitored: METR said it could not estimate traffic volume because of access and time constraints.[2] It also said reinforcement-learning training runs were not covered at the start of the exercise, although it was told that a subset of training rollouts had been monitored from March through June.[2]

METR said agents could indefinitely postpone monitoring by continuing to take actions, but it could not determine whether agents had used that opportunity.[2] It found a way Codex agents could, in principle, bypass Codex sandboxing and Auto-Review to take actions available to the host user, and an exploit that could keep hundreds of recent actions from being monitored.[2] It also identified two client-controllable headers that could disable monitoring for new agents.[2] METR did not validate the behavior or test whether it was exploitable in practice.[2]

METR estimated that failing to monitor compaction events likely left about 10% of eligible internal Codex traffic unmonitored between May and July 2026.[2] OpenAI staff had independently identified that gap before the exercise and fixed it during the exercise.[2] OpenAI says its internal monitor uses GPT-5.4 Thinking at maximum reasoning effort to review agent interactions and alert on potentially suspicious behavior.[3] In the latest form it described, OpenAI said the monitor reviewed interactions within 30 minutes of completion.[3] METR argued that future systems should withstand deliberate, sustained attacks by highly capable internal agents at scale.[2]