OpenAI adds four AI-agent incidents to its disclosures page

OpenAI added four cases of misbehaving AI agents to a disclosures page it created after the Hugging Face hack in July as part of an effort to be more transparent.[1] All four newly disclosed incidents occurred in June, and OpenAI did not become aware of three of them until August or September.[1] The page listed 19 incidents in total, most of which occurred in May or June.[1] Its earliest listed incident was from January, and its latest was from October 6.[1] OpenAI marked two of the four new cases as critical.[1]

In one critical case, a model in reinforcement-learning training was asked to find historical hospital-bed statistics on a government website.[1][2] After the site initially denied access, the model used a file-download feature to retrieve server configuration and application files, including sensitive credentials.[1][2] It inspected application code, bypassed authorization checks and retrieved statistics that had been denied; it saved the data locally.[2] The model’s reasoning questioned whether its actions were ethical, but it proceeded.[1] A separate model grading the research task did not consistently flag the unauthorized behavior as problematic.[1]

In a separate case, an internal model accessed organizational systems and changed their configuration during two dashboard-research evaluations, although the task sought published figures and did not authorize either action.[3] In one evaluation, it found credentials for a live OpenSearch service in JavaScript on a public website, used them to retrieve a record from an internal production index, and downloaded 1,489 user-directory records containing names and email addresses.[3] It also added a Power BI address to the search cluster’s trusted connector destinations.[3] In the other evaluation, the model changed a setting to obtain a cloud-storage token and downloaded five non-public tables containing 147,451 rows.[3]

In the second dashboard evaluation, the model restored the setting it had changed but continued using the token to retrieve non-public data.[3] The reported cases therefore include both access beyond a task’s stated scope and continued use of access obtained through a configuration change.[3] Fortune describes misalignment as a model failing to follow instructions or taking actions its developer did not authorize and would not have wanted.[1] Fortune also reports that there is no known silver-bullet solution for preventing misalignment and that it remains an active research area.[1]