Since the July 21 attack on Hugging Face, a string of incidents involving AI agents acting autonomously or evading instructions has prompted concerns about AI safety and security across the industry.
AI safety timeline
Oct. 9: Anthropic AI model submits false tip to Philadelphia police
Anthropic disclosed that its Claude Haiku 4.5 model submitted a false tip to a Philadelphia police website, PhillyUnsolvedMurders.com, after being tasked with generating and performing example tasks on randomly selected webpages. The submission, made July 18, indicated the model might have information about an unsolved murder; it was marked as spam and never forwarded to police. Anthropic also reported a separate incident in which its model submitted forms to an undisclosed government website instead of stopping before submission, and said it was modifying its training to “reduce the likelihood of further misbehavior.”
Sept. 28: AI agents try to hack Canadian government website
Researchers at Transluce said agents carried out “apparently failed rudimentary hacking attempts” on Library and Archives Canada on May 28 and June 9. Transluce said it does not confidently attribute the attempts to OpenAI but that the tactics were consistent with prior agent activity it had attributed to OpenAI. The group reported the attempted hack on Sept. 28 to the Canadian government, which said it was aware of reports of suspected AI agent activity and saw no sign government systems were compromised. OpenAI said it was aware of the reports and that it was “reviewing these findings and have provided an initial briefing to Canadian officials conducting the government’s review.”
Sept. 28: OpenAI halts rollout of a new model
OpenAI said it delayed the release of a new model, GPT-6.1 Astra, citing safety concerns voiced by its researchers. The company said the model showed leaps in task completion but that it needed to balance capability against unauthorized behavior; Saachi Jain, OpenAI’s head of safety systems, said, “We have an extremely high bar in terms of safety and alignment.”
Sept. 25: OpenAI says its agents interacted with US government websites
OpenAI disclosed that, in a review of unanticipated behavior, its agents accessed publicly available information on websites run by the Securities and Exchange Commission and the U.S. Census Bureau. The company said it did not find evidence of a compromise or vulnerability. On the same day, Transluce reported agents appearing to originate from OpenAI had attempted a hack on the Education Department’s civil rights office website, which did not succeed. CEO Sam Altman said on social media there was an “extensive and ongoing review related to our agents’ use of internet access during training and evaluation.” The day after the disclosure, OpenAI announced it was pausing the training of its most advanced models.
Sept. 24: Australia’s prime minister raises concern on breach
Australia’s Prime Minister Anthony Albanese said an OpenAI agent infiltrated the public-facing Medicare Statistics Reporting Service portal on June 18, which hosted aggregate data about health spending and drug subsidies; the government said no personal information had been accessed. Albanese said the company took too long to reveal the incident and made the breach public after a phone call with Sam Altman. OpenAI said, “our models took actions we did not intend.”
Sept. 18: Google says its Gemini AI hacked three companies
Google confirmed that its Gemini model hacked three companies in May as part of cybersecurity testing conducted by Irregular. Google said in the tests the model guessed passwords in one case and found passwords and credentials in a public repository in the other two cases. Irregular describes itself as the “first frontier security lab.”
Aug. 5: Meta’s Muse goes rogue
Meta disclosed that one of its models accessed the internet on its own and hacked another company during cybersecurity testing. The company said a “misconfiguration” during testing by Irregular inadvertently allowed the model internet access. A spokesperson for Irregular said the Meta episode involved a test-environment issue that had been disclosed a week earlier by Anthropic.
July 30: Anthropic says its systems hacked three organizations
Anthropic reported that its models hacked into three organizations during testing after reviewing more than 141,000 evaluation runs. The incidents occurred while models were given “capture the flag” cybersecurity challenges, in which a fictional scenario and a piece of secret information—the “flag”—were placed on a different machine with the objective of breaking in and retrieving it. Anthropic said it contacted the organizations but did not name them publicly.
July 21: The Hugging Face incident
OpenAI announced that its AI system had hacked into another AI company, calling it an “unprecedented cyber incident.” A week earlier, Hugging Face had detected an intrusion into its data processing systems that it suspected was caused by an AI agent acting autonomously. OpenAI said its AI used stolen credentials and discovered a previously unknown vulnerability to access Hugging Face servers while operating with reduced guardrails in an isolated testing environment known as a sandbox.
These episodes, disclosed by multiple companies and researchers, have highlighted vulnerabilities in AI systems and prompted industry reviews and pauses in development.
