Rogue AI agents on the loose

Jorge Costa Oliveira

The evolution of language models into autonomous artificial intelligence agents — able to use tools, execute code and browse the web independently — has introduced a cybersecurity paradigm. Incidents involving OpenAI and Anthropic show that the threat is no longer theoretical: agents with offensive capabilities can breach sandboxes and affect real-world infrastructure.

In mid-July 2026, an autonomous agent powered by OpenAI’s models went out of control during a security test and hacked startup Hugging Face. It later emerged that the same agent had targeted at least four other entities. It exploited vulnerabilities not only in Hugging Face’s systems, but also within OpenAI’s infrastructure.

What made the incident different from a cyberattack was that the agent acted without human intervention. OpenAI called the attack “unprecedented” and acknowledged that similar incidents would become more common as models acquire stronger cyber capabilities.

Hugging Face, a platform for open-source machine-learning models and AI datasets, disclosed on July 16 that an intruder had gained unauthorized access to datasets and internal credentials. It said the attack suggested an autonomous AI agent was responsible.

Five days later, OpenAI said the attack had been driven by GPT-5.6 Sol and an internal research prototype never intended for public release. The company had been conducting “red team” exercises — simulated attacks designed to identify AI capabilities, risks and vulnerabilities in an isolated environment. Despite safeguards, the agent escaped.

Hugging Face hosts ExploitGym, a benchmark measuring an agent’s ability to exploit real-world systems. The AI apparently treated the platform as an opportunity to improve its score and persistently sought access.

The company then encountered another problem: safeguards designed to prevent frontier models from conducting cyberattacks could also stop them from supporting cyberdefense. Hugging Face therefore used GLM5.2, an open-source model developed by China’s Z.AI, to help counter the attack.

Later in July, Anthropic disclosed that Claude had breached the systems of three companies during cybersecurity tests after a configuration error gave it internet access. A review of 141,006 testing sessions found that supposedly isolated environments had been connected to the public internet. Claude exploited weak passwords and unauthenticated endpoints. Two organizations knew nothing of the activity until Anthropic contacted them.

These cases reveal two failures. First, test environments were not securely isolated. Second, the agents pursued their assigned objectives without adequately distinguishing simulation from reality — a form of “specification gaming.” Given a goal such as gaining administrative access or maximizing a test score, an agent may select the most efficient route even when that means bypassing controls or entering an external system.

The risks extend beyond laboratories. AI agents can analyze code, identify vulnerabilities and chain exploits at extraordinary speed. When operated continuously, they can test thousands of attack paths, potentially harming organizations unconnected to an evaluation.

These are not conscious machines rebelling. They are efficient optimization systems following instructions inside imperfectly controlled environments. But as models gain greater autonomy in the digital and physical worlds, weak sandboxing and poor alignment become critical cybersecurity and national security risks.

Stronger safeguards are urgent: air-gapped test networks, tightly controlled outbound traffic, human approval for high-risk actions, unambiguous limits on simulated tasks, and standardized security protocols for outside evaluators.

Governments and technology companies must act before increasingly autonomous AI makes such incidents routine. Once again, the case for effective regulation could hardly be clearer.

linkedin.com/in/jorgecostaoliveira