A 5-day cyber attack breached Hugging Face, a popular American-French company that offers an open-source platform for machine learning and artificial intelligence between July 9th to July 13th. The company disclosed the security incident after their security team detected and contained an AI agent that compromised their infrastructure noting that “the breach marked the first time they handled a cyber event that was driven, end to end, by an autonomous AI agent system”.
This incident occurred during a joint internal evaluation between OpenAI’s research environment and Hugging Face production infrastructure which aimed to quantify the AI model’s cyber capabilities. These tests usually occur in a highly isolated environment with network access restricted to an internal third-party host. However, the model exploited a zero-day vulnerability(a hidden security flaw in a software/hardware completely unknown to the owner) in order to gain internet access. It further chained together multiple attack vectors(specific method used by hackers to gain access to a computer, network or server) using stolen credentials in order to form an execution path towards Hugging Face’s servers.
This anomalous activity was discovered internally by OpenAI’s security team that expressed “we have not identified any other activity at the level of severity and scale of what we’ve shared related to Hugging Face’”. The rogue model publicly exposed the stolen credentials across four accounts on four services. OpenAI CEO Sam Altman has paused all training in order to determine the security of it’s testing environments. “We may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels” Altman stated.
 |
| Photo: OpenAI CEO Sam Altman |
This security incident raised alarms for industry experts, researchers and government officials who believe that the rapid advancement of AI attack capabilities is a demonstration of how far AI agents will go in order to complete a task.
OpenAI’s rival company Anthropic, carried out their own cybersecurity evaluation following the Hugging Face breach and discovered three cases where its Claude AI model accessed the internet during evaluation and “gained unauthorised access to the real systems of three different organisations.
The breach was contained through the assistance of an open-weigh model leveraged from Chinese Company Zu.
Steps taken after incident
1. More than a 1000 employees from OpenAI, Anthropic and other AI companies signed a letter called “Pacing the Frontier” urging the US government to build the technical and governance tools necessary to slow down AI development.
 |
| Photo: Extract from Pacing the Frontier document |
2. Republicans Ted Lieu and Nathaniel Moran announced the AI Kill Switch Act, which will require AI companies to maintain the ability to shut down, throttle or suspend their models.
3. Erik Block, vice president of security at breach containment company Illumio said “this incident serves as a warning of what’s to come considering that the existing defensive tools used are already behind”.
OpenAi has taken full accountability and has deactivated, encrypted and restricted the rogue model from research access. It is also using this incident to strengthen protection surrounding it’s infrastructure configuration and model evaluation environments. The company’s official statement stated; “We encourage other defenders to apply for trusted access and experiment with these model now to translate these capabilities into better prevention, faster detection and more effective incident response.
Follow the Investigation process below:
Comments
Post a Comment