Hugging Face Breach: Internal Datasets and Credentials Compromised, Users Urged to Rotate Keys

Hugging Face has disclosed a security breach that compromised its internal datasets and service credentials, prompting the company to urge users to rotate any API keys or tokens stored on the platform. The incident, which occurred last week, was detailed in a blog post on Friday, though the company is still investigating whether customer or partner data was stolen.

The attack vector is particularly alarming for the AI/ML community: a dataset uploaded to Hugging Face's platform exploited a security vulnerability to execute malicious code on its servers. That initial foothold allowed the attackers to escalate privileges and gain broader access to internal systems, including credentials that the company has since revoked and rotated.

Hugging Face attributed the breach to an external AI agent that performed "many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services." The company did not immediately provide evidence for this claim, but it underscores a new frontier of threats where AI agents are used to automate and scale intrusion attempts in ways traditional attackers cannot.

Interestingly, Hugging Face's own anomaly detection—powered by an AI model—was what spotted the attack. Initially, the company tried using a frontier commercial model from an unnamed provider to analyze server logs, but found the analysis was blocked by the provider's guardrails. Hugging Face then switched to its own local large language model, which not only avoided uploading sensitive attack logs to a third party but also provided the analysis needed to understand the breach.

This incident highlights systemic challenges for platforms hosting open AI models and datasets: they are inherently exposed to malicious uploads, and securing that surface area requires constant vigilance. For developers and organizations using Hugging Face, the immediate action is to rotate any tokens or credentials stored on the platform, review account activity for suspicious behavior, and consider adopting a zero-trust posture even for internal AI services.