İşte OpenAI Modeli Nasıl Dolandırıldı ve Hacklendi Hugging Face

Almost every week, something happens in security that ruins my sleep. A new model drops that I want to evaluate, we find some crazy 0-day, or someone else drops one . But this incident probably tops all of them.

Today, we’re looking at what might be the craziest and concerning sci-fi, paperclip-maxxing event to come out of OpenAI. In short, while trying to solve an ExploitGym challenge, one of OpenAI’s models reward-hacked/cheated the task and compromised Hugging Face production using two 0-day vulnerabilities to steal the solution which most probably is not hosted on hugging face.

LLMs finding zero-days serious enough to compromise companies is old news. Look at our work on Supabase , Palo Alto , and PostHog . Reward hacking isn’t new either. We see plenty of petty reward hacks internally, and there are documented examples from Anthropic and Alibaba .

What makes this different is how far the models went in a single-session, without any human hints or guidance, just to pass an exam. I don’t think this was a marketing stunt. We might never have heard about it if Hugging Face had not detected the attack first.

A model capable of solving unsolved math problems and finding zero-days in Linux, WordPress, and Chrome still does not know when to stop. It may be better than almost any human at the task in front of it, yet it still lacks the basic judgment of a child.

It appears to have treated “solve ExploitGym” as “do whatever it takes, even if that means hacking other companies.” As Zvi puts it, excusing this as simply following instructions is like saying, “You told me to make money. I don’t know why you are so upset about all the bank robberies.”

In any case, this is a serious alignment failure. What I really want to know is how difficult the hack was. Did the models exploit a few dumb bugs that were easy to find, or did they build the kind of complex chain only a top hacker could pull off? The answer tells us whether this was mainly an alignment failure or also a major jump in cyber capability. And, I know that OpenAI or Hugging Face won’t be revealing any technical details soon, so I decided to figure out.

Note: This reconstruction is based on the public information available and my experience using LLMs to develop exploits for tasks similar to ExploitGym.