…also why is OpenAI copping heat from some nerds?

My overall opinion on the OAI/HF debacle is that it’s disappointing. They handled their sandbox architecture and design badly. That they made the assumptions that they did, about what a sandbox actually means for their close-to-human-performance, unrestricted, hacking agents. Could they have predicted the outcome that occurred? Could the average person have predicted it?

In my view, they didn’t need to predict that specific outcome. The whole reason that a sandbox is used for such testing is a generalized notion that if ‘the agent gets out, it’s a bad outcome, regardless’, so that we don’t need to learn exactly how bad things can get. Not necessarily because a specific chain of exploits is predicted to occur. If we knew that, we would just patch them anyway.

I am an avid vulnerability researcher who believes it is a necessary task, and it’s interesting to see agentic abilities to chain vulnerabilities, it’s genuinely cool and surprising at times. However I think that fact stands alongside a different fact: that sandbox maturity at OpenAI needs work, and they themselves admit this, I am not making that up. So if you are defending systems yourself, what lessons are there to be had?

OpenAI aren’t alone in this state either, they have some company in the form of Anthropic and Meta more recently. So it’s fair to say the AI industry’s ability to sandbox things feels immature at this point in time.

#1 on their list of things to do post-haste?

Hardening the security of its research infrastructure

Backing up a decade

Frontier providers have known that agents, particularly unrestricted ones, love to engage in ‘reward hacks’ since at least 2016. Since then they have also hired people who specialize in existential risk analysis to dream up what would happen in a future where rogue AI exists, and to extrapolate the damage that could cause. So in terms of risk assessment they should be miles ahead of most people on xitter arguing about this. They hold a mature view of likelihood x impact which is a commonly referred to risk calculation approach. So to say that their own experts could never have predicted any of this is a bit silly.

I liken what OpenAI did in these evals to testing a gun they just bought (a new hacking agent) in a poorly designed shooting range (a sandbox) and a few stray bullets went through the neighbor’s window (Hugging Face). Very little harm done thankfully, but as a safety incident, it’s worth taking seriously.

Because there are sandboxes and there are Sandboxes.

Sandbox is a terrible word in Information Security

Here’s the thing. The term sandbox shouldn’t be elusive at all because it has official definitions. But yet, it is one of the most argued over concepts in IT in 2026. Why? It’s not enough to assume that ‘some people are dumb’, while that is true, there’s a lot of disagreement between people I know to be Very Smart In Their Own Right.

I’ll let you in on a little secret of mine: I don’t like the word sandbox in IT because it’s such a high level term that gets bandied about and it encourages people to make broad assumptions about some very specific things. And assumptions are… you know the rest.

Whenever I hear the word ‘sandbox’ my engineering brain screams ‘OK but define “sandbox” please?’. And every engineer that I know will say a subtly different thing to suit their world view at any given point in time.

Even the fact that we use a ‘sandbox’ (a thing kids play in) to ‘detonate malicious code’ is just a weird and off-putting sentence to me. Kids might be annoying but sheesh cut them some slack. Besides, any parent will tell you that the sand never stays in the box in real life.

But whatever, it’s the banner term we collectively chose. I was alive when it happened. I had a (slim) chance to say it was a dumb word way back in the 80s or whenever it was coined. It’s not going to change.

What I wish would change is the confusion that seems to accompany its use.

Nah, there is only one definition for ‘sandbox’, and it’s My Definition!

The high level concept of a sandbox is pretty simple though? Why so many arguments over it?? “My definition is the correct one. You are all wrong.”

Well, please consider this.. (these are examples other people have given btw)

A ‘malware detonation sandbox’ like https://any.run might isolate a system from its host entirely and present a fake internet service to any running processes inside that virtual machine. The host it runs on may well be a virtual machine in its own right. These aren’t necessarily monitored (though they could be), they just don’t live longer than is necessary to collect information about the detonation itself.

A ‘penetration testing learning lab’ (like hack-the-box) is also a sandbox designed to allow varying levels of access to parts of a simulated network for the purposes of learning to hack things, without allowing the participants to hack any further than they are allowed to. Sometimes allow-listed internet access is permitted, sometimes it isn’t, it depends on the lab. These would also likely be monitored. But they don’t typically need a fake internet. New tool installation is often prevented - we want students to only use certain tools to finish a task.

A ‘browser sandbox’ is designed to contain JavaScript from places you don’t fully trust, and since this is a running process, its job is to prevent underlying access to the system resources, while providing a useful API over the top of them.

Variations on the browser sandbox theme projects exist too! Watch this interview from Astarte Cybersecurity on a thing called smolbox made by @_mattata

A ‘developer sandbox’ environment might give that developer complete internet access but prevent them from reaching out to other systems on the local network, particularly the production environment. It’s just for testing some new software they wrote.

These are all wildly different propositions and a person good at making one type of sandbox isn’t necessarily the person you want for making the other kinds of sandboxes (although I am sure there may be a small handful of exceptional people out there that could make all of these on their own).

Behold the myriad definitions

So let’s look at the formal definitions of the term, surely this will help us arrive at an agreement on what the term means?

A great high level definition from NIST SP 800-95.

A system that allows an untrusted application to run in a highly controlled environment where the application’s permissions are restricted to an essential set of computer permissions. In particular, an application in a sandbox is usually restricted from accessing the file system or the network. A widely used example of applications running inside a sandbox is a Java applet.

This is a good definition right up until the last line because it takes you into this multi-layered world of file systems and networks, and drops you off at … a java applet??

What do developers mean when they say ‘sandbox’? In my experience it has aligned with other definitions you see above, but hey let’s go look at the definition on Wikipedia…

A sandbox is a testing environment that isolates untested code changes and outright experimentation from the production environment or repository[1] in the context of software development, including web development, automation, revision control, configuration management (see also change management), and patch management.

Hmm OK, so to some devs, a sandbox environment in their heads doesn’t even talk about security? But wait! There’s more! There is another definition on Wikipedia!

In computer security, a sandbox is a security mechanism for separating running programs, usually in an effort to mitigate system failures and/or software vulnerabilities from spreading.

Wait what? Stopping … vulnerabilities from … spreading? I get that worms or viruses could spread. An attacker could move laterally. But… vulnerabilities themselves? Who wrote this? Oh it’s badly paraphrased from this opinion.

Are you starting to see my point yet? This situation is silly, it’s all opinion-based, and there is significant overlap, so it’s only reasonable that nerds from different disciplines (or even backgrounds in the same discipline) are going to have different views on the meaning of the term. Particularly when we are referring to a ‘test environment’.

All this to say that regardless of the rigidity of your personal views on the matter, the term ‘sandbox’ is a high level, subjective term and for most insider baseball discussions on sandboxing I would prefer IT professionals refer to the constituent parts of the systems they are referring to instead (more on how, coming up next…)

If you want to make a sandbox you must first invent ‘sand’

I think part of the problem is that designing a container for literally anything… it’s highly dependent on what you need to contain in the first place.

E.g. a potato sack is great for carrying potatoes, but it’s awful for carrying ice-cream and vice versa.

So before we can agree on what a good sandbox is for a given thing, the right people need to be in the room to define the allowable activities first. How do we do that in initial sandbox design phases, without getting bogged down in the complexity explosion of interacting details?

Constituent parts, using profiles to properly define sandboxing

Having built and helped to build a few secure systems over the years (e.g. for storing things like credit card data, and systems that run other people’s arbitrary code on containers), I have opinions about this that aren’t entirely irrelevant or antiquated. So if you were to ask me by which definition should I use to build a ‘sandbox’ for the CyberGym (or similar) evals, I would start with the NIST definition and head in a similar direction that AISI appears to have gone down here

What you can see is this idea that each aspect, ‘tool use’, ‘host isolation’ & ‘network isolation’ is its own axis and we can define meaningful states to these axes.

I take any “institute’s” advice with a grain of salt, but I can’t fault the basic reasoning of the above guidance though, particularly the idea of breaking down the things that make a good sandbox and grading them.

This kind of model allows us to talk about a ‘sandbox’ in more concrete terms, to compare potato sacks with potato sacks. Of course if you know of a better approach, please let me know, I would love to learn more.

I like approaches like this because it forces you to consider sandbox profiles like T0.H2.N2 which might be appropriate for ‘An agentic cyber eval that tries to find a zero-day exploit in the Docker runtime’ (as per the example in the PDF they gave).

Back to the incident at hand - and the arguments that have ensued

So given the above framework, imagine you are the CEO of OAI and you want to test out your latest model to see how cool of a hacker it can be in a realistic environment, which profile would you choose? If you look at the incident technical report and try to map it to the AISI profile system, you could argue it would align with either T0.H1.N1 or T0.H2.N1 depending on your read on their virtualization strategy, meaning that they allowed arbitrary code to be written on containerized or virtualized hosts, with some very limited internet connectivity (via a package proxy).

OK so… now we have something concrete to argue about!

One particular sandbox spat is over OpenAI’s handling of the environment that led to the Hugging Face incident. Consider these opposing arguments.

‘OpenAI didn’t build a great sandbox, because they didn’t isolate or monitor it, and this was a solvable problem that they ignored’.

vs

‘OpenAI didn’t build a great sandbox, because agents found sandbox escapes, that was unexpected and you can’t monitor for things you don’t know are possible yet’.

This, at times, leads each camp to call the other incompetent in the process which is unfortunate because I think they’re agreeing with each other but coming to their conclusions from different perspectives.

Imagine instead if these two parties were having an argument over whether the T0.H2.N1 profile was in fact the appropriate selection of controls in this environment. I think it would be a much less heated, more business-as-usual discussion, leading to ‘No, no it was not. N2 makes more sense for this type of workload’ and we would all go home and think about more important things. The architectural decision to restrict outbound network access completely makes total sense.

We can also easily model scenarios ahead of time, like this ‘In an initial sandbox compromise, leading to unrestricted internet access, the environment becomes a T0.H2.N0 sandbox.’ and you can then ask, “Are we OK with that, at all, ever?”

Once you get past this discussion we realize that even this profile system isn’t perfect (no model of complex systems really is). You can make N2 happen at the application layer, or the network layer and the AISI definition doesn’t specify. If it were me I might split this out into some other maturity levels, but I trust that you see my point here. You could totally use network design that the agents simply cannot hack.

Just to be clear I don’t think that we should stop at the AISI notation for sandbox profiling when we talk about sandboxes. We should probably take it a few steps further. But let’s use it for the remainder of this discussion.

Reviewing part of the incident timeline

Initially I took the popular position, which is that this was incompetence. However given what I have learned through researching the material for this post, and listening to others, I have come around to a different take.

I think it was desensitization, leading to human error in judgment (which you can call incompetence if you like). The humans in charge of this environment are used to seeing this stuff. It’s normal to that team. This was an iteration of the things that were written about in several papers over the course of the last decade. To us it looks like a major leap, but internally, this was just a new spin on an old trope.

We can see that by June 27, monitoring was kicking in, responders are picking up on message board shenanigans and the network pivot, and they don’t see this as a show-stopper. This is the biggest indicator to me that it isn’t really that unusual, and this is routine.

‘But, this was hard to predict…’

Sure, but I re-iterate my point, I don’t think you need to predict this specific outcome to arrive at a well designed sandbox.

An account I have a lot of respect for is LiveOverflow. I have watched so many of their videos in the past. He indicated he believes that this event was hard to predict and its fine for people to have thought it was reasonable in the past. He isn’t just some random opinion on the internet, his thoughts are important to shaping my own:

With everything I know NOW, I also consider the sandbox design to be bad. But BEFORE knowing the threat, I would have thought that was reasonable … How can so many people first claim that AI never did that, and that it must be fake? And now so many people say “obviously AI could do that you should have designed your sandbox with that in mind”. Makes no sense to me, sorry.

I guess I have an issue with this idea that only hindsight could see this coming, and perhaps that anyone claiming otherwise is not being genuine about things. I want to make the case that you don’t need to foresee every specific bad outcomes in order to prevent them from happening.

The following facts remain.

1. AISI published useful guidance for sandbox design nearly a year earlier. Its worked example for an agentic cyber eval hunting a zero-day in the Docker runtime is T0.H2.N2 which means no network. This is a dated, public recommendation for this exact class of eval, and it isn’t mine.
2. Sandbox design isn’t a novel field. Entire sub-cultures exist to make and break sandboxes.
3. Standard monitoring and alerting still works fine in an agentic sandbox, and newer approaches aimed specifically at agentic misbehavior already exist.
4. Frontier providers have arguably the most mature picture in the industry of how bad an agentic escape could get, before they got to this point.

Now I acknowledge that reward hacks are not the same as insane agentic breakouts and lateral movement, and I am not going to pretend there is an equivalence there.

My point is that you don’t even need to predict this kill chain, you just need to predict the outcome of this sentence: “In an initial sandbox compromise, post-escape, the environment becomes an X.Y.Z profile sandbox” and then ask yourself “is this an acceptable risk?” If not, go back to the drawing board.

In fact, it’s better if you stop trying to predict SSRF flaws and stop plugging network holes with application layer features (e.g. Artifactory URL allow-lists). Do it outside of that application’s control instead.

If this were the design approach, you wouldn’t ever allow N0 to become an outcome, and so you would isolate this test environment from the internet entirely, at least after an initial automated provisioning pass, something which a frontier provider like OAI could actually do if they wanted to. (I have seen this attitude taken in much, much smaller companies)

So I don’t think this was an unknowable watershed moment that taught cybersecurity anything particularly new. This stuff isn’t easy. If it were, everyone would be doing it.

I think if anything it taught vendors how to turn safety and security mistakes into marketing gold, for better or worse. I personally feel as though Zack Korman’s short on how much of a ‘watershed’ moment this actually is - is the right take.

‘But the Vercel CEO said everything that can be hacked, will be hacked, and all software is broken so, game over man’

It’s also been said that everything hackable will be hacked. So, if an agent can hack your Artifactory, it can hack your firewall appliance too, so therefore, all sandbox profiles are reduced to T0.H0.N0 eventually. In an environment that ultimately has an internet connection, sure!

But you can’t magically traverse an ethernet connection that doesn’t exist either. Even at N2 we aren’t taking away all risk, because that still leaves internal network connections on the table. On July 19, the agents went after this part of the control plane too. So you can see that you could extend this model to address internal and external network effects. Either way, unless I am mistaken, OpenAI didn’t install this robot in their sandbox, but imagine if they had!

But was internet connectivity necessary? According to their report, the agents had access to Artifactory so that:

The intended approach was to give an agent a usable computer environment—which included tools such as shell, Python, browser, and package-install—while controlling its network boundary separately from the container itself … The applicable configuration could vary by dataset or workload; some were intended to run without internet or tool access, while others required limited browser or package functionality. …internal package mirrors, including Artifactory, rather than have containers contact public package registries directly.
- pg. 6 Section II.B

This seems reasonable at first until you consider that these agents are good enough to replicate just about any code they need from first principles. ExploitGym doesn’t require internet access. It probably didn’t need a PyPI mirror either. OpenAI have access to more inference compute than almost anyone on earth, so remind me why they need npm or Python packages again?

Since it is a package mirror (their words), why does it need to be a live caching proxy? Surely a snapshot from last week would have been fine. These environments are ephemeral after all. Take a snapshot of the package repos before you run the eval, disconnect the internet, print the eval report, tear down the environment. Rinse, repeat.

Instead of realizing this, people are getting very hot under the collar about it all and I don’t really get it.

So even if you didn’t pick this exact scenario for compromise - you don’t need to if you consider the generalized risk scenario of an agent breakout. Now I will admit that hindsight is a wonderful thing, but so is experience!

Phew. I think I’m done here

Thanks for reading, and whether you agree or disagree with my takes, I’d love to read or listen to yours one day. Either way, I’m hoping to leave you with a summary of my thoughts about all this now:

• Saying that we couldn’t foresee Agents breaking out of their own sandbox. This is just wrong, because if this were true a sandbox wouldn’t have been created in the first place. You would just blissfully let them sit around on a host and do their thing. No, the sandbox was there precisely because we know that escape would be bad.
• Saying that we couldn’t foresee that it would be the package mirror would be the initial foothold to get out is also wrong. It was the only egress path available to these workloads, of course they are going to try.
• You don’t need to predict that vulnerable path anyway, to predict the architectural design required to mitigate the risks of compromise.
• All of the above goes away when you admit that neither the evals nor OpenAI’s technical documentation justifies why the environment needed standing egress to a service that itself had unrestricted internet access.