Sandboxing Agent-Generated Code: Containing What You Can’t Fully Trust

If you build agents that generate and execute code, the generated code is untrusted code you are executing, and operating it without sandboxing is executing untrusted code with the access the execution environment provides, which is a security risk. Containing the code in a sandbox that isolates it and limits its access is what makes executing agent-generated code safe, and operating it unsandboxed is trusting code you cannot fully trust. Understanding why agent-generated code cannot be trusted and how sandboxing contains it is what lets you use the agent’s code-generation capability safely.

Why agent-generated code cannot be trusted

Code that an agent generates is code you did not write and did not review, produced by a non-deterministic model, which means it cannot be trusted the way reviewed code can. The code could be wrong, doing something other than intended because the model generated incorrect code, and it could have unintended effects, doing something harmful not because it was manipulated but because the model generated code with consequences it did not foresee. The non-determinism of the model means the generated code varies and cannot be fully predicted, so you cannot trust that the code the agent generates does only what it should, because the model generating it is not reliable enough to trust its code unconditionally.

The code could also be manipulated, because the agent’s behaviour is influenced by its inputs, which means a manipulated input could induce the agent to generate harmful code, turning the agent’s code generation into an attack vector. An attacker who manipulates the agent into generating malicious code, and the agent operates that code, has used the agent’s code-execution capability to operate their malicious code with the access the execution has, which is a serious security risk that the agent’s code-generation capability creates. Agent-generated code cannot be trusted because it is unreviewed code from a non-deterministic model that could be wrong, have unintended effects, or be manipulated into being harmful, and operating it is executing untrusted code, which is a security risk regardless of the agent’s intent, because the code’s trustworthiness cannot be assured. The untrustworthiness of the generated code is why operating it requires containment.

How sandboxing contains the code

Sandboxing operates the agent-generated code in an isolated environment that limits what the code can do, containing the untrusted code so that even if it is wrong, harmful, or manipulated, its effects are bounded to the sandbox. The sandbox isolates the code from the systems it should not access, limiting the code’s access to only what it needs and containing its effects within the isolated environment, so that code doing something wrong or harmful is contained rather than affecting the broader system. This is the bounded action space applied to code execution, limiting what the executed code can do so its blast radius is contained, and it is what makes executing untrusted code safe, because the containment bounds the harm the untrusted code can cause.

The sandbox limits the code’s access and effects along the dimensions that matter for containment. It limits what systems and data the code can access, so the code cannot reach systems and data beyond what the task requires, containing the data access the code has. It limits what actions the code can take, so the code cannot take harmful actions beyond the sandbox, containing the code’s effects. And it limits the resources the code can consume, so the code cannot exhaust resources, containing uncontrolled or malicious resource use. These limits contain the untrusted code, bounding what it can access, do, and consume to what the sandbox permits, so that wrong, harmful, or manipulated code is contained within the sandbox rather than affecting the broader system. Sandboxing contains the code by isolating it and limiting its access, actions, and resources, which bounds the harm the untrusted code can cause to the sandbox, making operating the untrusted code safe.

The trade-off, and sandboxing matched to the risk

Sandboxing costs engineering and constrains the code, and naming that trade-off keeps the containment matched to the risk. Building the sandbox that isolates and limits the code is real engineering, more than operating the code without containment, and the sandbox constrains what the code can do, which limits the code’s capability to what the sandbox permits, so sandboxing trades the engineering and the constraint for the containment of the untrusted code. This cost is justified for any agent executing generated code that could cause harm, because the alternative, executing untrusted code without containment, is a security risk that the sandbox exists to contain, and the cost of the sandbox is far less than the cost of untrusted code operating with unbounded access.

Matching the sandbox to the risk is the judgment, because the containment the sandbox provides should match the harm the untrusted code could cause, which depends on what the code has access to and what it could do. Code that operates with access to sensitive systems and data needs strong sandboxing that tightly limits its access, because the harm of that code being wrong or manipulated is severe, and code that operates in a context where the harm is limited needs less. The least-privilege principle applies, limiting the sandboxed code’s access to the minimum the task requires, so the containment is as tight as the task allows, bounding the blast radius to the minimum. The deeper point is that executing agent-generated code is executing untrusted code, and the sandbox is what makes that safe, so the sandbox’s containment should match the risk of the code it contains, tighter for code that could cause more harm. Matching the sandbox to the risk, containing the untrusted code as tightly as the task allows, is what makes executing agent-generated code safe in proportion to the harm the code could cause.

Where it almost broke

We operated an agent that generated and executed code, and the code was operating with more access than the containment warranted, because the code execution had been set up for the agent to accomplish its tasks without tightly limiting what the code could access and do. The generated code was untrusted, produced by the model and unreviewed, and it was operating with access that would have let wrong, harmful, or manipulated code affect systems beyond the task, which was a security risk, because the agent’s code-generation capability was executing untrusted code with access that was not contained. No harm had occurred, and the exposure was serious, because the untrusted code could have caused harm across the access it had if it had been wrong or manipulated.

The correction sandboxed the code. We operated the generated code in an isolated environment that limited its access to only what the task required, contained its effects within the sandbox, and limited the resources it could consume, so that wrong, harmful, or manipulated code was contained rather than affecting the broader system. We matched the sandbox’s containment to the risk, tightly limiting the access of code that could cause more harm. The agent’s code-generation capability became safe to operate, because the untrusted code was contained in the sandbox rather than operating with unbounded access, so its blast radius was bounded to what the sandbox permitted. The lesson was that agent-generated code is untrusted code, and operating it requires sandboxing that contains it, matched to the harm the code could cause.

Contain the code you cannot trust

The reframe that makes agent code generation safe is treating the generated code as the untrusted code it is and containing it in a sandbox, rather than trusting code you did not review from a non-deterministic model. Sandboxing agent-generated code contains code that could be wrong, harmful, or manipulated by isolating it and limiting its access, actions, and resources, which bounds the harm the untrusted code can cause to the sandbox. Executing agent-generated code without sandboxing is executing untrusted code with the execution’s access, which is a security risk.

The teams that use agent code generation safely are the ones that sandboxed the generated code, contained it in an isolated environment that limited what it could access and do, and matched the containment to the harm the code could cause. Agent-generated code is unreviewed code from a non-deterministic model that could be wrong or manipulated, and operating it is executing untrusted code that a sandbox exists to contain. Containing the code you cannot trust, as tightly as the task allows, is what makes the agent’s powerful code-generation capability safe to operate, because the power and the risk are the same generated code, and the sandbox is what bounds the risk while keeping the power.