AI agent sandboxes and auto mode: what they stop and miss

What checks does a command pass before it runs?

When a coding agent wants to run a command, the decision is not made in one place. The documentation of all five tools describes the same four layers: first the rules you write yourself, then the sandbox the operating system enforces, then a separate model that reviews the command, and finally you. We covered how to write rules with hooks in the hooks post. This post looks at the other three, with the vendors’ own measurements:

The vendors’ own data shows why a model was put in between. According to Anthropic, Claude Code users approve 97% of permission prompts. The same users reject 39% of the plans the agent presents, but for individual permission prompts the rejection rate is only 3%. It is like approving a pull request without reading it: the prompts come so often that people click through without reading.

An experiment in the same Anthropic post puts numbers on the gap. A single dangerous command was slipped into the sessions of 1,053 paid test users. In fact only the text of the prompt had been changed; even if they approved the command, nothing ran. People stopped this command 13.6% of the time, auto mode 89%. The longer the session, the more people missed: early on they stopped about 17% of dangerous commands, and after 50 or more prompts about 5%. These results come from the vendor’s own experiment.

OpenAI has seen the same problem among its own users. According to OpenAI’s post, when approval prompts come often, users either switch to Full Access mode, where no rules are enforced, or write broad rules that allow every command starting with python. According to Anthropic, in June 43% of active CLI users had an interpreter rule such as Bash(python:*). Both vendors settled on the same fix: put the question to a separate model instead of a person.

How do auto mode and auto-review decide?

A call that the rules and the sandbox do not settle goes to the model that grants permission, and that model decides whether it runs. All five tools now have such a model; but what it sees, when it hands the decision back to you and whether it is on by default differ from tool to tool. As of 20 October 2026:

Source: the tools’ documentation and release notes, 20 October 2026. “Not stated in the documentation”: in the pages we read that day, we found no sentence that describes this. Copilot’s judge came to the CLI on 7 July; VS Code’s documentation lists this permission level as experimental.

In Claude Code, the decision order is fixed: first your allow, ask and deny rules, then reads and edits in the working directory, and last the classifier. When you switch to auto mode, broad rules that allow running any code, such as Bash(*) or Bash(python*), are dropped; narrow rules such as Bash(npm test) stay. The classifier does not see tool output, so an instruction hidden in a file or a web page cannot steer it directly. When it blocks a call, the screen shows Denied by auto mode classifier, next to the name of the rule it matched, such as [Data Exfiltration].

Codex makes the opposite choice. Auto-review also sees tool output and, when needed, runs read-only checks of its own. But it only looks at calls that want to cross the sandbox boundary; a command that stays inside the sandbox is never reviewed. According to its documentation, this is not a permission, only a change of reviewer: auto-review does not widen the writable directories or turn on the network. The default policy sits in the open-source Codex repository, in a folder called guardian. If you write your own policy, it does not merge with the default; it replaces it. According to OpenAI’s post of 30 April 2026, the reviewer ran on GPT-5.4 Thinking at the time; the documentation does not name today’s model.

Cursor released Auto-review on 29 May and has offered it as the recommended default ever since. The classifier runs with small models on Cursor’s servers and can read files on your machine, read-only, to make its decision. In Copilot, the judge came to the CLI first. According to VS Code’s documentation, Assisted permissions is an experimental level: an LLM judge evaluates every tool call, and if the judge does not approve, you are asked. Antigravity CLI added Auto mode on 14 October: it approves file edits and safe commands without asking, while a separate agent first checks any command that would leave the sandbox. Antigravity’s modes page does not list this mode yet; the information is in the release notes.

If you still use Gemini CLI, its documentation describes no model that grants permission: its policy engine decides with fixed rules, and the first matching rule determines the outcome. Google stopped serving Gemini CLI to Google AI Pro and Ultra subscribers and free individual users on 18 June 2026 and pointed them to Antigravity CLI; you can keep using Gemini CLI with a paid API key. Watch out for a name clash too: in Copilot and Cursor, “Auto” is also the name of the router that picks a model. We covered that in the model selection post; the auto mode in this post is about permissions.

How often is the model that grants permission wrong?

The model that grants permission can go wrong in two ways: it lets a dangerous call through, or it stops a harmless call for nothing. In the first case the command runs; in the second the agent tries another way or asks you. Two vendors measure and publish both: Anthropic and OpenAI. We found no measurement in Cursor’s, GitHub’s or Google’s documentation. Anthropic compared people and the classifier on the same commands:

These numbers come from test sets; in real use the picture is less bright. In Anthropic’s engineering post of 25 March 2026, the classifier needlessly stopped 0.4% of harmless calls across 10,000 real tool calls by company employees. On the other hand, the agent went beyond what the user asked for in 52 real actions, and the classifier missed 17% of them. The post itself calls this figure “the honest number”. In most of the misses, the classifier saw the danger but assumed the user’s consent covered this action too: saying “Clean up the PR” is not permission to force-push. At the time the classifier ran on Sonnet 4.6; today the default model is Claude Sonnet 5.

OpenAI ran its own measurement on synthetic data. According to OpenAI, auto-review caught 90.3% of the cases where the agent went beyond what was asked. For prompt injection, the catch rate is 99.3% across three attack types; counting every type, deletion and data tampering included, it drops to 90.2%. In red-team work they found cases where auto-review could be tricked into approving a command without the user’s approval; they also write that they do not expect deterministic guarantees from this kind of system. In a synthetic attack test Anthropic ran with an outside team, the classifier’s miss rate fell from 12% to 7% after fixes; Anthropic asks that this figure not be taken as the miss rate on real traffic. We covered how to read a result a vendor reports in the SWE-bench post.

The documentation also spells out what the model cannot see. The model that grants permission only sees the call in front of it: in Codex, commands that stay inside the sandbox never reach it, and in Claude Code, neither do reads and edits in the working directory. OpenAI writes that a model hiding its intent could produce a harmless-looking request, or do things inside the sandbox that auto-review never sees. Claude Code’s classifier does not see the output of earlier commands. So when the target of an rm -rf is a shell variable whose value never appeared in the conversation, it cannot verify the deletion and blocks it. A limit you set in the conversation, such as “don’t push”, is not stored as a rule either; according to the documentation, that message can be lost when the context is compacted. We explained how that happens in the auto compact post. For a hard ban, write a deny rule.

The balance between a missed danger and a needless stop is the same as the balance between real bugs and false alarms in code review: a setting that reduces one increases the other. The vendors’ own words are clear too. Cursor says in a heading that Auto-review is not a security boundary; VS Code says the same of Assisted permissions. OpenAI writes that auto-review is not a deterministic security guarantee, and Anthropic that auto mode does not guarantee safety. It is the sandbox that draws the boundary.

What does a sandbox actually limit?

A sandbox limits the command the agent runs with the operating system’s own tools: Seatbelt on macOS, and bubblewrap, Landlock or kernel namespaces on Linux. Even if the model gets it wrong, the command cannot write to a file it is not allowed to, or connect to a server it is not allowed to. But the defaults of the five tools are very different:

Source: the tools’ sandbox documentation, 20 October 2026. Copilot’s local sandbox became generally available on 7 October. In Gemini CLI, too, you turn the sandbox on yourself, with a flag, an environment variable or a setting.

The most important column of the table is the last one. Claude Code’s documentation says the sandbox covers shell commands only: file tools such as Read and Edit follow the permission rules, and hooks and local MCP servers run with your full access. According to GitHub’s documentation, the operating system sandbox never sees what Copilot CLI’s own file tools do either; only a check in the software limits them. We covered why local MCP servers need extra care in the MCP security post.

Most recorded sandbox escapes come out of exactly this gap: a command inside the sandbox prepares something, and a part outside the sandbox runs it or writes through it. The NVD has three examples, all three fixed:

• Claude Code (CVE-2026-39861, before 2.1.64): a command in the sandbox could create a symlink pointing outside the working directory. When Claude Code, running outside the sandbox, wrote to that path, the file landed outside the directory. According to the record, neither could write outside the directory on its own; the escape happened when the two came together.
• Cursor (CVE-2026-26268, before 2.5): an agent steered by prompt injection could write to poorly protected .git settings and git hooks; git then ran those hooks on its own, outside the sandbox. Cursor now always blocks writes to .git/hooks and .git/config.
• Codex (CVE-2026-19591): the parser that picks out safe commands wrongly counted some PowerShell commands as safe, and a git command that writes to a file could run without asking for approval. According to the record, on macOS and Linux the default sandbox could still block writes to places that were not allowed: when the rule layer got it wrong, the sandbox could hold the boundary.

VS Code’s documentation also describes this weakness of the rule layer: rule-based auto-approval can only parse a command on a best-effort basis; quote concatenation or shell aliases can get around the rules. There is a known gap on the network side too. According to Claude Code’s documentation, the proxy decides based on the domain the client names, without inspecting TLS; so servers not on the list can be reached through techniques such as domain fronting. The same page warns that allowing a broad domain such as github.com can open a path for data exfiltration. The sandbox’s own bugs show up in the release notes too: version 2.1.224 fixed a bug in which a denyRead entry ending in a slash could be silently bypassed on Linux and macOS.

The tools also treat secrets differently. Claude Code has no built-in credential deny list; only the files and variables you list are restricted. If you want, you can mask a key: the command sees a per-session placeholder instead of the real value, and the proxy swaps in the real value only on requests to servers you allow. Claude Code does not read this setting from the repository’s .claude/settings.json, only from user or managed settings. Copilot uses the same method by default for Git and gh credentials when the sandbox is on; gh’s real token goes only to github.com addresses. Antigravity blocks ~/.ssh and .env from the start. GitHub’s documentation spells out what happens in Copilot when the sandbox is off: commands can read and write anywhere with your account’s access, reach any network your machine can reach and use your credentials without restriction.

Which mode should you use for which job?

All the evidence points the same way: the model that grants permission cuts the prompts that reach you, and the sandbox draws the boundary. Use the two together and pick the setting by the job:

For everyday work you are watching over, use the model that grants permission together with the sandbox. In Claude Code, auto mode is already the starting mode; turn on the sandbox too with /sandbox. In Codex the sandbox is already on; pick “Approve for me” in the permissions menu. In Cursor, leave Auto-review on. In Copilot CLI, turn on the sandbox with /sandbox enable. In Antigravity CLI, turn on the sandbox with --sandbox and choose --mode auto. If you want to see what the model blocks, Claude Code has written the classifier’s decision to OpenTelemetry events since 15 October, and Codex keeps auto-review transcripts under ~/.codex/sessions.

If the agent keeps asking for permission, adjust the boundary instead of opening everything up. That is also OpenAI’s advice: if ordinary requests keep landing in review, fix the sandbox boundary first, and write narrow rules such as ["cargo", "test"] instead of broad prefixes such as ["python"]. In Claude Code, if you do not want a classifier, use Manual mode together with the sandbox’s auto-allow mode; commands that stay inside the sandbox run without asking. Grant permission once, as a narrow rule, not every time the agent asks.

In unattended runs there is nobody to ask. Claude Code starts runs launched with -p in Manual unless you give a mode, and denies calls that need approval. Cursor’s cloud agents run on a machine of their own and never ask for approval. In such a run, keep the sandbox on, limit the network to a list of domains and put no keys in the environment that the run does not need. If you must give full access, do it only in a disposable container; there the container itself draws the boundary. Codex’s documentation writes that even in such a container, a malicious project can exfiltrate everything inside, Codex’s credentials included.

Some of these settings live in the repository and can change with a pull request. Claude Code does not let the repository’s settings file turn on auto mode; but unless an admin makes the sandbox mandatory, it reads entries that loosen the boundary, such as excludedCommands and network.allowedDomains, from that file. Its documentation writes that with a broad excludedCommands entry, the agent can write a file and run it outside the sandbox. Cursor merges the repository’s .cursor/sandbox.json with the user setting and gives the repository file priority; Codex loads the repository’s .codex/config.toml in a project you trust. A pull request that touches one of these files changes the boundary for everyone who opens the repository; review it like a security change. Before you open a repository you don’t trust with an agent, look at these files too.

We apply the same principle in our own product. In diffox, our AI code review tool, the step that maps a pull request’s blast radius reads the customer’s files and calls nobody outside. This step builds its environment from an allow-list: it takes no variable other than PATH, HOME, the locale settings and the temp directory. A deny list would let through a new key added tomorrow.

Frequently asked questions

What is Claude Code auto mode?

Auto mode is one of Claude Code’s permission modes: a separate classifier model answers permission prompts for you. Your allow, ask and deny rules apply first, and reads and edits in the working directory go through without asking; the remaining calls are reviewed by the classifier, which runs on Claude Sonnet 5 by default. You can change the model with CLAUDE_CODE_AUTO_MODE_MODEL. By default it blocks sending sensitive data out, production deploys and force pushes; after 3 blocks in a row or 20 in total in a session, it asks you again. On 14 August it became the starting mode on the Pro, Max and Team plans; since 28 September, interactive sessions on every plan start in this mode. Anthropic’s documentation says auto mode does not guarantee safety.

What does “Denied by auto mode classifier” mean?

It means Claude Code’s classifier found that call unsafe. The name in square brackets, for example [Data Exfiltration] or [Production Deploy], is the name of the rule it matched. You can see blocked calls in the Recently denied tab of /permissions, and press r to tell the agent it may retry that call. If what was blocked is a package registry or an internal domain the agent needs all the time, add it to autoMode.environment; /auto-mode-setup drafts these entries for you. If it is a one-off task, state your intent clearly in your next message.

What is the difference between Codex auto-review and full access?

Both cut approval prompts, but they put the boundary in different places. Auto-review (“Approve for me”) leaves the workspace-write sandbox as it is; it only sends calls that want to cross the sandbox to a separate reviewer agent instead of you. Full access means sandbox_mode = "danger-full-access" with approval_policy = "never": no sandbox and no prompts; the agent can reach the internet and every file on your computer. In OpenAI’s own measurement, auto-review caught 90.3% of synthetic scenarios where the agent crossed the boundary, and the session stopped for human approval about 200 times less often. Full access has no such check at all.

How do you turn on the Claude Code sandbox?

Type /sandbox in a session, or set sandbox.enabled to true in ~/.claude/settings.json. On macOS nothing else is needed; on Linux and WSL2, bubblewrap and socat must be installed. On native Windows, commands run without a sandbox. When it is on, commands can write to the working directory and the temp directory, and reach the network only at the domains you allow. The sandbox covers shell commands only; file tools, hooks and local MCP servers stay outside it. If you don’t want commands retried outside the sandbox, set allowUnsandboxedCommands to false.

Can the agent read ~/.ssh with the sandbox on?

It depends on the tool. By default, Claude Code’s sandbox can read most of the machine, ~/.ssh and ~/.aws/credentials included; closing that takes the filesystem.denyRead or credentials setting. In Cursor, the read boundary is “system” by default and ~/.ssh can be read; if you choose “workspace”, only known_hosts stays readable in ~/.ssh. Copilot’s sandbox closes every path on the file system that you have not allowed, but with the default developer-tool access, commands can read the tokens package managers store. Antigravity blocks ~/.ssh and .env from the start. Codex’s documentation does not state the default read boundary clearly; it recommends a deny rule for paths such as ~/.ssh.

Why does Claude Code keep asking for permission?

Usually because the session started in Manual mode. Since 28 September, Claude Code has started interactive sessions in auto mode; but if defaultMode in your user settings points to another mode, if your model does not support auto mode, or if your organization has turned auto mode off, the session starts in Manual. You can switch to auto mode with Shift+Tab. Auto mode still asks you on ask rules such as Bash(git push *) and on the first read outside the working directory. If you don’t want a classifier, turn on the sandbox in auto-allow mode; commands inside the sandbox run without asking. Don’t write a rule like Bash(*) that opens everything.

Does auto mode cost extra?

It depends on the tool. Since 7 August, Anthropic has not charged for the classifier’s extra tokens on the Pro, Max and Team plans. On the Enterprise plan and on accounts that use the Claude API, Bedrock, Google Cloud or Foundry, classifier calls count toward token usage; when the review happens on the server side, there is no separate call. Cursor writes that the Auto-review classifier adds no extra charge on top of agent usage. Codex’s, Copilot’s and Antigravity’s documentation does not state what the model that grants permission costs.