Container-ness is the wrong question

agentfs gives an agent a writable view of your project without letting it touch the original. Until recently that was always a FUSE server we wrote: your files as a read-only lower layer, an upper layer catching every write, snapshots taken from the upper.

It works, and on the inner loop it costs a lot. An incremental cargo build through the sandbox ran 2.21× slower than the same build outside it. The kernel has had its own overlay filesystem for a decade. The question was when we could use it instead.

The obvious probe is wrong twice

The intuition is that overlayfs needs privilege, so it works on a host and not in a container — check /.dockerenv or /proc/1/cgroup and branch. Both halves of that are false.

• Default Docker cannot mount overlayfs at all. No CAP_SYS_ADMIN, no mount. So container-ness is not sufficient to rule it out — it correctly predicts this case and nothing else.
• A plain host can, with no container anywhere in sight. An unprivileged user namespace holds CAP_SYS_ADMIN over itself, and the kernel has allowed overlayfs inside one since 5.11. We already create a user namespace for every rootless run, so we already had the privilege we thought we lacked. This is a user-namespace feature, not a container feature.
• --cap-add SYS_ADMIN is still not enough when upperdir is itself on overlayfs — which is exactly where a container’s ~/.agentfs lands by default. EINVAL. You may not stack an upper on an overlay.

That third case is the one that kills the heuristic outright. The capability is present, the container is privileged, and the mount still fails — because the constraint was never about privilege. It was about which filesystem the upper directory happens to sit on.

As a bonus, the detection itself has rotted: on cgroup v2, /proc/1/cgroup says 0::/ and nothing more.

So don’t ask. Try it.

The probe performs the real mount, in a throwaway namespace, against scratch directories inside the project directory — not /tmp. That last detail is the whole point: the constraint is a property of the filesystem the upper lives on, so probing somewhere else answers a different question convincingly.

If the mount succeeds, we tear it down and use the kernel backend. If it fails for any reason, we fall back to FUSE and say why.

What it bought

Measured end to end through a full agentfs run, so the numbers include namespace setup, the bind set and the mount:

task	native	fuse	kernel
cargo build (incremental)	0.824 s	1.819 s	0.839 s
grep -r	11.6 ms	77.1 ms	13.4 ms
git status (dirty)	8.3 ms	26.4 ms	9.6 ms
git add + commit	10.2 ms	50.1 ms	27.7 ms

The inner loop goes 2.21× → 1.02×: on the kernel backend, building inside the sandbox costs what building outside it costs. grep -r collapses the same way, 6.67× → 1.16×.

What is left is git add + commit at 2.72×, and it is structural rather than incidental. Git renames every object into its final place, and a rename across overlay layers is a copy-up. That is the one primitive where layering itself has a price.

Two things we had to get right

The mount happens inside the sandbox’s own mount namespace, not on the host. That is what makes it work without root, and it has a second effect worth more than the first: the workspace is invisible from outside. There is no mountpoint for a stray editor to write through, and nothing left to unmount if the sandbox dies.

-o userxattr, always — in both privilege modes. Left to itself the kernel uses trusted.overlay.* when privileged, and a later host-side restore running as an ordinary user cannot write those. One dialect everywhere beats matching the kernel’s default.

The cost of asking

The probe runs on every invocation and adds ~2.6 ms. It does not pay for itself at startup — we published the opposite claim first and had to take it back — it pays a second later, on the first real filesystem work.

One implementation detail that cost an afternoon: the probe forks from a process that is already multi-threaded, so everything the child touches has to be built before the fork. A malloc in the child of a threaded fork deadlocks if another thread happened to hold the allocator lock at fork time. The bug does not reproduce under a debugger, naturally.