Engineering notes from the team building an open-source agent runtime — sandbox design, LXC and Incus internals, container isolation, eBPF network policy, and honest comparisons of the tools in this space. We write up what actually broke, not just what shipped.
Kilo gives one excellent coding agent access to 500+ models. Containarium lets Claude, Codex and Gemini agents work on one job, each in its own box with its own credentials, as a pipeline or a parallel task graph. What each is for, and where we're still early.
"We're six people, we have nobody to run a cloud account" is a fair objection to yesterday's post, and the answer isn't "go hire an ops team." Owning something and running it are two different jobs. The four things you never hand over, the four tests of a healthy delegation, and how to move out without an IT department.
The official statement says only environment variables were exported; the rumors say far more. You can verify neither — and that verification gap is what you signed up for when you handed over custody. On trust vs. verify vs. custody, and the four rules of real BYOC.
A PaaS platform leaked tenant environment variables — API keys in the open. The leak path in these incidents isn't lateral movement between clusters; it's one secret store scraped for every tenant at once. What to do in the first hour, why firewalls can't stop it, and the seven rules of a security boundary we build by.
Containarium scored half of upstream agent-sandbox's density on Kubernetes — 186 vs. 373. We root-caused it to a real gap in our own CLI, shipped a one-flag fix, and re-ran the exact same comparison: exact match, 373 vs. 373.
Turn on gVisor and kubectl port-forward stops reaching the pod —
kubectl exec, it turns out, doesn't. We hit the gap independently, then found
our own SSH gateway was ALSO broken, for two unrelated reasons. What broke, what we got
wrong the first time, and what's proven end to end now.
"Untrusted" is three different threats, and most teams defend the wrong one — shopping for isolation technology while running with unrestricted outbound network access. What a container boundary actually buys you, what it doesn't, and the case where our own product is the wrong tool.
Everyone uses a coding agent to write code; almost nobody talks about how to scale one up. Two things made it work for us — a software quality process the agents actually follow, and one isolated Linux box per agent. Nobody contaminates anybody, and nothing blows up your laptop.
Containarium, E2B, Modal, and Daytona side by side on the things that actually decide the choice: self-hosting, persistence, isolation technology, license, MCP support, and cost. Written by a vendor in the space, so it says plainly where each competitor wins.
More posts are on the way — LXC versus microVMs for agent workloads, and what we found taking a debugger to a liblxc deadlock in production. Subscribe by RSS to get them.
Self-host the open source on your VM, or get a seat on the hosted cloud.