I would guess OpenAI Codex and Claude Code are well into the millions subscriber range at this point. I would venture to guess the majority of them run in yolo mode. I have only seen a few horror stories on reddit. The same way any time you drive a car you can crash and die (many times through no fault of your own).
All that said, no way in hell I’m giving either access to production databases or environments.
I have heard of more than a few horror stories including filesystems lost and force pushes done.
These tools have only been in use for a short time and the current harnesses/system prompts are quite limited. Claude code is mostly limited to your codebase where you have version control. Excel is different.
I foresee that once people hand over more power to full agents there will be some nasty surprises. Im sure there will eventually be demand for some kind of limits
It's a variation of sandboxing which is a great idea. Even just using a separate user account on your laptop provides some useful level of isolation (as long as you don't give it sudo privileges). AI tools of course do some sandboxing of their own. It's just that the constant nagging for permissions causes people to negate most of that by giving very broad access outside the sandbox.
The downside for me and the main reason I do use vms less than I did a few months ago is that I need my agentic coding tools to use development tools a lot. And those tools need a lot of resources. And I have those resources on my laptop. Which is a nice mac book pro with plenty of RAM and 16 CPUs. I can run vms on this thing without issues of course. But tools just run a lot faster when I run them outside those VMs. And agentic coding tools run builds all the time. We're talking some really non trivial time savings here. Watching qemu build a thing for 10 minutes that I know should build in 45 seconds is painful. Especially if it happens over and over again.
The trick is doing sandboxing without performance impact. And very soon you'll also want to be able to run local models. I've been toying with the latest qwen and gemma models on my laptop. I haven't gotten around to doing coding with those just yet. But apparently they aren't completely horrible at it. That won't work on most cloud based vms. Unless you get a really big and expensive one. You could actually make that work if you only use them for a few minutes.
OrcaBot does this with the VM but whereas the author mentions the risk of GitHub keys being leaked, OrcaBot uses a key broker to ensure the LLM doesn’t have access to any keys. It even works on the API keys to the LLMs themselves.
https://orcabot.com/blog#breaking-the-lethal-trifecta
This is what dev.exe is great for, and it comes with Pi and its own coding agent Shelley preloaded. Also it’s incredibly fast and you can easily access the VMs from your phone.
Generally a good idea, but I'm not sure why you should even want to fork a git repo when a local clone should be sufficient. But this is probably a terminology mixup from the way github presents forks and clones.
I believe the author's idea is to do dev work from a Github account that only has access to the fork, but not to the main repo. Then, as a contributor, you'd open PRs from your fork to the main repo. I think this would only work if your Github account doesn't have write access to the main repo, though. I know you can use 'deployment keys' to give read-access to a single repo using an SSH key, but not sure if you can otherwise restrict access to a single repo with write access. Essentially, though, you'd want to find a way to give the remote host the most limited possible privileges to your Github account.
You could also just set the development machine up as a remote on the repo on your local host and then pull, diff, and merge locally. Then the llm agent doesn’t have access to any github account at all.
I use an overlay copy of my workdir, then the sandboxed LLM doesn't get any of my secrets, can do its own commits, and I pull the ones back that I want.
Oh, a separate GitHub account that has its own forks of the repos the agent is working on. Yeah, that's probably the most secure, isolated, and safest. The merge to the canonical repo then needs to go through a human, or at least separately controlled, process via a GitHub pull request.
I’m building a claw / vibe coding platform that’s business safe at https://housecat.com and am also finding all the old Unix tricks working as well as ever…
- user and home directory for data
- crontab for scheduled jobs
- cgi for serving user space apps
- rsync for backups
We even rediscovered email patches but with agent to agent help making and applying them.
It’s simpler for us to operate and the agent to figure out.
# Create a new sandbox copying . as workdir (default container, but you can choose vm)
yoloai new mybugfix . --isolation vm
# attach to it (it has tmux already)
yoloai attach mybugfix
# Chat with the bot inside...
# Happy with its work? Diff it to be sure
yoloai diff mybugfix
# Happy with the changes? Apply them to your workdir
yoloai apply mybugfix
# All done? Destroy the sandbox
yoloai destroy mybugfix
The agent stays isolated at all times. No access to your secrets (except what you want), no access to your workdir until you apply. You can also easily restrict network access.
This does the same thing as in the blog post, except that there are a LOT of gotchas and minutiae and some yak shaving involved if you want to keep doing it manually.
I've gone through the whole path the author has, and finally had to admit that it's too much fiddling around to do it manually. Easier to just have a cmdline tool that does it for you. That's why I built it in the first place.
It actually runs git (with hooks disabled) to generate the diff. It happens on the host when using copy mode, and inside the sandbox when using overlay mode.
The above example doesn't specify workdir mounting mode, so it would be copy, not overlay.
IM new to Claude code but doesnt auth require a gui browser to authenticate the Claude session first time login?? Do you have to setup a desktop environment just for that?
You can copy your claude credentials into the VM and run off that. Just beware that the subscription credentials file expires every half hour and then the agent tries to refresh which is annoying (especially if you have multiple sandboxed agents), so the better way is to get a long-running subscription API key (no extra cost for that) and just pass it in.
If the Claude (or similar) can't open a browser on a headless server, they typically print a URL you can copy to your browser on your local system-with-GUI. From there you authenticate and get back some kind of token, which you copy and paste back into your remote SSH session.
Ten more years and our tricks become the ancient practices and rituals of the olden hack0rs; stuff like apt update and apt upgrade on a fresh server. Some of the most ancient even use apt-get, while the really old ones will scold you for using Ubuntu
I would guess OpenAI Codex and Claude Code are well into the millions subscriber range at this point. I would venture to guess the majority of them run in yolo mode. I have only seen a few horror stories on reddit. The same way any time you drive a car you can crash and die (many times through no fault of your own).
All that said, no way in hell I’m giving either access to production databases or environments.
I have heard of more than a few horror stories including filesystems lost and force pushes done.
These tools have only been in use for a short time and the current harnesses/system prompts are quite limited. Claude code is mostly limited to your codebase where you have version control. Excel is different.
I foresee that once people hand over more power to full agents there will be some nasty surprises. Im sure there will eventually be demand for some kind of limits
It's a variation of sandboxing which is a great idea. Even just using a separate user account on your laptop provides some useful level of isolation (as long as you don't give it sudo privileges). AI tools of course do some sandboxing of their own. It's just that the constant nagging for permissions causes people to negate most of that by giving very broad access outside the sandbox.
The downside for me and the main reason I do use vms less than I did a few months ago is that I need my agentic coding tools to use development tools a lot. And those tools need a lot of resources. And I have those resources on my laptop. Which is a nice mac book pro with plenty of RAM and 16 CPUs. I can run vms on this thing without issues of course. But tools just run a lot faster when I run them outside those VMs. And agentic coding tools run builds all the time. We're talking some really non trivial time savings here. Watching qemu build a thing for 10 minutes that I know should build in 45 seconds is painful. Especially if it happens over and over again.
The trick is doing sandboxing without performance impact. And very soon you'll also want to be able to run local models. I've been toying with the latest qwen and gemma models on my laptop. I haven't gotten around to doing coding with those just yet. But apparently they aren't completely horrible at it. That won't work on most cloud based vms. Unless you get a really big and expensive one. You could actually make that work if you only use them for a few minutes.
You can run inside of a tart vm which gives you a virtual mac. It's pretty speedy once it's up and running.
OrcaBot does this with the VM but whereas the author mentions the risk of GitHub keys being leaked, OrcaBot uses a key broker to ensure the LLM doesn’t have access to any keys. It even works on the API keys to the LLMs themselves. https://orcabot.com/blog#breaking-the-lethal-trifecta
I've been successful with getting incus running with this for sandboxing the opencode session. I plan on writing a blog post on it.
This might interest you (disclaimer my project. Well mostly Opus')
https://github.com/jgbrwn/vibebin
This is what dev.exe is great for, and it comes with Pi and its own coding agent Shelley preloaded. Also it’s incredibly fast and you can easily access the VMs from your phone.
Generally a good idea, but I'm not sure why you should even want to fork a git repo when a local clone should be sufficient. But this is probably a terminology mixup from the way github presents forks and clones.
I believe the author's idea is to do dev work from a Github account that only has access to the fork, but not to the main repo. Then, as a contributor, you'd open PRs from your fork to the main repo. I think this would only work if your Github account doesn't have write access to the main repo, though. I know you can use 'deployment keys' to give read-access to a single repo using an SSH key, but not sure if you can otherwise restrict access to a single repo with write access. Essentially, though, you'd want to find a way to give the remote host the most limited possible privileges to your Github account.
You could also just set the development machine up as a remote on the repo on your local host and then pull, diff, and merge locally. Then the llm agent doesn’t have access to any github account at all.
I use an overlay copy of my workdir, then the sandboxed LLM doesn't get any of my secrets, can do its own commits, and I pull the ones back that I want.
On a GitHub project, agents must just be considered untrusted external contributors.
Oh, a separate GitHub account that has its own forks of the repos the agent is working on. Yeah, that's probably the most secure, isolated, and safest. The merge to the canonical repo then needs to go through a human, or at least separately controlled, process via a GitHub pull request.
They mention that as a mechanism for protecting the SSH keys for the repo.
Essentially using a repo that doesn’t matter with the coding agent and then creating a cross-repo PR to the real repo.
I’m building a claw / vibe coding platform that’s business safe at https://housecat.com and am also finding all the old Unix tricks working as well as ever…
- user and home directory for data
- crontab for scheduled jobs
- cgi for serving user space apps
- rsync for backups
We even rediscovered email patches but with agent to agent help making and applying them.
It’s simpler for us to operate and the agent to figure out.
This is what yoloAI does. Automatically.
The agent stays isolated at all times. No access to your secrets (except what you want), no access to your workdir until you apply. You can also easily restrict network access.https://github.com/kstenerud/yoloai
Spammy ai-generated self promotion.
In what way?
This does the same thing as in the blog post, except that there are a LOT of gotchas and minutiae and some yak shaving involved if you want to keep doing it manually.
I've gone through the whole path the author has, and finally had to admit that it's too much fiddling around to do it manually. Easier to just have a cmdline tool that does it for you. That's why I built it in the first place.
The part that worries me here is the diff. Does it happen in the host or in the guest? What code gets run when you run `yoloai diff`?
It actually runs git (with hooks disabled) to generate the diff. It happens on the host when using copy mode, and inside the sandbox when using overlay mode.
The above example doesn't specify workdir mounting mode, so it would be copy, not overlay.
If it runs inside the sandbox and the guest is compromised, can't the guest just lie?
Absolutely. That's why overlay is not the default.
That's... uh, an interesting approach to security.
What is? Defaulting to the most secure method?
IM new to Claude code but doesnt auth require a gui browser to authenticate the Claude session first time login?? Do you have to setup a desktop environment just for that?
You can copy your claude credentials into the VM and run off that. Just beware that the subscription credentials file expires every half hour and then the agent tries to refresh which is annoying (especially if you have multiple sandboxed agents), so the better way is to get a long-running subscription API key (no extra cost for that) and just pass it in.
If the Claude (or similar) can't open a browser on a headless server, they typically print a URL you can copy to your browser on your local system-with-GUI. From there you authenticate and get back some kind of token, which you copy and paste back into your remote SSH session.
the old hacker trick of using ssh
Ten more years and our tricks become the ancient practices and rituals of the olden hack0rs; stuff like apt update and apt upgrade on a fresh server. Some of the most ancient even use apt-get, while the really old ones will scold you for using Ubuntu
Hey! I resemble that comment :-P (I still tend to use apt-get out of habit...)
...and proxy your llm traffic with your own per session tokens – if it leaks, it doesn't matter, not a real token.
"Vibecoding?"
Gotta love how someone downvoted this.