I would really like to see a good local sandboxing solution in this space, something that is truly local-first. This is especially important since many coding models / agentic builders will eventually become lightweight enough to run them on-device instead of having to buy tokens and share user data with big LLM cloud providers.
Hey, we built coderunner[1] exactly for this purpose. It's completely local. We use apple containers for this (which are 1:1 mapped to a lightweight VM).
(sorry I didn't reply in-thread, I'm new to HN, re-posting response here:)
Exactly! The main local requirement is to have hardware virtualization available (e.g. /dev/kvm) but that should be fine on your local linux machine. Won't work in cloud machines or on Mac ARM in current form but maybe if I extend
Are there any such solutions that can adequately protect against side-channel attacks (à la rowhammer, meltdown, spectre, ...)? I mean protecting local file access and network access is pretty easy, but side-channels and VM escaping attacks seem like a bigger concern.
That's an interesting direction! TEE support would be relatively straightforward with current stack (and it's on my roadmap), so that could be a first step forward.
That seems like a naive take. If any of your local VMs are internet connected and are compromised, side channel attacks could be used to exfiltrate data from other VMs or the host.
Yes, blocking DNS exfiltration requires DNS filtering at cluster level. This is what will be added with the Cilium integration which is top-3 on the roadmap (top of readme).
DNS resolution is required for basic Kubernetes functionality and hostname resolution within the cluster.
That's said explicitly in several places in the docs: "DNS to CoreDNS allowed"
One thing I could do is make it exposed in config, to allow the user to block all DNS resolutions until Cilium is integrated. LMK if desired!
> One thing I could do is make it exposed in config, to allow the user to block all DNS resolutions until Cilium is integrated. LMK if desired!
Yes, but it's not great for it to be an optional config option. Trivially easy to use data exfiltration methods shouldn't be possible at all in a tool like this, let alone enabled by default.
I want to recommend ppl to try this out and not have to tell them about the 5 different options they need to configure in order for it to actually be safe. It ends up defeating the purpose of the tool in my opinion.
Some use cases will require mitmproxy whitelists as well, eg default deny pulling container image except matching the container whitelist.
as someone in the space this ticks a lot of boxes: kubernetes-native, strong isolation, python sdk (ideal for ML scenarios). devmapper is a nice ootb approach.
Glancing at the readme, is your business model technical support? Or what's your plan with this?
Anything interesting to share around startup time for large artifacts, scaling, passing through persistent storage (or GPUs) to these sandboxes?
Curious what things like 'Multi-node cluster capabilities for distributed workloads' mean exactly? inter-VM networking?
No business model short-term. My goal is broad adoption, 100% open-source.
By multi-node I mean so far I only support 1 k8s node, i.e. 1 machine, but soon adding support for multiple. Still, on 20 CPUs I can run +50 VM pods with fractional vCPU limits.
For GPU passthrough: not possible today because I use Firecracker as VMM. On roadmap: Add support for Qemu, then GPU passthrough possible.
Inter-VM networking: it's already possible on single-node: 1 VM = 1 pod. Can have multiple pods per node (have a look at utils/stress-test.sh). Right now I default deny-all ingress for safety (because by default k8s allows inter pod communication), but can make ingress configurable.
Startup time: a second, or a few seconds, depending on which base image (alpine, ubuntu, etc...) and whether you use a before_script or not (what I execute before the network lockdown)
Large artifacts: you can configure resource allocated to a VM pod in the sandbox config and it basically uses k8s resource limits.
> No business model short-term. My goal is broad adoption, 100% open-source.
IMHO that's kind of a red flag. There's a happy path here where it's successful but stays low-maintenance enough that you just work on it in your spare time, or it takes of and gets community support, or you get sponsorships or such. But there's also an option where in a year or two it becomes your job and you decide to monetize by rug-pulling and announce that actually paying the bills is more important than staying 100% open source. Not a dig at you, just something that's happened enough times that I get nervous when people don't have a plan and therefore don't have a plan to avoid the outcome that creates problems for users.
Sure one day if it really kicks off I could think of offering additionally a SaaS solution with paid enterprise features like SOC 2 compliance, RBAC, multiple clouds supported, etc. Why not. But I strongly believe that for it to be successful, it needs a strong open-source base. Then, billing huge companies for compliance features or huge usage makes sense. That would support development of the open-source part too.
I like the Docker model, for instance: free for companies under 250 employees or $10m/y revenue.
In any case, it will always be open-source.
Those paid enterprise features wouldn't come from closed-source: they would come from compliance of a particular SaaS-offered infra setup, that anybody else could reproduce. Just like HuggingFace.
Great project. There's been multiple approaches/tools in the space(top of my head I can think of e2b, arrakis, claude's new tool). how is this different?
This seems like an amazing stack that ticks a lot of boxes for me, but I really dislike cli or a custom api as the UX for this and would prefer to manage all of this with CRDs so i can just use the k8s client for everything.
Actually you can! After you run "k7 install" you'll have a k3s cluster up and running, with Kata as a runtime class, and Firecracker specified in Kata config. So nothing prevents you from hitting the Kubernetes API! kubectl will work.
Note: I use k3s' internal kubectl and containerd, to avoid messing with your own if you have some already installed. That means you can run commands like "k3s kubectl ..."
If you have any suggestion on how I can make this more friendly UX-wise to your personal usage, I am most interested to hear! And this will shape my roadmap.
It is well known that containers do not provide you safe isolation. It is not their purpose. They share kernel and page cache with the host. Any kernel exploit gives to someone in a container potential root control of the host (see DirtyPipe, DirtyCow). That's why you need VM-level isolation.
From an outside perspective, this looks silly. Like fitting a square peg in a round hole. But I do ack "what if we could run vm's as easily as we run containers" use case and atm it seems like things like this (and katacontainers) are the only ways to do it. Wondering a few things: do all the layers of abstraction make things brittle and how is performance impacted?
It uses Kata with Firecracker which gives you as light of a boot as it gets. Subsecond booting for instance is accessible with a lighter rootfs, which is also on the roadmap (one of the easiest items, actually). The k8s layer doesn't add overhead either compared to any other VM. If you want to compare to bare containers, depending on workload, you could see a 5% overhead due to virtualization. Exact overhead would depend on workload.
I would really like to see a good local sandboxing solution in this space, something that is truly local-first. This is especially important since many coding models / agentic builders will eventually become lightweight enough to run them on-device instead of having to buy tokens and share user data with big LLM cloud providers.
Anthropic recently released a sandboxing tool based on bubblewrap (Linux, quite lightweight) and sandbox-exec (macOS). https://github.com/anthropic-experimental/sandbox-runtime
I wonder if nsjails or gVisor may be useful as well. Here's a more comprehensive list of sandboxing solutions: https://github.com/restyler/awesome-sandbox
wow that's super new! Thanks for that, will look deeply into it and compare
> something that is truly local-first
Hey, we built coderunner[1] exactly for this purpose. It's completely local. We use apple containers for this (which are 1:1 mapped to a lightweight VM).
1. Coderunner - https://github.com/instavm/coderunner
https://rstrict.cloud/ is a CLI built in Rust on top of the Landlock API for the Linux kernel.
It lets you narrow the permission scope of an executable using simple command line wrappers.
What about this: https://github.com/apple/container
(sorry I didn't reply in-thread, I'm new to HN, re-posting response here:)
Exactly! The main local requirement is to have hardware virtualization available (e.g. /dev/kvm) but that should be fine on your local linux machine. Won't work in cloud machines or on Mac ARM in current form but maybe if I extend
There are some providers that offer KVM nested virtualization, I think Google Cloud, Digital Ocean ... any others?
True! GCP does. I haven't tested it yet. I didn't know D.O does. If anyone knows others, I'm interested too!
We (NanoVMs) can run (both unikernel and normal linux) virtualized workloads on plain old ec2 instances (eg: t2.small).
Are there any such solutions that can adequately protect against side-channel attacks (à la rowhammer, meltdown, spectre, ...)? I mean protecting local file access and network access is pretty easy, but side-channels and VM escaping attacks seem like a bigger concern.
That's an interesting direction! TEE support would be relatively straightforward with current stack (and it's on my roadmap), so that could be a first step forward.
Side-channel attacks apply to multi-tenant cloud environments, not local.
That seems like a naive take. If any of your local VMs are internet connected and are compromised, side channel attacks could be used to exfiltrate data from other VMs or the host.
Then why only apply to VMs, why not apps?
Malicious code just has to resolve <secret>.evil.com and Google/CF will forward that query to evil resolver.
That's a config example.
Yes, blocking DNS exfiltration requires DNS filtering at cluster level. This is what will be added with the Cilium integration which is top-3 on the roadmap (top of readme).
DNS resolution is required for basic Kubernetes functionality and hostname resolution within the cluster.
That's said explicitly in several places in the docs: "DNS to CoreDNS allowed"
One thing I could do is make it exposed in config, to allow the user to block all DNS resolutions until Cilium is integrated. LMK if desired!
> One thing I could do is make it exposed in config, to allow the user to block all DNS resolutions until Cilium is integrated. LMK if desired!
Yes, but it's not great for it to be an optional config option. Trivially easy to use data exfiltration methods shouldn't be possible at all in a tool like this, let alone enabled by default.
I want to recommend ppl to try this out and not have to tell them about the 5 different options they need to configure in order for it to actually be safe. It ends up defeating the purpose of the tool in my opinion.
Some use cases will require mitmproxy whitelists as well, eg default deny pulling container image except matching the container whitelist.
as someone in the space this ticks a lot of boxes: kubernetes-native, strong isolation, python sdk (ideal for ML scenarios). devmapper is a nice ootb approach.
Glancing at the readme, is your business model technical support? Or what's your plan with this?
Anything interesting to share around startup time for large artifacts, scaling, passing through persistent storage (or GPUs) to these sandboxes?
Curious what things like 'Multi-node cluster capabilities for distributed workloads' mean exactly? inter-VM networking?
No business model short-term. My goal is broad adoption, 100% open-source.
By multi-node I mean so far I only support 1 k8s node, i.e. 1 machine, but soon adding support for multiple. Still, on 20 CPUs I can run +50 VM pods with fractional vCPU limits.
For GPU passthrough: not possible today because I use Firecracker as VMM. On roadmap: Add support for Qemu, then GPU passthrough possible.
Inter-VM networking: it's already possible on single-node: 1 VM = 1 pod. Can have multiple pods per node (have a look at utils/stress-test.sh). Right now I default deny-all ingress for safety (because by default k8s allows inter pod communication), but can make ingress configurable.
Startup time: a second, or a few seconds, depending on which base image (alpine, ubuntu, etc...) and whether you use a before_script or not (what I execute before the network lockdown)
Large artifacts: you can configure resource allocated to a VM pod in the sandbox config and it basically uses k8s resource limits.
Let me know if any other question! Happy to help
> No business model short-term. My goal is broad adoption, 100% open-source.
IMHO that's kind of a red flag. There's a happy path here where it's successful but stays low-maintenance enough that you just work on it in your spare time, or it takes of and gets community support, or you get sponsorships or such. But there's also an option where in a year or two it becomes your job and you decide to monetize by rug-pulling and announce that actually paying the bills is more important than staying 100% open source. Not a dig at you, just something that's happened enough times that I get nervous when people don't have a plan and therefore don't have a plan to avoid the outcome that creates problems for users.
Sure one day if it really kicks off I could think of offering additionally a SaaS solution with paid enterprise features like SOC 2 compliance, RBAC, multiple clouds supported, etc. Why not. But I strongly believe that for it to be successful, it needs a strong open-source base. Then, billing huge companies for compliance features or huge usage makes sense. That would support development of the open-source part too.
I like the Docker model, for instance: free for companies under 250 employees or $10m/y revenue.
In any case, it will always be open-source.
Those paid enterprise features wouldn't come from closed-source: they would come from compliance of a particular SaaS-offered infra setup, that anybody else could reproduce. Just like HuggingFace.
Thanks everyone for the amazing feedback and discussion!
For anyone curious:
– Docs: https://docs.katakate.org
- LangChain Agent tutorial: https://docs.katakate.org/guides/langchain-agent
It's getting late where I am, so I'm heading to bed — looking forward to replying to any new comments tomorrow!
Great project. There's been multiple approaches/tools in the space(top of my head I can think of e2b, arrakis, claude's new tool). how is this different?
Is this basically an open-source DIY version of E2B?
If so, cool! AFAICT E2B is open-source licensed but tricky to setup.
hey, I work at E2B, anything we can do to improve the setup for you?
I dig E2B, it's a great service and very cost effective. Thanks for all your hard work!
This seems like an amazing stack that ticks a lot of boxes for me, but I really dislike cli or a custom api as the UX for this and would prefer to manage all of this with CRDs so i can just use the k8s client for everything.
Actually you can! After you run "k7 install" you'll have a k3s cluster up and running, with Kata as a runtime class, and Firecracker specified in Kata config. So nothing prevents you from hitting the Kubernetes API! kubectl will work.
Note: I use k3s' internal kubectl and containerd, to avoid messing with your own if you have some already installed. That means you can run commands like "k3s kubectl ..."
And thank you for the compliments on the stack.
If you have any suggestion on how I can make this more friendly UX-wise to your personal usage, I am most interested to hear! And this will shape my roadmap.
Why do I need this if I already have containers and k8s for running agents?
It is well known that containers do not provide you safe isolation. It is not their purpose. They share kernel and page cache with the host. Any kernel exploit gives to someone in a container potential root control of the host (see DirtyPipe, DirtyCow). That's why you need VM-level isolation.
today i'm one of the lucky 10k https://xkcd.com/1053/
what does Katakana add on top of Kata?
From an outside perspective, this looks silly. Like fitting a square peg in a round hole. But I do ack "what if we could run vm's as easily as we run containers" use case and atm it seems like things like this (and katacontainers) are the only ways to do it. Wondering a few things: do all the layers of abstraction make things brittle and how is performance impacted?
It uses Kata with Firecracker which gives you as light of a boot as it gets. Subsecond booting for instance is accessible with a lighter rootfs, which is also on the roadmap (one of the easiest items, actually). The k8s layer doesn't add overhead either compared to any other VM. If you want to compare to bare containers, depending on workload, you could see a 5% overhead due to virtualization. Exact overhead would depend on workload.