Thanks for the kind words, and I'm delighted you found some value in the talk.
A few thoughts:
You might be interested in version 3 of [Elixir's Jido framework](https://jido.run/) as another harness that could plug into your OS. I'm not sure if v3 is public, but Mike Hostetler has made some comments in "The Swarm: Elixir AI Collective" Discord that leads me to believe it's coming soon.
I've read your blog post a couple of times and think it's intriguing. One thing that feels like an unsolved problem (at least for me) is being able to clearly separate the responsibilities of the harness from the underlying OS. I've frequently dreamt of an "ultimate harness" that could function as an OS for agents, but your Unix and `libc` analogy has caused me to reevaluate this.
I don't necessarily agree with you that the harness only solves the problem of one agent, unless I've misunderstood your point, or unless what you're saying doesn't apply to harnesses built on the BEAM. In my view, a useful harness could/should be capable of orchestrating virtually any number of agents. And the control plane for that harness (ideally) should make it easy to manage that level of orchestration.
Accounting and Identity are fascinating to think about, especially for scenarios where large organizations may be running millions of concurrent agents. The authors of the [Agentic Mesh](https://www.oreilly.com/library/view/agentic-mesh/9798341621...) book discuss this in their book as well as their eponymous podcast. According to them, we'd benefit from building components of agentic systems that effectively function like HR.
I'm attempting to build that kind of functionality into my "ultimate harness", and I think it's possible, but with your Unix and `libc` analogy in mind I'm looking forward to challenging some of the assumptions I've made up to this point.
In any event, it would be nice to grab a virtual coffee and talk shop. I'll send you an email shortly.
-Charles
p.s. I don't know if you're in the Elixir Slack group, but the chat in the `#ai-engineering` channel is frequently lively, and if you were interested, there are probably some people who'd love to see what you're building and engage with you. I know at least one person who's posted a bit about Zig in that channel, so discussion there isn't limited to just BEAM-related stack.
I think this article points out a bunch of ways in which a persistent agent orchestrator does directly analogous things to an OS.
I'm building a kind of orchestrator and I haven't thought of the OS analogy in a serious way other than the interface, but indeed I do already have scheduling, isolation, observability and identity, and I'm currently building a filesystem (versioned project-wide agentic artifacts).
Maybe I really do ought to call my project an "agentic desktop OS" and it isn't a misnomer after all.
I am also pursuing something similar (complementary?) to this (recently started writing a Rust-based "distributed OS" that manages ML resources in a network of heterogenous systems: varied cores, system RAM, GPU VRAM, I/O). So my focus is not so much distributed agents, but more distributed inference (and fine tuning) that would benefit agents, distributed or not. I may blog about my work soon.
Thanks, softwarewright; would love to hear more about your work. I've been implementing small-model inference in Zug inside WunderOS, but it's not done yet, so nothing public just yet.
I'm in a research/prototyping phase with no truly verified results yet, but I have all the hardware I need to do actual testing. I have an OS that boots in a VM, it just isn't verified to have practical value yet. It could be an AI coding agent's fever dream until I use it anger. So far Apple Silicon only, but most of my systems run Linux and most of my GPUs are NVIDIA, so moving development from Mac to Linux soon. It does have tests, demos, docs. Iterating on it daily.
I should clarify that by Apple Silicon I mean it boots Rust no_std on ARM. Does not use GPU yet. Plan is to use Rust without CUDA libraries. This project is more likely to use an NPU on a ARM dev board before it can use an NVIDIA GPU, and might never be able to use Apple GPUs. Goal: run on no-longer-supported by CUDA GPUs.
Good stuff here, much of which resonates with me. FWIW, my notion of an OS for agents isn't as close to the metal yet, ie, my view of agent lifecycle is that they look a lot like WhatsApp or Discord traffic. So, based on that DEEP analysis, I decided to build on BEAM/OTP for the control plane: basically, Elixir for the UX and Gleam for everything else. And the data plane is pure Zig: data sidecar into BEAM/OTP via NIF.
So that gives me a certain freedom for deploy: bare metal, containers, VMs, even K8S. And since there are some tools for pickling all that into a single binary, and running BEAM/OTP on u-kernel sorts of things, I can get all the way to the metal in the way that you are. Whether or when that happens remains to be seen, etc.
Interesting link/content and it seems complimentary.
One thing that is missing from both of our approaches is the ability re-train (fine-tune) coding models "overnight" so that they can "learn" from the prior day and changes since their training cutoff date.
I have found some things I can do to improve my work based on this, thanks.
_Pentad idea_ -/- _MLOS relevance_ -/- _Action_
Closed autonomic loops -/- Very high -/- Adopt architecture vocabulary
Deterministic replay -/- Very high -/- Strengthen event/replay contract
Model minimalism -/- Very high. -/- Extend later to compute-placement ladder
Durable vs active population -/- High -/- Define registered vs resident capacity metrics
Standing queries -/- High -/- Future policy/watch abstraction
Provenance by construction -/- High -/- Record policy decision causality
No model/NLP in hot path -/- High -/- State explicitly as invariant
Thanks for the kind words, and I'm delighted you found some value in the talk.
A few thoughts:
You might be interested in version 3 of [Elixir's Jido framework](https://jido.run/) as another harness that could plug into your OS. I'm not sure if v3 is public, but Mike Hostetler has made some comments in "The Swarm: Elixir AI Collective" Discord that leads me to believe it's coming soon.
I've read your blog post a couple of times and think it's intriguing. One thing that feels like an unsolved problem (at least for me) is being able to clearly separate the responsibilities of the harness from the underlying OS. I've frequently dreamt of an "ultimate harness" that could function as an OS for agents, but your Unix and `libc` analogy has caused me to reevaluate this.
I don't necessarily agree with you that the harness only solves the problem of one agent, unless I've misunderstood your point, or unless what you're saying doesn't apply to harnesses built on the BEAM. In my view, a useful harness could/should be capable of orchestrating virtually any number of agents. And the control plane for that harness (ideally) should make it easy to manage that level of orchestration.
Accounting and Identity are fascinating to think about, especially for scenarios where large organizations may be running millions of concurrent agents. The authors of the [Agentic Mesh](https://www.oreilly.com/library/view/agentic-mesh/9798341621...) book discuss this in their book as well as their eponymous podcast. According to them, we'd benefit from building components of agentic systems that effectively function like HR.
I'm attempting to build that kind of functionality into my "ultimate harness", and I think it's possible, but with your Unix and `libc` analogy in mind I'm looking forward to challenging some of the assumptions I've made up to this point.
In any event, it would be nice to grab a virtual coffee and talk shop. I'll send you an email shortly.
-Charles
p.s. I don't know if you're in the Elixir Slack group, but the chat in the `#ai-engineering` channel is frequently lively, and if you were interested, there are probably some people who'd love to see what you're building and engage with you. I know at least one person who's posted a bit about Zig in that channel, so discussion there isn't limited to just BEAM-related stack.
I think this article points out a bunch of ways in which a persistent agent orchestrator does directly analogous things to an OS.
I'm building a kind of orchestrator and I haven't thought of the OS analogy in a serious way other than the interface, but indeed I do already have scheduling, isolation, observability and identity, and I'm currently building a filesystem (versioned project-wide agentic artifacts).
Maybe I really do ought to call my project an "agentic desktop OS" and it isn't a misnomer after all.
Isn't that what pewdiepie made?
https://odysseusai.dev/
No, if you look carefully these are very different things targeting very different use cases (enterprise-wide agents vs personal agent).
So I see your OS as a seL4 + semantic layer?
I am also pursuing something similar (complementary?) to this (recently started writing a Rust-based "distributed OS" that manages ML resources in a network of heterogenous systems: varied cores, system RAM, GPU VRAM, I/O). So my focus is not so much distributed agents, but more distributed inference (and fine tuning) that would benefit agents, distributed or not. I may blog about my work soon.
Your posted link is helpful, thanks.
Thanks, softwarewright; would love to hear more about your work. I've been implementing small-model inference in Zug inside WunderOS, but it's not done yet, so nothing public just yet.
I'm in a research/prototyping phase with no truly verified results yet, but I have all the hardware I need to do actual testing. I have an OS that boots in a VM, it just isn't verified to have practical value yet. It could be an AI coding agent's fever dream until I use it anger. So far Apple Silicon only, but most of my systems run Linux and most of my GPUs are NVIDIA, so moving development from Mac to Linux soon. It does have tests, demos, docs. Iterating on it daily.
https://github.com/sw-ml-study/sw-os-ml
I also have been implementing small model inference:
https://github.com/sw-ml-study/moe-microscope
I should clarify that by Apple Silicon I mean it boots Rust no_std on ARM. Does not use GPU yet. Plan is to use Rust without CUDA libraries. This project is more likely to use an NPU on a ARM dev board before it can use an NVIDIA GPU, and might never be able to use Apple GPUs. Goal: run on no-longer-supported by CUDA GPUs.
Good stuff here, much of which resonates with me. FWIW, my notion of an OS for agents isn't as close to the metal yet, ie, my view of agent lifecycle is that they look a lot like WhatsApp or Discord traffic. So, based on that DEEP analysis, I decided to build on BEAM/OTP for the control plane: basically, Elixir for the UX and Gleam for everything else. And the data plane is pure Zig: data sidecar into BEAM/OTP via NIF.
So that gives me a certain freedom for deploy: bare metal, containers, VMs, even K8S. And since there are some tools for pickling all that into a single binary, and running BEAM/OTP on u-kernel sorts of things, I can get all the way to the metal in the way that you are. Whether or when that happens remains to be seen, etc.
Thanks for sharing! See https://pentad.ai/PLRN for more about what I'm up to.
Interesting link/content and it seems complimentary.
One thing that is missing from both of our approaches is the ability re-train (fine-tune) coding models "overnight" so that they can "learn" from the prior day and changes since their training cutoff date.
I have found some things I can do to improve my work based on this, thanks.
_Pentad idea_ -/- _MLOS relevance_ -/- _Action_
Closed autonomic loops -/- Very high -/- Adopt architecture vocabulary
Deterministic replay -/- Very high -/- Strengthen event/replay contract
Model minimalism -/- Very high. -/- Extend later to compute-placement ladder
Durable vs active population -/- High -/- Define registered vs resident capacity metrics
Standing queries -/- High -/- Future policy/watch abstraction
Provenance by construction -/- High -/- Record policy decision causality
No model/NLP in hot path -/- High -/- State explicitly as invariant