Streaming AI agent desktops with gaming protocols

(blog.helix.ml)

39 points | by quesobob 10 days ago ago

19 comments

  • _pdp_ an hour ago

    IMHO, the goals is not to have to watch what agents do and let them do the work.

    I would personally invest into making agents more autonomous (yes hard problem today) then building a desktop video session protocol to watch them do the work.

  • lewq an hour ago

    Author of helix code here. Here's a demo of the full system working. https://youtu.be/vVmnpcnLDGM?si=b6LxW6lmM7843LY0

    We're opening the private beta where we provide a hosted environment for testing, or you can install the latest Helix release and run the installer with --code to try it on your own GPUs

  • momocowcow 30 minutes ago

    What’s the most intricate system that’s been written with this?

    • lewq 17 minutes ago

      Itself, more recently

  • theelix 2 hours ago

    Moonlight-Web? I guess it's https://github.com/MrCreativ3001/moonlight-web-stream but there's no information in the article

    > Moonlight expects: Each client connects to start their own private game session

    Nope, it's a Wolf design choice, eg. Sunshine allows users to concorrenly connect to the same instance/game

    • lewq an hour ago

      Wolf now supports multiple clients connecting to the same session via the wolf-ui branch that landed recently. After lots of stability work we are now running that mode in production (and in the latest release) https://github.com/helixml/helix/releases/tag/2.5.3

  • asmor 4 hours ago

    > because we’re all going to become managers of coding agents whether we like it or not

    I will join the woodworking people before that happens, thanks.

    • DrewADesign 2 hours ago

      A career change that left me as a recent graduate in a decimated marketplace missing the bottom ten rungs on the ladder and no interest in getting back into the software world has led me to advanced manufacturing as a metal worker. I code a little, move heavy steel pieces periodically which is a nice way to break up the standing/sitting but not nearly as much as a general laborer, solve lots of problems, keep my trigonometry muscles toned, am forced to take breaks, get paid for my overtime, there’s a union that the company ownership is totally willing to work with, and when I’m not at work, work isn’t with me. There’s something very satisfying about leaving work with exercised muscles, smelling slightly of cutting oil. The money sucks comparatively so early in my career, but the rate increases more for performance than seniority so its rising quickly, the benefits are good, the career trajectory is pointing upwards, and longevity-wise, it’s certainly a whole lot better than gig work.

      There’s a huge crisis in US manufacturing: we’re bleeding craft knowledge because off-shoring let companies hire existing experienced workers for decades, so they never had to train a new generation of tradespeople. Now all those folks are dying and retiring and they need people to pick up that deep knowledge quickly. Codifying and automating is going to kill jobs either way, but one factory employing a few people making things for other factories with local materials is better than everything perpetually shifting to the cheap labor market du jour. I’m feeling much more optimistic about the future of this than the future of tech careers.

      I think over the next few years, a very large percentage of folks in tech will find themselves on the other side of the fence, quickly realize that their existing expertise doesn’t qualify them for any other white collar jobs where vibe coding experience is a bullet point in the pluses section, that tech consulting is declining even faster than salaried jobs, and that they’re vastly less qualified than the competition for blue collar jobs. Gonna be a rough road for a lot of folks. I wouldn’t invest in SF real estate any time soon.

      • seemaze 35 minutes ago

        There’s 1,000 established industries that don’t offer the rapid growth and pay outs of the modern tech ecosystem. I’m excited to see some of the current industrial backwaters soak up technical talent freed up by the SV AI brain drain.

        To think we’ve handsomely paid our best and brightest the last few decades in pursuit of.. advertising?

        • DrewADesign 29 minutes ago

          Hopefully the exodus from the tech industry won't kill demand for too many job markets that are close comfortable cousins to the tech world.

    • danielbln 2 hours ago

      You will always be able to produce artisanal hand-set code, same as how artisanal woodworking exists alongside industrial manufacturing. There will be a lot less demand for it, and compensation will align accordingly, but it won't go away.

  • reactordev an hour ago

    Whilst impressive to “bend a protocol to your will”, why did you not just take Moonlight and build on top of it, making your own?

    No shoehorns needed. Just take what you like and build what you need.

    • lewq an hour ago

      It's nice for unmodified moonlight clients to be able to connect - they have tons of them, you can even run it on a Nintendo DS

      • reactordev an hour ago

        But is the ability to run it on the DS a feature? I highly doubt it.

        I’m not trashing anything, I’m just saying that if they focused on what their market is, it would be clear no one is going to be coding/working on a Nintendo DS.

        • lewq an hour ago

          I suppose but we got it working, and the primary interface is webrtc in the browser, and going via moonlight internally is just an implementation detail that got us here quickly. We are open to refactoring in the future of course :)

  • CuriouslyC 43 minutes ago

    This is beautiful madness.