Elephant-Goldfish Model

(egm.dev)

12 points | by _tk_ a day ago ago

9 comments

  • sathish316 44 minutes ago

    Bitter lesson - anything that can be optimised outside the model or harness will be optimised by the model or harness in the next 2 months.

    After trying any such pattern or skill or next big thing, I keep returning to Peter Steinberger’s advice in 2 weeks and follow “Just talk to it” engineering - https://steipete.me/posts/just-talk-to-it

    • jerrygenser 16 minutes ago

      Yes, but people trying new harness improvements like superpowers and this one generate training data which then gets reinforcement learned. It needs to be bootstrapped somehow. If you are an early adopter there's not much to lose by experimenting with harness tweaks.

  • dcss_gardener an hour ago

    You can tell the idea makes sense because so many people have ended up doing some version of it manually.

    If this is the same one that was floating around a few weeks ago from google it's overengineered to the point where I think I probably just don't understand what they actually use it for internally. The intermediate output is very hard to understand and modify along the way and it really grinds through tokens for something that has such limited tools and context.

    It may make sense if they're having autonomous agents drive this or something similar. I still have to pay for tokens though so the 20 line skill version I made for myself works better for me. ymmv.

  • bob1029 an hour ago

    I think this risks wasting time reinventing a lot of what you get with frontier reasoning models. If your goal is independence from the frontier providers then this makes sense, but I feel like 6.1 terra on xhigh would probably run circles around any manifestation of this that I could deploy.

    I've spent a non-trivial amount of energy attempting to DIY reasoning models and the conclusion I reached is that I have zero hope of competing with the combined engineering resources of OAI, Google and Anthropic.

  • xnx an hour ago

    This is a smart technique to work around context limitations. Is it anything that an user needs to know?

    I've avoided any special configurations (skills, etc.) expecting that the system prompt and harness will incorporate the best techniques in short order.

  • XTXinverseXTY an hour ago

    Why can't I replicate this with "spin up a review subagent"?

    • ForHackernews an hour ago

      Do most harnesses hand off a lot of context to subagents?

      • dcss_gardener an hour ago

        Not anymore. Pretty much everyone has converged on treating them like tool calls, maybe with an optional "fork" mode or something that pushes in the context.

  • paulglx an hour ago

    Reviewing code in fresh sessions with low to no context avoids the model collapsing over its own slop. I've been doing this manually for a while. This is very promising !