I added a 2nd while loop so that it takes user input. And vendored my tiny llm lib (so it's 150 lines now, and dependency free :)
---
As for context-sculpting, the economics are different when not touching the context gives you the >98% discount everyone's doing now. (Although it might be worth fiddling with the suffix... not sure yet!)
e.g. this issue: "ToolSearch saves ~15K tokens per request in prompt size, but at the cost of breaking prefix-based caching for models like DeepSeek that rely on stable prefixes. For heavy users of DeepSeek through OpenRouter, the savings from smaller prompts are dwarfed by the increased cost from cache misses."
i definitely considered something like this for the local-first harness i made ... i just don't think most people have the RAM to be able to run two good models yet. maybe i'm wrong though. but i also think a single "agent" can compartmentalize itself into subdivisions better than we imagine (i.e., much much better than any single human can). i ended up creating a broker, though, so at least the tool calls don't eat up as much context. and the auto-reset thing is definitely legit.
Not a single mention of prompt caching in this article, which is a massive benefit of append-only context.
See also: agent harness in 50 lines (based on mini-swe-agent).
https://minimal-agent.com/
I followed this tutorial earlier today and I'm having a lot of fun with it.
https://gist.github.com/a-n-d-a-i/cb5e929b4c87b8d185760d0264...
I added a 2nd while loop so that it takes user input. And vendored my tiny llm lib (so it's 150 lines now, and dependency free :)
---
As for context-sculpting, the economics are different when not touching the context gives you the >98% discount everyone's doing now. (Although it might be worth fiddling with the suffix... not sure yet!)
e.g. this issue: "ToolSearch saves ~15K tokens per request in prompt size, but at the cost of breaking prefix-based caching for models like DeepSeek that rely on stable prefixes. For heavy users of DeepSeek through OpenRouter, the savings from smaller prompts are dwarfed by the increased cost from cache misses."
https://github.com/QwenLM/qwen-code/discussions/4065
All this mucking about with harnesses and context is really just Markdown engineering.
i definitely considered something like this for the local-first harness i made ... i just don't think most people have the RAM to be able to run two good models yet. maybe i'm wrong though. but i also think a single "agent" can compartmentalize itself into subdivisions better than we imagine (i.e., much much better than any single human can). i ended up creating a broker, though, so at least the tool calls don't eat up as much context. and the auto-reset thing is definitely legit.