The self‑improving part of AgentRun is very interesting to me. In my project https://loopseo.org I faced difficulties, in keeping autonomy while also having controllable behavior. I would like to know what safety and oversight methods you use for this situation.
Sourcery CEO here. Wanted to comment on the claims of 1.6x Greptile review comments.
With Greptile there's a precision-recall trade-off with code review comments. Greptile is at the precision end of the scale, so every comment that they make is accurate. We've seen on GitHub that over 80% of the comments they make get addressed. At the other end of the spectrum is Sourcery [1], which produces over 2x as many comments as Greptile, and this agent run example has 1.6x.
Shameless self-promotion: we believe that outputting a lot of comments, even if they're nitpicky, is the right way to go for coding agents because they can triage all of the comments very rapidly and decide which to address and which are not relevant.
I have done this with my own agent frameworks. They have their own issue trackers and record notes there, so the next agent to work an issue can pick up and go. In addition to providing the specific context for individual tasks, the notes then get swept into per-project libraries that distill the information for future planning.
If it is not written down, the agents can’t know it, so the rule is to write everything down. File an issue before starting work, update with a note at every step. One task per issue. You know… the stuff we would do for each other, if we all didn’t hate Jira and its ilk so very very much.
How is this different to a simple harness? For example, I have a harness, made in 1.5 days, that has an executive agent answering the "what done looks like" task, then chunking the work, creating a dependency graph, and assigning it to sub-agents (which can be recursive, ie. the sub-agent becomes an executive for its own workers). And there's an independent "reviewer" for each work chunk, before potentially merging and kicking off any tasks whose dependencies are satisfied. Interfaces with remote hosts, GPU machines from different providers. So what would your thing give me that mine can't do?
Would like to hear from everyone: How are you dealing with PRs in 2026? It's our biggest bottle neck.
We are considering looking into attestations, e.g. through the SLSA spec: https://slsa.dev/ to ensure build provenance and verification before it is deployed.
For my own projects: Agents have access to managed tools which enforce quality metrics, build, test, and deploy all more or less in parallel and get it ready to go into git. Examples of metrics are: test coverage, allowed dependencies, linters and the usual. These tools run on the same machine where the software will be deployed (for performance reasons). These tools submit attestations for everything they do. I made my own system for it, it keeps track of everything. The whole process including docker build and deploy to a test env is 10s or less. (I find 10s is reasonable to achieve for all projects I've worked on). Secondly I run a daily process for all projects that runs another set of quality metrics ; picture for example dependencies out of date, tests getting too slow and similar topics. These become work for the agents to pick up. Review is me checking if the app does what it should at best. If it passes all the quality metrics, I don't care much for the code.
For clients (who are not that AI-pilled): They're liking CodeRabbit and developers are not reviewing as strictly anymore. Quality metrics are obtained via CI/CD and attached to PR's. If it's all green then most developers would give it a LGTM
1. Agentic merge queue. Lower risk PRs enter a queue and are auto-merged by a specialized agent. Higher risk PRs are still reviewed by humans. We're currently doing this for Dependabot PRs via a product we built: https://patchwave.ai/ , but the approach generalizes to any kind of PR.
2. Stop doing PRs. Review moves upstream into a multiplayer agent thread that's visible to the team - tag a teammate to pop in and look when you need feedback. PR checks move upstream into this thread as well, the agent just runs them as part of verifying its work. Push to main, ci checks run and deploy as normal from there.
We're at 1 now, but actively working towards moving to 2 and want to help other teams do the same.
1. No pull requests
2. I have separate strategy, code-writing, and commit agents
3. A reader that never saw the builder's reasoning checks each fix first
4. Full test run must be green on the exact commit
5. Verification / feature walk on staging
On the app I'm building (and its platform dependencies) I've made it through ~1200 commits and ~300k LOC without more than an occasional trivial regression/having to roll back... so, I think my strategy is working.
What I do is while tasks are being assigned to agents I work the the manager agent to ensure the tasks are correct and actually correspond to the goal of the project. The manager agent reaches out with blocking questions it cannot figure out itself. Then for review pre-release the code is verified against the assigned task, checked for bugs and security risks. Once all found issues are fixed it goes out.
If your team is using subscriptions, it is a lot more cost effective (and higher quality) to simply use a local orchestrator to automatically deal with PR reviews. We built https://github.com/codealong/codealong in part to help with this.
Very cool. I was waiting for something like this to pop up.
Is it possible to be added as a data source? I see you're mostly pointing the agent to easy / publicly available sources like the UK.
We have ~200 government registrar integrations now and 500M entities in the system. We essentially have better coverage than OpenCorporates at this stage. https://savvyiq.ai
I've noticed on some of my own research and testing that even with caps some agents will work around the per-call time limit using background processes. Do your caps cover compute in the detached processes?
Cool project, I myself have a similar autonomous agent, in python
At rinkesh2010rpp/sudarshana. But I liked your mission control. Will for sure take some inspiration.
Also, what I have is ability to find other agents on run time, running on Modal, and a safety layer that checks every actions taken by the agent.
The core concept is to have a frontier or very smart model run as an agent first for N times. The model get tools, skills etc and instructions. It does the job. At the end it leaves a note (like a retro) explaining what it did and what it could have done better.
As the agent runs, it also writes workflows to do the parts of the job it knows it can do in a more deterministic way (based on previous nodes and thinking traces of other runs etc). We made a dsl for this that has routing, classification, code nodes, etc to build these workflows.
For repetitive tasks (which is our focus) we find that after ~100 cases, the agent most times just runs a workflow end to end instead of looping towards a solution. Workflows are cheap, so the job ends up being way cheaper than an agent in a good model.
Interesting, this is basically what happens organically in my “office” agentic structure (just a bunch of documents that govern an agentic team with defined roles in Claude code). They end up writing python tools to do 90 percent of the repetitive tasks. I also give them access to A farm of specialised local models to use as well as some reasonably capable small LLMs.
It hasn't gotten better since a year ago, I don't have high hopes. People need to get better at making smaller decisions and having more opinions, not just prompt "Create page for dashboard".
This is fair feedback. We spent a ton of time in the UI for our hosted product and the main website (agentrun.ai) but very little in the OSS console. We’re spending more time on this now and making it better.
Issues I personally feel with managing learnings or code feedback as skills, memory, etc:
0. How do you ensure the learnings necessarily apply?
1. How do you ensure learnings are deterministic?
2. How do you prove that the outcome of the learning’s enforcement yields what you actually want?
3. As more learnings are piled on, how do you manage costs and latency?
The self‑improving part of AgentRun is very interesting to me. In my project https://loopseo.org I faced difficulties, in keeping autonomy while also having controllable behavior. I would like to know what safety and oversight methods you use for this situation.
Sourcery CEO here. Wanted to comment on the claims of 1.6x Greptile review comments.
With Greptile there's a precision-recall trade-off with code review comments. Greptile is at the precision end of the scale, so every comment that they make is accurate. We've seen on GitHub that over 80% of the comments they make get addressed. At the other end of the spectrum is Sourcery [1], which produces over 2x as many comments as Greptile, and this agent run example has 1.6x.
Shameless self-promotion: we believe that outputting a lot of comments, even if they're nitpicky, is the right way to go for coding agents because they can triage all of the comments very rapidly and decide which to address and which are not relevant.
[1] https://sourcery.ai
How do you manage review loops where the agents nit their way through nonsensical (but real) edge cases?
I have done this with my own agent frameworks. They have their own issue trackers and record notes there, so the next agent to work an issue can pick up and go. In addition to providing the specific context for individual tasks, the notes then get swept into per-project libraries that distill the information for future planning.
If it is not written down, the agents can’t know it, so the rule is to write everything down. File an issue before starting work, update with a note at every step. One task per issue. You know… the stuff we would do for each other, if we all didn’t hate Jira and its ilk so very very much.
> we believe that outputting a lot of comments, even if they're nitpicky, is the right way to go for coding agents
Is this what they mean by "tokenmaxxing"?
How is this different to a simple harness? For example, I have a harness, made in 1.5 days, that has an executive agent answering the "what done looks like" task, then chunking the work, creating a dependency graph, and assigning it to sub-agents (which can be recursive, ie. the sub-agent becomes an executive for its own workers). And there's an independent "reviewer" for each work chunk, before potentially merging and kicking off any tasks whose dependencies are satisfied. Interfaces with remote hosts, GPU machines from different providers. So what would your thing give me that mine can't do?
https://knowhow.ml/harness1.png
https://knowhow.ml/harness2.png
https://knowhow.ml/harness3.png
Would like to hear from everyone: How are you dealing with PRs in 2026? It's our biggest bottle neck.
We are considering looking into attestations, e.g. through the SLSA spec: https://slsa.dev/ to ensure build provenance and verification before it is deployed.
For my own projects: Agents have access to managed tools which enforce quality metrics, build, test, and deploy all more or less in parallel and get it ready to go into git. Examples of metrics are: test coverage, allowed dependencies, linters and the usual. These tools run on the same machine where the software will be deployed (for performance reasons). These tools submit attestations for everything they do. I made my own system for it, it keeps track of everything. The whole process including docker build and deploy to a test env is 10s or less. (I find 10s is reasonable to achieve for all projects I've worked on). Secondly I run a daily process for all projects that runs another set of quality metrics ; picture for example dependencies out of date, tests getting too slow and similar topics. These become work for the agents to pick up. Review is me checking if the app does what it should at best. If it passes all the quality metrics, I don't care much for the code.
For clients (who are not that AI-pilled): They're liking CodeRabbit and developers are not reviewing as strictly anymore. Quality metrics are obtained via CI/CD and attached to PR's. If it's all green then most developers would give it a LGTM
You have two options:
1. Agentic merge queue. Lower risk PRs enter a queue and are auto-merged by a specialized agent. Higher risk PRs are still reviewed by humans. We're currently doing this for Dependabot PRs via a product we built: https://patchwave.ai/ , but the approach generalizes to any kind of PR.
2. Stop doing PRs. Review moves upstream into a multiplayer agent thread that's visible to the team - tag a teammate to pop in and look when you need feedback. PR checks move upstream into this thread as well, the agent just runs them as part of verifying its work. Push to main, ci checks run and deploy as normal from there.
We're at 1 now, but actively working towards moving to 2 and want to help other teams do the same.
My own personal recipe:
1. No pull requests 2. I have separate strategy, code-writing, and commit agents 3. A reader that never saw the builder's reasoning checks each fix first 4. Full test run must be green on the exact commit 5. Verification / feature walk on staging
On the app I'm building (and its platform dependencies) I've made it through ~1200 commits and ~300k LOC without more than an occasional trivial regression/having to roll back... so, I think my strategy is working.
What I do is while tasks are being assigned to agents I work the the manager agent to ensure the tasks are correct and actually correspond to the goal of the project. The manager agent reaches out with blocking questions it cannot figure out itself. Then for review pre-release the code is verified against the assigned task, checked for bugs and security risks. Once all found issues are fixed it goes out.
Would love to hear that too... it's getting incredibly expensive. One of our templates is a code review agent. https://agentrun.ai/blog/saving-80-percent-on-code-reviews-u.... But we're looking to do more beyond making the review cheap.
How do you tell when production runs start leaving the compiled workflow?
Kubit CEO here. That's the reason why we visually group runs by execution path after labeling every span, so the new branches will popup quickly.
If your team is using subscriptions, it is a lot more cost effective (and higher quality) to simply use a local orchestrator to automatically deal with PR reviews. We built https://github.com/codealong/codealong in part to help with this.
very cool. thanks for sharing!
Very cool. I was waiting for something like this to pop up.
Is it possible to be added as a data source? I see you're mostly pointing the agent to easy / publicly available sources like the UK.
We have ~200 government registrar integrations now and 500M entities in the system. We essentially have better coverage than OpenCorporates at this stage. https://savvyiq.ai
Do you have an MCP? I am adding our internal tool proxy which can be used to plug in any MCP into it.
Yeah, but it's out of date. About to replace it with a new version that has SSO support and other updated APIs. Will be available next week.
I've noticed on some of my own research and testing that even with caps some agents will work around the per-call time limit using background processes. Do your caps cover compute in the detached processes?
uh, interesting. I will take a look into this.
Cool project, I myself have a similar autonomous agent, in python At rinkesh2010rpp/sudarshana. But I liked your mission control. Will for sure take some inspiration.
Also, what I have is ability to find other agents on run time, running on Modal, and a safety layer that checks every actions taken by the agent.
and happy to talk more.
Thanks
Very cool. Feel free to borrow from us what's helpful and send feedback our way!
Thats what I am gonna ask my claude code to do lol.
If you are interested, check out The agent: https://github.com/rinkesh2010rpp/sudarshana The A2a Hotplug which gives agents ability to add a2a agent in runtime https://github.com/rinkesh2010rpp/a2a-hotplug Security layer for your agent https://github.com/rinkesh2010rpp/taster-ai
Could you explain how the self-improving part works?
Yes here is an article about it but I’ll will explain the tldr below. https://agentrun.ai/blog/agentrun-a-harness-for-repetitive-k...
The core concept is to have a frontier or very smart model run as an agent first for N times. The model get tools, skills etc and instructions. It does the job. At the end it leaves a note (like a retro) explaining what it did and what it could have done better.
As the agent runs, it also writes workflows to do the parts of the job it knows it can do in a more deterministic way (based on previous nodes and thinking traces of other runs etc). We made a dsl for this that has routing, classification, code nodes, etc to build these workflows.
For repetitive tasks (which is our focus) we find that after ~100 cases, the agent most times just runs a workflow end to end instead of looping towards a solution. Workflows are cheap, so the job ends up being way cheaper than an agent in a good model.
Interesting, this is basically what happens organically in my “office” agentic structure (just a bunch of documents that govern an agentic team with defined roles in Claude code). They end up writing python tools to do 90 percent of the repetitive tasks. I also give them access to A farm of specialised local models to use as well as some reasonably capable small LLMs.
I think these are their relevant docs:
https://agentrun.ai/docs/self-improvement
https://agentrun.ai/docs/learning-reference
Do it uses the claude code harness (for example) or bypass it? I ask because of the subscription. thanks
It supports Claude Agents' harness but their sub is still not supported. I am working on it. Same for Codex / ChatGPT.
How does this compare to Letta or Hermes?
It looks impressive.
Is it more than an ACP connection to other agents?
Do the agents have to exist already created in the harness or can you make agents inside AgentRun?
You can create the agents inside of Agentrun and pick what harness / model to use. You can also install them from templates. https://github.com/Parcha-ai/agentrun#templates
I am publishing a library of templates tomorrow with many more.
I think I'll need like at least another 3-4 months of new models before I can deal with AI UI/UX design.
It hasn't gotten better since a year ago, I don't have high hopes. People need to get better at making smaller decisions and having more opinions, not just prompt "Create page for dashboard".
This is fair feedback. We spent a ton of time in the UI for our hosted product and the main website (agentrun.ai) but very little in the OSS console. We’re spending more time on this now and making it better.
Hey .Nice
what are "Web-grounded workflows"?
Yikes. I missed a word or two there... I meant workflows using parallel's web-grounded decisions API. It's like Jev but grounded with their web search https://docs.parallel.ai/decisions/decisions-quickstart.
You can read more about how we turn agents into workflows here and examples of using Parallel's decisions API here: https://agentrun.ai/blog/you-may-not-need-an-agent-after-all