Oh this is really useful. There's definitely a problem to be solved here. agent guidance files, like all forms of documentation, can quickly grow stale.
I've tried to tackle a similar problem with a couple different approaches.
One is a command I call "/retro" which basically goes through all recent history on a project - commits, prs, pr comments, etc, and analyzes the existing documentation to identify how to improve it in ways that would prevent any observed issues from happening again. This is less about adding structure to the docs (as AgentLint does) and more about identifying coverage gaps.
The other is a set of tooling I've built to introduce multiple layers of checks on the outputs of agentic code. The initial observation was that many directives in CLAUDE.md files can actually be implemented as deterministic checks (ex: "never use 'as any'" --> "grep 'as any'"; by creating those deterministic checks and running them after every agent turn, I'm able to effectively force the agent to retain appropriate context for directions.
The results are pretty astounding - among early users, 40% of agent turns produce code that doesn't comply with a project's own conventions.
The system then layers on a sequence of increasingly AI-driven reviews at further checkpoints.
this sounds a little AI-generated :) but thank you, appreciate it.
And yes, I built the landing page with AI after the actual project, MCP, and CLI had already matured.
Oh this is really useful. There's definitely a problem to be solved here. agent guidance files, like all forms of documentation, can quickly grow stale.
I've tried to tackle a similar problem with a couple different approaches.
One is a command I call "/retro" which basically goes through all recent history on a project - commits, prs, pr comments, etc, and analyzes the existing documentation to identify how to improve it in ways that would prevent any observed issues from happening again. This is less about adding structure to the docs (as AgentLint does) and more about identifying coverage gaps.
The other is a set of tooling I've built to introduce multiple layers of checks on the outputs of agentic code. The initial observation was that many directives in CLAUDE.md files can actually be implemented as deterministic checks (ex: "never use 'as any'" --> "grep 'as any'"; by creating those deterministic checks and running them after every agent turn, I'm able to effectively force the agent to retain appropriate context for directions.
The results are pretty astounding - among early users, 40% of agent turns produce code that doesn't comply with a project's own conventions.
The system then layers on a sequence of increasingly AI-driven reviews at further checkpoints.
I'd love feedback: http://getcaliper.dev
This is very cool and I love the website design alone. Did you build that with AI as well? Would love to hear the process.
Author here.
this sounds a little AI-generated :) but thank you, appreciate it. And yes, I built the landing page with AI after the actual project, MCP, and CLI had already matured.