This problem+solution, like many others in the agentic-space, have nothing agent-specific. Giving a "box" API keys was always considered a risk, and auth-proxying has existed as a solution forever. See tokenizer[0] by the fly.io team, which makes it a stateless service for eg - no database or dashboard. Or the buzzfeed SSO proxy, which lets you do the same via an OAuth2-dance at the frontend, and a upstream config at the backend which injects secrets: https://github.com/buzzfeed/sso/blob/549155a64d6c5f8916ed909....
Secret and credential sprawl is a real problem in agent pipelines specifically -- each agent needs its own scoped access and the blast radius of a leaked credential is much larger when an agent can act autonomously. We ended up with a tiered secret model: agents get short-lived derived tokens scoped to exactly the tools they need for a given task, not broad API keys. Revocation on task completion, not on schedule. More ops overhead upfront but caught two misuse cases that would have been invisible otherwise.
1) Not all systems respect HTTP_PROXY. Node in particular is very uncooperative in this regard.
2) AWS access keys can’t be handled by simple credential swap; the requests need to be resigned with the real keys. Replicating the SigV4 and SigV4A exactly was bit of a pain.
3) To be secure, this system needs to run outside of the execution sandbox so that the agent can’t just read the keys from the proxy process.
For Airut I settled on a transparent (mitm)proxy, running in a separate container, and injecting proxy cert to the cert store in the container where the agent runs. This solved 1 and 3.
Im literally working on the exact same solution. Difference is I'm running the system in a Kubernetes cluster.
I essentially run a sidecar container that sets up ip tables that redirect all requests through my mitm proxy. This was specifically required because of Node not respecting HTTP_PROXY.
Also had to inject a self signed cert to ensure SSL could be proxied and terminated by the mitm proxy, which then injects the secrets, and forwards the request on.
Have you run into any issues with this setup? I'm trying to figure out if there's anything I'm missing that might come back to bite me?
I’ve been running this with workloads accessing Anthropic, GitHub, Gemini, and AWS & CF R2 APIs for a while now, and have not ran into issues. I’m sure there’s an API out there that won’t work out of the box but I’m positive that support could be added.
Another thing I did was to allow configuring which hosts each credential is scoped to. Replacement /resigning doesn’t happen unless host matches. That way it is not possible to leak keys by making requests to malicious hosts.
This can also be done using existing Vaults or Secrets manager. Hashicorp Vault can do this and agents can be instructed to get secrets, which are set without the agent's knowledge. I use these 2 simple scripts with OpenClaw to achieve this, along with time-scoped expiration. The call to vault_get.sh is inside the agent's skill script so that the secrets are not leaked to LLMs or in any trace logs:
It seems that the architecture you describe still gives the key to the agent (who could email it to red team or perform nefarious actions with it). The advantage of OP's architecture is that the agent never sees the key and you could inspect the request before proxying it. Is that right or do I misunderstand something?
IronClaw seems to do this natively, I like the idea in general, so it's good too see this pulled out.
I have few questions:
- How can a proxy inject stuff if it's TLS encrypted? (same for IronClaw and others)
- Any adapters for existing secret stores? like maybe my fake credential can be a 1Password entry path (like 1Password:vault-name/entry/field and it would pull from 1P instead of having to have yet another place for me to store secrets?
The fake key for real key thing seems like a problem. A lot of enterprise scanning tools look for keys in repos and other locations and you will get a lot of false positives.
Otherwise this is cool, we need more competition here.
A program making a call to github.com needs an authentication token.
What are you suggesting? The program makes a call to retrieve the secret from AWS? Then has full access to do with it what they want? That's exactly the risk and the problem this, and related solutions mentioned in this thread, is trying to solve.
This problem+solution, like many others in the agentic-space, have nothing agent-specific. Giving a "box" API keys was always considered a risk, and auth-proxying has existed as a solution forever. See tokenizer[0] by the fly.io team, which makes it a stateless service for eg - no database or dashboard. Or the buzzfeed SSO proxy, which lets you do the same via an OAuth2-dance at the frontend, and a upstream config at the backend which injects secrets: https://github.com/buzzfeed/sso/blob/549155a64d6c5f8916ed909....
[0]: https://github.com/superfly/tokenizer
Just because it's been done before for a different use-case doesn't mean that building exclusively for this use-case doesn't remove friction.
Secret and credential sprawl is a real problem in agent pipelines specifically -- each agent needs its own scoped access and the blast radius of a leaked credential is much larger when an agent can act autonomously. We ended up with a tiered secret model: agents get short-lived derived tokens scoped to exactly the tools they need for a given task, not broad API keys. Revocation on task completion, not on schedule. More ops overhead upfront but caught two misuse cases that would have been invisible otherwise.
This is the right approach. I built a similar system to https://github.com/airutorg/airut - couple of learnings to share:
1) Not all systems respect HTTP_PROXY. Node in particular is very uncooperative in this regard.
2) AWS access keys can’t be handled by simple credential swap; the requests need to be resigned with the real keys. Replicating the SigV4 and SigV4A exactly was bit of a pain.
3) To be secure, this system needs to run outside of the execution sandbox so that the agent can’t just read the keys from the proxy process.
For Airut I settled on a transparent (mitm)proxy, running in a separate container, and injecting proxy cert to the cert store in the container where the agent runs. This solved 1 and 3.
Im literally working on the exact same solution. Difference is I'm running the system in a Kubernetes cluster.
I essentially run a sidecar container that sets up ip tables that redirect all requests through my mitm proxy. This was specifically required because of Node not respecting HTTP_PROXY.
Also had to inject a self signed cert to ensure SSL could be proxied and terminated by the mitm proxy, which then injects the secrets, and forwards the request on.
Have you run into any issues with this setup? I'm trying to figure out if there's anything I'm missing that might come back to bite me?
I’ve been running this with workloads accessing Anthropic, GitHub, Gemini, and AWS & CF R2 APIs for a while now, and have not ran into issues. I’m sure there’s an API out there that won’t work out of the box but I’m positive that support could be added.
Another thing I did was to allow configuring which hosts each credential is scoped to. Replacement /resigning doesn’t happen unless host matches. That way it is not possible to leak keys by making requests to malicious hosts.
This is basically what https://www.verygoodsecurity.com/ (their main product), but it's heavily focused on credit card data.
You don't want to give the agent a raw key, so you give it a dummy one which will automatically be converted into the real key in the proxy.
So how does that help exactly? The agent can still do exactly what it could have done if it had the real key.
For one thing, it cannot leak secrets between services.
This can also be done using existing Vaults or Secrets manager. Hashicorp Vault can do this and agents can be instructed to get secrets, which are set without the agent's knowledge. I use these 2 simple scripts with OpenClaw to achieve this, along with time-scoped expiration. The call to vault_get.sh is inside the agent's skill script so that the secrets are not leaked to LLMs or in any trace logs:
vault_get.sh: https://gist.github.com/sathish316/1ca3fe1b124577d1354ee254a...
vault_set.sh: https://gist.github.com/sathish316/1f4e6549a8f85ac5c5ac8a088...
Blog about the full setup for OpenClaw: https://x.com/sathish316/status/2019496552419717390
It seems that the architecture you describe still gives the key to the agent (who could email it to red team or perform nefarious actions with it). The advantage of OP's architecture is that the agent never sees the key and you could inspect the request before proxying it. Is that right or do I misunderstand something?
IronClaw seems to do this natively, I like the idea in general, so it's good too see this pulled out.
I have few questions:
- How can a proxy inject stuff if it's TLS encrypted? (same for IronClaw and others)
- Any adapters for existing secret stores? like maybe my fake credential can be a 1Password entry path (like 1Password:vault-name/entry/field and it would pull from 1P instead of having to have yet another place for me to store secrets?
You use a forward proxy that can MITM.
The fake key for real key thing seems like a problem. A lot of enterprise scanning tools look for keys in repos and other locations and you will get a lot of false positives.
Otherwise this is cool, we need more competition here.
It's a good point, I don't think the placeholders we use will trigger a secret scanner, but we can adjust if it's an issue.
https://github.com/onecli/onecli/blob/942cfc6c6fd6e184504e01...
Does it act like an auth proxy?
tl;dr "scrt [set|get|list|....]" is also a great option
---
If this is of interest, I also recommend looking into: https://github.com/loderunner/scrt.
To me, it's a compliment to 1password.
I use it to save every new secret/api key I get via the CLI.
It's intentionally very feature limited.
Haven't tried it with agents, but wouldn't be surprised if the CLI (as is) would be enough.
Don't see any reason to use this over vault.
Why not just use AWS Secrets Manager?
A program making a call to github.com needs an authentication token.
What are you suggesting? The program makes a call to retrieve the secret from AWS? Then has full access to do with it what they want? That's exactly the risk and the problem this, and related solutions mentioned in this thread, is trying to solve.