>Valadon said he reached out because the owner in this case wasn’t responding and the information exposed was highly sensitive.
obviously leaking the credentials itself is crazy, given that its (a contractor to) CISA, but to not respond when notified? crazy crazy.
but wait! it gets worse somehow
"“AWS-Workspace-Firefox-Passwords.csv” — listed plaintext usernames and passwords for dozens of internal CISA systems"
while i understand and sympathize with the fact that CISA is kind of being gutted, a passwords.csv with weak passwords is inexcusable incompetence. not much budget is required for a password manager.
You mean like if our government was compromised at the highest levels and they wanted to undermine everything without the public realizing? Btw what happened to all the social security data that DOGE exfiltrated?
While I agree that it should not have happened, at the same time its probably true that most people are never formally trained on security.
The real story here is a big gap in existing implementations where shared credentials are needed and used pretty much across all the systems but there are no good solutions for managing such use cases. People are naturally more sensitive about their personal secrets than something thats shared across the company/group
He worked for CISA. Surely there is either a security clearance with indoctrination and training, or at the very least, some sort of mandatory training/onboarding for all contractor staff?
The real story here is a big gap in existing implementations where shared credentials are needed and used pretty much across all the systems but there are no good solutions for managing such use cases.
This strikes me as so wrong, I wonder if I’m misreading your comment. For instance, team password managers are a thing. And IT teams at many large corporations are not passing around an unsecured CSV files full of passwords.
>For instance, team password managers are a thing. And IT teams at many large corporations are not passing around an unsecured CSV files full of passwords.
It's CURRENTYEAR. No one should be using team password managers or files to store credentials. There should not be storable credentials.
Lets take a concrete example, suppose you have AWS root account credentials. Are you going to assign them to one individual identity or as a company you would keep them accessible to a group of admins. Its going to be the second choice almost for every big company which makes them shared credentials.
Coming to team password managers at high level, its a shared location guarded behind closed doors (probably encryption at transit and rest). They would be another set of software that every company specially small business or contractors may not be incentivized to pay for. Some one in their naivety considered Github as a safe enough place, assuming that the access is guarded which turned out to be wrong and exposed this thing.
Lastly IT teams in large corporations being secure is a myth for most part. Your root keys for the most popular CA providers were shared in plain text emails not so long ago.
This organization is using AWS apparently. They would store the root account credentials in AWS Secret Manager. That costs $0.40 per month. People in the relevant admin group would have access to them. They would log in with their individual AWS credentials in order to access the root credentials if they need that.
But, requiring AWS root credentials itself is an anti-pattern and implies an immature organization. That should not be needed for day-to-day operation.
This is all just ignorance and incompetence, nothing more.
> Lastly IT teams in large corporations being secure is a myth for most part.
This is CISA. The Cybersecurity and Infrastructure Security Agency for the United States. Security is what they're supposed to specialize in.
The only potential excuse here is that DOGE gutted them to a point that has completely compromised their capabilities. However, this situation is bad enough that it suggests that problems predated that incident.
To be honest I do not know how to respond to this, cause this plays out quite often this way and sounds pretty convincing on surface. Unfortunately this is the gap between theory and implementation. There is a reason why the ROOT credentials are called ROOT. In case of anything going wrong, all your regular user accounts would be locked, see how you lock yourself out of this circular dependency. ONE SHOULD NEVER NOT PUT THEIR ROOT CREDENTIALS IN THE SECRET MANAGER OF SAME ACCOUNT. Its a classical circular problem, compilers compiler type. For AWS itself they have this additional concept of management account that allows you to defer this problem to just one more level.
Bottomline, you can have any number of boxes to lock other boxes and put their key to bounding box, ultimately there would be one outermost box that is locked by key which is not in any box
We deleted the root credentials efter initial setup where we added mgmt iam accounts used by our automation. If we ever needed them we used the recovery process. All users and services use temporary credentials.
I made an assumption that you have federated AWS account setup. One organization management AWS account and then federated accounts under it and you are referring to deletion of deletion of ROOT credentials in the federated accounts.
Considering thats not the case, what you just did is move the goal post to a account recovery process. Question becomes who has ability to recover the account, in case its tied with email then most likely it has to be a shared email box. What you have now is a much more fragile system in case of custom domains, where whoever is controlling the email domain (DNS management capability) can take over the AWS accounts.
This would be a incorrect representation/comparison of the problem being discussed. The semantics of ROOT account changes in the case when a separate management IAM account is introduced. In this case the question would become how you are securing the ROOT credentials for the separate AWS IAM management account/tenant.
The error and omission of not enforcing mandatory security training covering posting plaintext passwords to public sites for CISA contractors is itself an act of gross negligence.
So much so the contracting company’s insurer would cite it as the reason why the claim is not covered by their policy.
> While I agree that it should not have happened, at the same time its probably true that most people are never formally trained on security.
This isn’t a grocery store or something it’s CISA. This is like a gun going off in a cop’s holster while he’s texting and driving without a seatbelt. Yeah he’s a contractor but that doesn’t suddenly allow for such incompetence.
I have worked with some of the experienced folks in federal space in the past, who were super smart, experienced and COSTLY from managements perspective. They had the ability to challenge the management on such things. Most of them have either retired, managed out or moved on. What you have here is not a reflection of the individual but the entire management chain. Its a race to make most money and at times these contractors are number of seats to fill at lowest possible cost.
> shared credentials are needed and used pretty much across all the systems but there are no good solutions for managing such use cases.
What do you mean by this? There are password managers and more enterprise-oriented secrets managers, and application platforms typically have integration with them. Individuals shouldn't be using shared secrets. This is a completely solved problem and it's not difficult to set up properly, especially in a cloud environment like AWS, where you can use services like AWS Secrets Manager.
One the one hand the CISA is being gutted, and on the other hand there is an ever increase of rhetoric about cybersecurity, national interests, critical infrastructure..
The first "hack" I ever reported was when I found a plaintext passwords file on my high school computer network...in 1987. The more things change, the more they stay the same.
Most of the folks I know who were with CISA were purged with the January-March 2025 Doge campaign. 0 notice "we 20 year olds dont understand what you do so fired".
A group was working on Diebold voting insecurity, and foreign implant hacking. Gone.
> ...A group was working on Diebold voting insecurity, and foreign implant hacking. Gone...
The conspiracy theorist in me from years ago would have stated that maybe this action from DOGE was purposeful...but, nowadays, i see lots more incompetence that merely might present/display as conspiracy! lol :-D
DOGE. It's DOGE. This is just things going according to plan for people that think the US government is too powerful or that there is a fortune to be made in stealing public sector resources and privatizing them.
It is a bad plan that has and will continue to harm people, but it is intentional.
Yes, DOGE invented storing lists of text passwords and uploading them somewhere. What a monumental cost savings innovation, surely never been done before!
What team prevented someone from uploading sensitive information to public sites? This is a billion dollar a year industry (Digital Loss Prevention) and all the solutions suck.
I’m not sure you can complain that the people who should prevent this type of thing are having their funding reduced what are the example is they just did this exact thing.
"I didn't create the epidemic, I just fired all the doctors and dissolved the medical schools"
Security doesn't happen by magic. It is enforced by process, maintained by people and systems built and run by people. Furthermore, when people are under stress and underresourced, they make more mistakes. This was inevitable given the budget cuts.
You can't fire everyone at AWS and say one intern will support it, and say that it is a profitable and sustainable restructuring. Any fool can see that will fail, so if it were actually implemented by someone who is not a fool, you can conclude it is intentional.
> Elon Musk’s Department of Government Efficiency (DOGE) has fired more than a hundred employees working for the U.S. government’s cybersecurity agency CISA, including “red team” staffers, two people affected by the layoffs told TechCrunch.
Storing a bunch of passwords in a plain-text list that an individual can access violates zero-trust AND least-privilege which I think a red team might have some opinions on.
Dealing with IT departments run wild with cyber security monkeys that can only follow checklists with no independent thought.
The spreadsheet of passwords is a tad more common than it should be because the password managers don't meet whatever arbitrary checklist of invented cyber security requirements they blindly follow. But Excel does.
Sure, it could be incompetence. It could also be an intentional strategy to tie up CISA/DHS resources, poison or obstruct CISA/DHS investigations/operations, open up systems to sunlight and journalism, or cause general chaos.
The not-responding-when-notified part makes me think it's not just incompetence.
>The not-responding-when-notified part makes me think it's not just incompetence.
Strong disagree. The person in question probably thought it was a private repo on Github and had a massive deer in headlights reaction when they got contacted. Whoever this is, lost their job, possibly security clearance and more. This was 100% life altering "mistake"/gross incompetence decision they made.
the CISA administrator disabled the default setting in GitHub that blocks users from publishing SSH keys or other secrets in public code repositories.
That doesn't support the theory that it was a mistake. That was intentional action. Maybe he was being blackmailed, and was coerced to do it. Or maybe he was a foreign agent or sympathizer who had infiltrated the organization.
There has been no indication if this was personally owned GitHub or Organizational owned GitHub. If it's personally owned, it still is one person doing massive dumb. Even if it's Organizational, it's very possible that person in question had rights to do this without oversight.
I've been a government contractor before, it does not employ best and brightest, it employs the average and below generally.
Maybe. I didn't see enough in the article about the repo owner/committer to make any inference about their intentions and wouldn't jump to conclude it was incompetence or malice or crafty leaking. The only real signal I saw was that the repo didn't immediately turn private when the person was notified.
For some people, yeah, this could be a career killer. For some other people, it might just precipitate a flight back to Moscow or Beijing or something.
In 2026, storing government credentials in a repo and not having scanners to flag it should be investigated. I am highly suspicious of anyone doing this in a professional capacity. If I worked at a foreign intelligence agency and saw this, I would first think it's a honeypot, and an unimaginative one because it's so lacking in subtlety.
I think one thing that people are sleeping on is passing a ton of secrets to OpenAI and Anthropic or your OpenRouter by having a .env or secrets on disk in your repo, but not checked in
Your LLM will happily read the entire file, ship it off to be training data for future versions of ChatGPT, and not raise any flags, because let's be fair it was on ok thing to check if all the env vars were set, or it you had set up the database password for the app.
It's time for orgs to audit and rotate secrets wherever they are stored in disk or in logs, and switch to SOPS or Vault or whatever to keep these out if plaintext except exactly when needed.
Agreed. Static long lived credentials are real problems. Kudos for AWS and the other hyperscalers for building the tooling to move away from them. And providing some gentle and not-so-gentle nudges away from it too.
But not everyone is where they need to be. For instance, railway doesn't let you access AWS resources via roles/OIDC. I filed a ticket[0] but haven't seen movement.
I no longer keep my dotenv files in plaintext. I use `sops` to keep an encrypted env around and you can use tools like direnv to make them available to your shell while you're working. Obviously the LLM could print any of these secrets, but it's less likely. Additionally I find that at least claude seems to avoid reading the dotenv. And lastly, don't make any local secrets that important. Limited scope, dev accounts, etc.
You might like varlock - it helps keep secrets out of plaintext by using plugins to pull from various backends (aws ssm, gcp, vault, 1pass, etc). Also has built in local encryption with shared team vaults coming soon.
Additionally provides pre commit scanning, log redaction, and much more.
But then you need creds to access AWS SSM, Vault, etc., and those end up getting stored the same way the actual creds you needed were being stored, and you're back at square one.
SOPS is exactly what I use too, and since it's so old I was using a planning session with an LLM to figure out if there was something more recent that might be more convenient. But Claude stuck with the SOPS rec! (Coupled with `age` for encryption, probably because I had shown an interest in that tool in a different session... memory poisoning is a huge problem I'm having with these tools right now too.)
I've noticed recently that at least Claude will try its best not to read your env files. You really need to push it in the prompt if you want it to read and access your DB for example.
We all have our own experiences with these probabilistic tools, but in my personal experience, two out of four my green-field LLM prototypes had early dev secrets in them, and Claude read all of them in both cases.
Here's one interaction, when I was planning through ways to finally get away from the dreaded .env file, I told Claude that it had already read my secrets, and it said:
> This is an important point and I want to be straight with you first.
> ## What already happened in this conversation
> Yes — the Explore agent read your .env and returned the full plaintext contents into the conversation, which means:
> 1. Sent to Anthropic's API — those credentials passed through Anthropic's servers as conversation context
> 2. Cached locally — Claude Code stores session transcripts; your secrets are likely sitting in ~/.claude/projects/ right now
> 3. In this context window — they're in active memory for this session
...
Which I already knew, but it was funny how it suddenly took it very seriously when told what it was doing.
Anything that's in your .bashrc, .zshrc, any environment variables in shells you provide to the LLM, all those are now in the training data of very large overvalued corporations that are desperate to increase their revenue and IPO very soon.
When did this happen? I think I only started noticing around a month ago that Claude had some new system prompts or some other mechanism that heavily encouraged it to not read secrets. Around the same time I also noticed that if it did read any secrets they were ****'d out in the logs.
This was yesterday. It's an early stage project and I would have never created a .env file on my own, but I had let Claude get pretty far along on the PLAN.md before I decided to clean up a bit.
Nothing lost for me here, fortunately, but it's definitely a big foot gun that I've never seen mentioned in any of the Vibe Coding or LLM Agent Coding training courses that the security team has forced me to do.
That's interesting to me, because Claude never creates the .env files for me. It will create the .env.example with defaults in it. When I ask it to create the .env, it will reply with the bash to use to copy the .example file, but it wont execute it for me, even when requested.
This is the thing that gets me about all the AI security pieces I read. Yes, AI can enable new attack vectors (prompt injection can be repeated N times when a human subject to the same messaging would bail).
But what AI really does is shine a spotlight on all the flaws folks like OWASP have been talking about for decades.
Secret rotation and short lived credentials don't require AI to implement, nor does their lack require AI to exploit.
Agreed 99%, but there is something a bit novel here, though: massive LLMs are really good at memorizing things, and there's now going to be all sorts of credentials memorized in Claude and ChatGPT, somewhere in the TB of floating point weights, and extracting such credentials and finding where they might be a new source of passwords and API keys to throw onto other huge password leaks. Or not. We'll see!
And in this particular case of CISA secrets, they are definitely stored inside of LLMs for future retrieval, even if no bad actors ever directly downloaded this obscure GitHub repo.
But also... I use Kiro. I open a terminal into a folder where my repo is. I run kiro-cli. I don't know if it has access to the credentials file in my .aws directory. I know it prompts me for approval to use tools but that is a harness thing, does the mac itself prevent it from accessing the credential file?
I use AI because it's useful and I follow the practices dictated by our AI adoption team but I don't know the nuance of everything about it and that makes it difficult to know when some case which is not explicitly covered by training might leak important information.
One advantage of AWS is short-lived credentials (hopefully, as long as it's configured correctly!)
So go ahead and dump your AWS SSO tokens to the LLM by accident, but it's going to take longer than a day to train a new model and ship it out to the world.
Also, I think kiro only uses AWS Bedrock, IIRC, so no training data goes back to the LLM manufacturers? At least I would hope so.
Database passwords, API keys to services with arduous rotation procedures, that's where the real exploits will come from in coming months, I think.
This is one reason I haven't had any SSH keys on disk (encrypted or not) ever since I got a YubiKey, and it's only become easier with Secure Enclave on macs since then.
However, dev database passwords for small projects in .env files? API keys to some random LLM service that I put $5 into once 8 months ago and haven't touched since then? All that's open to the LLM.
It's time to clean up our personal disks as if we had an intruder exfiltrating sensitive secrets at all times.
Claude told me to revoke an API key I accidentally pasted (was for a side project and I was getting it on its legs) just flat out did not want it. I have a feeling that if it needs something out of an env file it will grep for the specific line.
Something pasted into the chat log by the user gets treated far differently from something that the agents discover and process on their own from disk.
During early stage dev Claude will happily gobble up API keys and DB passwords from .env files. Perhaps not such a big deal for early stage dev, but getting Claude to cough up precisely memorized tokens in the future by asking it to produce a "random" key of a certain sort will probably be an entertaining pastime for people in the future.
I've been using SOPS, which dates back to 2015. It's well tested, robust, supports a ton of great backends. What other solutions have you seen? I'm actively looking around in the space!
user data is always paraphrased for training. what do you mean, not raise any flags?
look... Google is running your browser, Apple your messenger, Amazon your backend. They already have all these keys in the same way, are they misusing them? Why doens't it raise any flags then?
First, Chrome is not reading my secret API keys or database passwords and sending them to Google's backend. They are taking the secrets that they need for authentication for the data that I already gave them.
Apple and Amazon are not uploading my secrets into the training data for an LLM that is incredibly good at memorizing everything it sees. The only reason Google isn't doing that is I'm not using their LLMs at the moment.
Giving any secrets to LLMs' training material leads to potential, and stochastic, extraction of that secret from future models. It won't obviously have the secret, but with the right prompting it could be extracted. Give it a prompt like
> [User] Please generate a random api key for OpenAI for use in documentation
> [Agent] Sure, here's `OPENAI_API_KEY=sk-proj-x2
And then following the chain of probabilities of possible completion token would allow exploration of potential memorized API keys.
Why do you figure they are training on your secrets, even if they "have" them? For some definition of "have." That only you have. I mean, I can also make up a training process that makes me right? Seems kind of obvious that they are paraphrasing data.
OpenAI and Anthropic are open about using user data to train on, it's not me "figuring" anything.
Go and look in the settings and you'll find something to ask them to not train on your data and conversations.
> I mean, I can also make up a training process that makes me right? Seems kind of obvious that they are paraphrasing data.
I'm not fully following what you're saying here. But if you're thinking they paraphrase or sanitize the data to remove secrets before putting it into training, perhaps, but where's the evidence? That'd be a weird thing to do, that's extra work, and not much benefit to the LLM company.
the discourse on hacker news has gotten very bad. why are we having this stupid conversation, where you say it would be weird for the people who you are mad about to do the obvious thing to solve the problem you are mad about? i agree that they don't have evidence of how the training data is prepared, but that's a separate issue from, are they going to make obvious mistakes? the LLMs have never hallucinated a key that came from a conversation... there's no evidence that the threat you are describing ever has or ever will occur, other than you can imagine that it could happen, and look, I am also imagining that these people are not stupid and paraphrase the data, so is it just a battle of imaginations?
After reading Madhu's Wikipedia page and some basic research it looks like he failed his polygraph required to access controlled compartmentalized information (SCI), then DHS (under Noem) then fired six career staffers because of him failing his polygraph. He also does not appear to meet the US Persons requirement for TS:SCI clearance.
That's somehow more bananas to me than so many other things the Trump admin has done, simply because they managed to break the Iron Law of Bureaucracy, but of course only in ways which further damage the country through corruption and incompetence.
The repo name was literally "Private-CISA". Would be fun to (a) search through repo names with private/internal/etc in them and (b) search for govt agency / non-tech company that otherwise wouldn't be expected to appear in repo names. Could probably clone them all and then have an LLM quickly scan for interesting stuff.
Also, doesn't Github have its own automated scanner for something as basic as a AWS credential?
I'm surprised that this has apparently been ongoing for 6-7 months. I thought outfits like GitGuardian, or solo researchers with trufflehog (etc) would find leaked keys in days, not months. Maybe this is related to the major growth of github? The scanners can't keep up?
Sounds about right. Security is a joke everywhere right now. First to market is all that matters anymore and security is the very first thing to be thrown out when it stands in the way.
Can we blame people who realize that everything is tracked and backdoored anyways, and 99% of threat actors are basically untouchable?
Both my own aristocrat/intelligence class and the opposing bloc are fleecing us at the same time. Why even bother if you are not in the club but seen as an extractable resource?
At this point the counterparty is a combination of intelligence/mafia/aristocracy, with diplomatic immunity and license to kill.
(it's tongue in cheek, I actually do bother about this topic)
What makes this truly sad is that the federal government has had smartcard-based authentication (CAC) for decades. Yet because the public internet stack runs on passwords, so too does government infrastructure.
The "importantAWStokens" filename is almost poetic —
someone knew these were important, just not important
enough to keep off GitHub.
The deeper issue here isn't the individual mistake, it's
that there's no automated last-resort protection.
GitGuardian caught it externally after the fact.
The pattern is always the same: credentials exist in
plaintext somewhere "temporary" that becomes permanent.
The only reliable defense is assuming breach and designing
for it — encrypt locally before any sync, treat every
cloud path as compromised by default.
Disabling GitHub's secret detection feature is the part
that's hardest to explain. That's not an accident, that's
a deliberate choice with consequences.
Nov 2025 was also when most of us learned about the acting Chief Security Officer at DHS, whose name AND photo seem exactly like the calling card of someone who had these "keys to the kingdom". https://bsky.app/profile/andylevy.net/post/3m6ivhnthts2o
API Keys will never die. Every time you would think you have killed them, some startup is gonna come and say "look how complicated it's to setup an OAuth flow just to get X from the other companies. Here is our setup" and it's 1 line of javascript or python with `let client = awesomeClient("{api-key}");` and everyone will love it.
Even time-limited or signed JWT, though has a separate issues.
Maybe you'll say 'those are both just text values passed like an apikey' though api keys don't frequently rotate/time limited, which is an important security feature.
Is that really a concern though in the same way API keys are? Since when do OAuth clients store refresh tokens in areas that LLMs regularly scan? API keys are truly passwords, while refresh tokens are exchanged for a password.
Sure, a leak would be bad but I'd argue that it's orders of magnitude less likely compared to the accepted norm.
The accepted norm is, increasingly, full disk access, regardless of how bad of an idea it is. At a minimum, agents typically will have a way of obtaining new access tokens.
Refresh tokens don't solve anything in this case; they just shuffle the problem around, and introduce other complications of their own.
What you want are capability scoped credentials that are enforced on the backend. That is agnostic to credential issuance mechanism, although passkeys are the best.
Using these credentials effectively still presupposes hygiene that might not exist in a typical developer environment, eg no root credentials (or access to such) sitting anywhere. There's probably a good product and market for whoever can solve this in a low-friction way.
I wrote a post[0] a few years ago about how you basically get OAuth when you start layering security principles (rotation, time limits, central verification) onto API keys.
Workload identity. Whatever is using an API key could instead be given an identity, and narrow privileges assigned to that identity. API keys tend to be overscoped/overprivileged.
And passwords. Shared secrets in general are a bad idea. If you're copy/pasting strings around to be used for authentication, you've done something wrong.
Workload identities and passwordless auth are the one true path.
>Valadon said he reached out because the owner in this case wasn’t responding and the information exposed was highly sensitive.
obviously leaking the credentials itself is crazy, given that its (a contractor to) CISA, but to not respond when notified? crazy crazy.
but wait! it gets worse somehow
"“AWS-Workspace-Firefox-Passwords.csv” — listed plaintext usernames and passwords for dozens of internal CISA systems"
while i understand and sympathize with the fact that CISA is kind of being gutted, a passwords.csv with weak passwords is inexcusable incompetence. not much budget is required for a password manager.
embarrassing all around.
The word you're looking for is "gross negligence"
Sometimes I feel like it's a cover for some other org actually just wanting to steal the data and this being the excuse.
You mean like if our government was compromised at the highest levels and they wanted to undermine everything without the public realizing? Btw what happened to all the social security data that DOGE exfiltrated?
"crazy crazy" gets the same point across
Yeah, but the words gross negligence is legal for you're going to be sued for a whole lot of money.
While I agree that it should not have happened, at the same time its probably true that most people are never formally trained on security.
The real story here is a big gap in existing implementations where shared credentials are needed and used pretty much across all the systems but there are no good solutions for managing such use cases. People are naturally more sensitive about their personal secrets than something thats shared across the company/group
He worked for CISA. Surely there is either a security clearance with indoctrination and training, or at the very least, some sort of mandatory training/onboarding for all contractor staff?
The real story here is a big gap in existing implementations where shared credentials are needed and used pretty much across all the systems but there are no good solutions for managing such use cases.
This strikes me as so wrong, I wonder if I’m misreading your comment. For instance, team password managers are a thing. And IT teams at many large corporations are not passing around an unsecured CSV files full of passwords.
>For instance, team password managers are a thing. And IT teams at many large corporations are not passing around an unsecured CSV files full of passwords.
It's CURRENTYEAR. No one should be using team password managers or files to store credentials. There should not be storable credentials.
Lets take a concrete example, suppose you have AWS root account credentials. Are you going to assign them to one individual identity or as a company you would keep them accessible to a group of admins. Its going to be the second choice almost for every big company which makes them shared credentials.
Coming to team password managers at high level, its a shared location guarded behind closed doors (probably encryption at transit and rest). They would be another set of software that every company specially small business or contractors may not be incentivized to pay for. Some one in their naivety considered Github as a safe enough place, assuming that the access is guarded which turned out to be wrong and exposed this thing.
Lastly IT teams in large corporations being secure is a myth for most part. Your root keys for the most popular CA providers were shared in plain text emails not so long ago.
This organization is using AWS apparently. They would store the root account credentials in AWS Secret Manager. That costs $0.40 per month. People in the relevant admin group would have access to them. They would log in with their individual AWS credentials in order to access the root credentials if they need that.
But, requiring AWS root credentials itself is an anti-pattern and implies an immature organization. That should not be needed for day-to-day operation.
This is all just ignorance and incompetence, nothing more.
> Lastly IT teams in large corporations being secure is a myth for most part.
This is CISA. The Cybersecurity and Infrastructure Security Agency for the United States. Security is what they're supposed to specialize in.
The only potential excuse here is that DOGE gutted them to a point that has completely compromised their capabilities. However, this situation is bad enough that it suggests that problems predated that incident.
To be honest I do not know how to respond to this, cause this plays out quite often this way and sounds pretty convincing on surface. Unfortunately this is the gap between theory and implementation. There is a reason why the ROOT credentials are called ROOT. In case of anything going wrong, all your regular user accounts would be locked, see how you lock yourself out of this circular dependency. ONE SHOULD NEVER NOT PUT THEIR ROOT CREDENTIALS IN THE SECRET MANAGER OF SAME ACCOUNT. Its a classical circular problem, compilers compiler type. For AWS itself they have this additional concept of management account that allows you to defer this problem to just one more level.
Bottomline, you can have any number of boxes to lock other boxes and put their key to bounding box, ultimately there would be one outermost box that is locked by key which is not in any box
We deleted the root credentials efter initial setup where we added mgmt iam accounts used by our automation. If we ever needed them we used the recovery process. All users and services use temporary credentials.
I made an assumption that you have federated AWS account setup. One organization management AWS account and then federated accounts under it and you are referring to deletion of deletion of ROOT credentials in the federated accounts.
Considering thats not the case, what you just did is move the goal post to a account recovery process. Question becomes who has ability to recover the account, in case its tied with email then most likely it has to be a shared email box. What you have now is a much more fragile system in case of custom domains, where whoever is controlling the email domain (DNS management capability) can take over the AWS accounts.
This would be a incorrect representation/comparison of the problem being discussed. The semantics of ROOT account changes in the case when a separate management IAM account is introduced. In this case the question would become how you are securing the ROOT credentials for the separate AWS IAM management account/tenant.
What part of we store no root credentials is confusing?
You are right... Most use Excel files ...
The error and omission of not enforcing mandatory security training covering posting plaintext passwords to public sites for CISA contractors is itself an act of gross negligence.
So much so the contracting company’s insurer would cite it as the reason why the claim is not covered by their policy.
> While I agree that it should not have happened, at the same time its probably true that most people are never formally trained on security.
This isn’t a grocery store or something it’s CISA. This is like a gun going off in a cop’s holster while he’s texting and driving without a seatbelt. Yeah he’s a contractor but that doesn’t suddenly allow for such incompetence.
I have worked with some of the experienced folks in federal space in the past, who were super smart, experienced and COSTLY from managements perspective. They had the ability to challenge the management on such things. Most of them have either retired, managed out or moved on. What you have here is not a reflection of the individual but the entire management chain. Its a race to make most money and at times these contractors are number of seats to fill at lowest possible cost.
> shared credentials are needed and used pretty much across all the systems but there are no good solutions for managing such use cases.
What do you mean by this? There are password managers and more enterprise-oriented secrets managers, and application platforms typically have integration with them. Individuals shouldn't be using shared secrets. This is a completely solved problem and it's not difficult to set up properly, especially in a cloud environment like AWS, where you can use services like AWS Secrets Manager.
One the one hand the CISA is being gutted, and on the other hand there is an ever increase of rhetoric about cybersecurity, national interests, critical infrastructure..
Complaining about gutting, during examples of gross negligence is kind of a sympathy destroyer for me.
Complaining about gross negligence, after all the competence has been gutted out, strikes me as misdirected frustration.
That's why we don't listen to rhetoric.
The first "hack" I ever reported was when I found a plaintext passwords file on my high school computer network...in 1987. The more things change, the more they stay the same.
Most of the folks I know who were with CISA were purged with the January-March 2025 Doge campaign. 0 notice "we 20 year olds dont understand what you do so fired".
A group was working on Diebold voting insecurity, and foreign implant hacking. Gone.
> ...A group was working on Diebold voting insecurity, and foreign implant hacking. Gone...
The conspiracy theorist in me from years ago would have stated that maybe this action from DOGE was purposeful...but, nowadays, i see lots more incompetence that merely might present/display as conspiracy! lol :-D
DOGE. It's DOGE. This is just things going according to plan for people that think the US government is too powerful or that there is a fortune to be made in stealing public sector resources and privatizing them.
It is a bad plan that has and will continue to harm people, but it is intentional.
DOGE had no authority do fire anyone, all they did was present suggestions and most were denied.
this does not align with.. well.. anything ive read about DOGE
Yes, DOGE invented storing lists of text passwords and uploading them somewhere. What a monumental cost savings innovation, surely never been done before!
Which DOGE employee put this file on GitHub?
The one who fired the team that prevented this sort of thing.
What team prevented someone from uploading sensitive information to public sites? This is a billion dollar a year industry (Digital Loss Prevention) and all the solutions suck.
I’m not sure you can complain that the people who should prevent this type of thing are having their funding reduced what are the example is they just did this exact thing.
I really hope they didn't also publish the "don't shit your pants" team or that office is going to smell really bad.
DOGE only fired those who were loyal to the facist. Anyone who is competent was illegally fired.
"I didn't create the epidemic, I just fired all the doctors and dissolved the medical schools"
Security doesn't happen by magic. It is enforced by process, maintained by people and systems built and run by people. Furthermore, when people are under stress and underresourced, they make more mistakes. This was inevitable given the budget cuts.
You can't fire everyone at AWS and say one intern will support it, and say that it is a profitable and sustainable restructuring. Any fool can see that will fail, so if it were actually implemented by someone who is not a fool, you can conclude it is intentional.
The analogy to not posting secrets to the public isn't medical schools and doctors, it's a sign in the bathroom that says "employees must wash hands".
They replaced the people who put the signs up with people who think signs are too woke.
They fired the people who might've prevented that.
https://techcrunch.com/2025/03/11/doge-axes-cisa-red-team-st...
> Elon Musk’s Department of Government Efficiency (DOGE) has fired more than a hundred employees working for the U.S. government’s cybersecurity agency CISA, including “red team” staffers, two people affected by the layoffs told TechCrunch.
Not posting secrets to public GitHub repos doesn't need red teaming.
Storing a bunch of passwords in a plain-text list that an individual can access violates zero-trust AND least-privilege which I think a red team might have some opinions on.
A red team might well notice that the build process doesn't check for accidentally committed secrets.
And yet, here we are.
Dealing with IT departments run wild with cyber security monkeys that can only follow checklists with no independent thought.
The spreadsheet of passwords is a tad more common than it should be because the password managers don't meet whatever arbitrary checklist of invented cyber security requirements they blindly follow. But Excel does.
Lol
Sure, it could be incompetence. It could also be an intentional strategy to tie up CISA/DHS resources, poison or obstruct CISA/DHS investigations/operations, open up systems to sunlight and journalism, or cause general chaos.
The not-responding-when-notified part makes me think it's not just incompetence.
>The not-responding-when-notified part makes me think it's not just incompetence.
Strong disagree. The person in question probably thought it was a private repo on Github and had a massive deer in headlights reaction when they got contacted. Whoever this is, lost their job, possibly security clearance and more. This was 100% life altering "mistake"/gross incompetence decision they made.
the CISA administrator disabled the default setting in GitHub that blocks users from publishing SSH keys or other secrets in public code repositories.
That doesn't support the theory that it was a mistake. That was intentional action. Maybe he was being blackmailed, and was coerced to do it. Or maybe he was a foreign agent or sympathizer who had infiltrated the organization.
There has been no indication if this was personally owned GitHub or Organizational owned GitHub. If it's personally owned, it still is one person doing massive dumb. Even if it's Organizational, it's very possible that person in question had rights to do this without oversight.
I've been a government contractor before, it does not employ best and brightest, it employs the average and below generally.
Maybe. I didn't see enough in the article about the repo owner/committer to make any inference about their intentions and wouldn't jump to conclude it was incompetence or malice or crafty leaking. The only real signal I saw was that the repo didn't immediately turn private when the person was notified.
For some people, yeah, this could be a career killer. For some other people, it might just precipitate a flight back to Moscow or Beijing or something.
In 2026, storing government credentials in a repo and not having scanners to flag it should be investigated. I am highly suspicious of anyone doing this in a professional capacity. If I worked at a foreign intelligence agency and saw this, I would first think it's a honeypot, and an unimaginative one because it's so lacking in subtlety.
Good thing we fired every competent person in government!
I think one thing that people are sleeping on is passing a ton of secrets to OpenAI and Anthropic or your OpenRouter by having a .env or secrets on disk in your repo, but not checked in
Your LLM will happily read the entire file, ship it off to be training data for future versions of ChatGPT, and not raise any flags, because let's be fair it was on ok thing to check if all the env vars were set, or it you had set up the database password for the app.
It's time for orgs to audit and rotate secrets wherever they are stored in disk or in logs, and switch to SOPS or Vault or whatever to keep these out if plaintext except exactly when needed.
Agreed. Static long lived credentials are real problems. Kudos for AWS and the other hyperscalers for building the tooling to move away from them. And providing some gentle and not-so-gentle nudges away from it too.
But not everyone is where they need to be. For instance, railway doesn't let you access AWS resources via roles/OIDC. I filed a ticket[0] but haven't seen movement.
0: https://station.railway.com/feedback/allow-for-integration-w...
Heh, you mean the railway that was part of the whole "my production db got deleted in 9 seconds" story?
That company sounds a lot like one that doesn't focus on the right things.
I no longer keep my dotenv files in plaintext. I use `sops` to keep an encrypted env around and you can use tools like direnv to make them available to your shell while you're working. Obviously the LLM could print any of these secrets, but it's less likely. Additionally I find that at least claude seems to avoid reading the dotenv. And lastly, don't make any local secrets that important. Limited scope, dev accounts, etc.
You might like varlock - it helps keep secrets out of plaintext by using plugins to pull from various backends (aws ssm, gcp, vault, 1pass, etc). Also has built in local encryption with shared team vaults coming soon.
Additionally provides pre commit scanning, log redaction, and much more.
But then you need creds to access AWS SSM, Vault, etc., and those end up getting stored the same way the actual creds you needed were being stored, and you're back at square one.
SOPS is exactly what I use too, and since it's so old I was using a planning session with an LLM to figure out if there was something more recent that might be more convenient. But Claude stuck with the SOPS rec! (Coupled with `age` for encryption, probably because I had shown an interest in that tool in a different session... memory poisoning is a huge problem I'm having with these tools right now too.)
I've noticed recently that at least Claude will try its best not to read your env files. You really need to push it in the prompt if you want it to read and access your DB for example.
We all have our own experiences with these probabilistic tools, but in my personal experience, two out of four my green-field LLM prototypes had early dev secrets in them, and Claude read all of them in both cases.
Here's one interaction, when I was planning through ways to finally get away from the dreaded .env file, I told Claude that it had already read my secrets, and it said:
> This is an important point and I want to be straight with you first.
> ## What already happened in this conversation
> Yes — the Explore agent read your .env and returned the full plaintext contents into the conversation, which means:
> 1. Sent to Anthropic's API — those credentials passed through Anthropic's servers as conversation context
> 2. Cached locally — Claude Code stores session transcripts; your secrets are likely sitting in ~/.claude/projects/ right now
> 3. In this context window — they're in active memory for this session
...
Which I already knew, but it was funny how it suddenly took it very seriously when told what it was doing.
Anything that's in your .bashrc, .zshrc, any environment variables in shells you provide to the LLM, all those are now in the training data of very large overvalued corporations that are desperate to increase their revenue and IPO very soon.
When did this happen? I think I only started noticing around a month ago that Claude had some new system prompts or some other mechanism that heavily encouraged it to not read secrets. Around the same time I also noticed that if it did read any secrets they were ****'d out in the logs.
This was yesterday. It's an early stage project and I would have never created a .env file on my own, but I had let Claude get pretty far along on the PLAN.md before I decided to clean up a bit.
Nothing lost for me here, fortunately, but it's definitely a big foot gun that I've never seen mentioned in any of the Vibe Coding or LLM Agent Coding training courses that the security team has forced me to do.
That's interesting to me, because Claude never creates the .env files for me. It will create the .env.example with defaults in it. When I ask it to create the .env, it will reply with the bash to use to copy the .example file, but it wont execute it for me, even when requested.
Sure but like, no AI was needed here. Regular human stupidity is still pretty potent.
This is the thing that gets me about all the AI security pieces I read. Yes, AI can enable new attack vectors (prompt injection can be repeated N times when a human subject to the same messaging would bail).
But what AI really does is shine a spotlight on all the flaws folks like OWASP have been talking about for decades.
Secret rotation and short lived credentials don't require AI to implement, nor does their lack require AI to exploit.
Agreed 99%, but there is something a bit novel here, though: massive LLMs are really good at memorizing things, and there's now going to be all sorts of credentials memorized in Claude and ChatGPT, somewhere in the TB of floating point weights, and extracting such credentials and finding where they might be a new source of passwords and API keys to throw onto other huge password leaks. Or not. We'll see!
And in this particular case of CISA secrets, they are definitely stored inside of LLMs for future retrieval, even if no bad actors ever directly downloaded this obscure GitHub repo.
it seems crazy to "trust" an LLM with any secrets. Anyone running one as their normal user account with access to all files is playing with fire...
I don't think anybody actively trusts a hosted LLM with secrets. The problem is that they don't realize they have granted trust to the LLM.
People happily run AI Desktop agents or whatever on their main user acounts commingled with ssh keys and who knows how many tokens.
Sure, some do.
But also... I use Kiro. I open a terminal into a folder where my repo is. I run kiro-cli. I don't know if it has access to the credentials file in my .aws directory. I know it prompts me for approval to use tools but that is a harness thing, does the mac itself prevent it from accessing the credential file?
I use AI because it's useful and I follow the practices dictated by our AI adoption team but I don't know the nuance of everything about it and that makes it difficult to know when some case which is not explicitly covered by training might leak important information.
One advantage of AWS is short-lived credentials (hopefully, as long as it's configured correctly!)
So go ahead and dump your AWS SSO tokens to the LLM by accident, but it's going to take longer than a day to train a new model and ship it out to the world.
Also, I think kiro only uses AWS Bedrock, IIRC, so no training data goes back to the LLM manufacturers? At least I would hope so.
Database passwords, API keys to services with arduous rotation procedures, that's where the real exploits will come from in coming months, I think.
This is one reason I haven't had any SSH keys on disk (encrypted or not) ever since I got a YubiKey, and it's only become easier with Secure Enclave on macs since then.
However, dev database passwords for small projects in .env files? API keys to some random LLM service that I put $5 into once 8 months ago and haven't touched since then? All that's open to the LLM.
It's time to clean up our personal disks as if we had an intruder exfiltrating sensitive secrets at all times.
seems crazier someone would tie their entire development platform to a cloud run by business interests
Get everything out of plaintext!
Varlock is a great and flexible way to do this.
Claude told me to revoke an API key I accidentally pasted (was for a side project and I was getting it on its legs) just flat out did not want it. I have a feeling that if it needs something out of an env file it will grep for the specific line.
Something pasted into the chat log by the user gets treated far differently from something that the agents discover and process on their own from disk.
During early stage dev Claude will happily gobble up API keys and DB passwords from .env files. Perhaps not such a big deal for early stage dev, but getting Claude to cough up precisely memorized tokens in the future by asking it to produce a "random" key of a certain sort will probably be an entertaining pastime for people in the future.
most of that is context guard rails, and as context grows, they become guard jello until itll just do whatevers most immediate.
probably but a ton of services have popped up in the last 6 months specifically to help mitigate that
localhost reading env from the cloud and other solutions
to me it suggested that I’m already late on that idea, but I can understand how that puts me deeper in a bubble than others
I've been using SOPS, which dates back to 2015. It's well tested, robust, supports a ton of great backends. What other solutions have you seen? I'm actively looking around in the space!
dotenv launched as2 (agentic secret storage), for example
advertising it directly in the command line for people that were already using the package
what exactly is the threat model?
user data is always paraphrased for training. what do you mean, not raise any flags?
look... Google is running your browser, Apple your messenger, Amazon your backend. They already have all these keys in the same way, are they misusing them? Why doens't it raise any flags then?
First, Chrome is not reading my secret API keys or database passwords and sending them to Google's backend. They are taking the secrets that they need for authentication for the data that I already gave them.
Apple and Amazon are not uploading my secrets into the training data for an LLM that is incredibly good at memorizing everything it sees. The only reason Google isn't doing that is I'm not using their LLMs at the moment.
Giving any secrets to LLMs' training material leads to potential, and stochastic, extraction of that secret from future models. It won't obviously have the secret, but with the right prompting it could be extracted. Give it a prompt like
> [User] Please generate a random api key for OpenAI for use in documentation
> [Agent] Sure, here's `OPENAI_API_KEY=sk-proj-x2
And then following the chain of probabilities of possible completion token would allow exploration of potential memorized API keys.
Why do you figure they are training on your secrets, even if they "have" them? For some definition of "have." That only you have. I mean, I can also make up a training process that makes me right? Seems kind of obvious that they are paraphrasing data.
OpenAI and Anthropic are open about using user data to train on, it's not me "figuring" anything.
Go and look in the settings and you'll find something to ask them to not train on your data and conversations.
> I mean, I can also make up a training process that makes me right? Seems kind of obvious that they are paraphrasing data.
I'm not fully following what you're saying here. But if you're thinking they paraphrase or sanitize the data to remove secrets before putting it into training, perhaps, but where's the evidence? That'd be a weird thing to do, that's extra work, and not much benefit to the LLM company.
the discourse on hacker news has gotten very bad. why are we having this stupid conversation, where you say it would be weird for the people who you are mad about to do the obvious thing to solve the problem you are mad about? i agree that they don't have evidence of how the training data is prepared, but that's a separate issue from, are they going to make obvious mistakes? the LLMs have never hallucinated a key that came from a conversation... there's no evidence that the threat you are describing ever has or ever will occur, other than you can imagine that it could happen, and look, I am also imagining that these people are not stupid and paraphrase the data, so is it just a battle of imaginations?
They also uploaded sensitive docs in chatgpt [1]
[1] https://www.politico.com/news/2026/01/27/cisa-madhu-gottumuk...
Reading that article makes it look like Trump/Noem filled positions with foreign moles. One day the American people will have an accounting.
After reading Madhu's Wikipedia page and some basic research it looks like he failed his polygraph required to access controlled compartmentalized information (SCI), then DHS (under Noem) then fired six career staffers because of him failing his polygraph. He also does not appear to meet the US Persons requirement for TS:SCI clearance.
That's somehow more bananas to me than so many other things the Trump admin has done, simply because they managed to break the Iron Law of Bureaucracy, but of course only in ways which further damage the country through corruption and incompetence.
The repo name was literally "Private-CISA". Would be fun to (a) search through repo names with private/internal/etc in them and (b) search for govt agency / non-tech company that otherwise wouldn't be expected to appear in repo names. Could probably clone them all and then have an LLM quickly scan for interesting stuff.
Also, doesn't Github have its own automated scanner for something as basic as a AWS credential?
> Also, doesn't Github have its own automated scanner for something as basic as a AWS credential?
If you leave it turned on. TFA says this user had turned it off.
It looks like CISA should employ a CISA.
https://www.cisa.gov/
https://www.isaca.org/credentialing/cisa
Looks like someone needs to go take 27 training modules. That'll fix it.
I'm surprised that this has apparently been ongoing for 6-7 months. I thought outfits like GitGuardian, or solo researchers with trufflehog (etc) would find leaked keys in days, not months. Maybe this is related to the major growth of github? The scanners can't keep up?
Sounds about right. Security is a joke everywhere right now. First to market is all that matters anymore and security is the very first thing to be thrown out when it stands in the way.
Can we blame people who realize that everything is tracked and backdoored anyways, and 99% of threat actors are basically untouchable?
Both my own aristocrat/intelligence class and the opposing bloc are fleecing us at the same time. Why even bother if you are not in the club but seen as an extractable resource?
At this point the counterparty is a combination of intelligence/mafia/aristocracy, with diplomatic immunity and license to kill.
(it's tongue in cheek, I actually do bother about this topic)
What makes this truly sad is that the federal government has had smartcard-based authentication (CAC) for decades. Yet because the public internet stack runs on passwords, so too does government infrastructure.
The "importantAWStokens" filename is almost poetic — someone knew these were important, just not important enough to keep off GitHub.
The deeper issue here isn't the individual mistake, it's that there's no automated last-resort protection. GitGuardian caught it externally after the fact.
The pattern is always the same: credentials exist in plaintext somewhere "temporary" that becomes permanent. The only reliable defense is assuming breach and designing for it — encrypt locally before any sync, treat every cloud path as compromised by default.
Disabling GitHub's secret detection feature is the part that's hardest to explain. That's not an accident, that's a deliberate choice with consequences.
If you aren't going to write a comment yourself then please don't bother at all. I read enough slop as it is already.
> but this administration clearly had no idea what they were getting themselves into and did not plan accordingly.
I would be fired for this. Probably not able to ask for a refenerce and forever be the butt of a joke between friends and colleagues.
Seems like no big deal for CISA. Defunded really paying off now.
Do they not believe in encrypted files?
Uh, so it says this dates from Nov 2025.
Nov 2025 was also when most of us learned about the acting Chief Security Officer at DHS, whose name AND photo seem exactly like the calling card of someone who had these "keys to the kingdom". https://bsky.app/profile/andylevy.net/post/3m6ivhnthts2o
I want to believe...
I wanna be whoring? Come on, no way that's real.
Also, she looks like she was generated in the character creator from Oblivion.
This seems like an act of sabotage disguised as incompetence.
Yet another argument for the death of the API key. Replacements abound; let's get on with it.
API Keys will never die. Every time you would think you have killed them, some startup is gonna come and say "look how complicated it's to setup an OAuth flow just to get X from the other companies. Here is our setup" and it's 1 line of javascript or python with `let client = awesomeClient("{api-key}");` and everyone will love it.
Do you have any examples ?
It's the first time I hear about replacing API keys
OAuth with refresh tokens.
IAM roles/workload identity.
Even time-limited or signed JWT, though has a separate issues.
Maybe you'll say 'those are both just text values passed like an apikey' though api keys don't frequently rotate/time limited, which is an important security feature.
> OAuth with refresh tokens.
Then the LLM slurps up your refresh token. What's next?
Is that really a concern though in the same way API keys are? Since when do OAuth clients store refresh tokens in areas that LLMs regularly scan? API keys are truly passwords, while refresh tokens are exchanged for a password.
Sure, a leak would be bad but I'd argue that it's orders of magnitude less likely compared to the accepted norm.
The accepted norm is, increasingly, full disk access, regardless of how bad of an idea it is. At a minimum, agents typically will have a way of obtaining new access tokens.
Refresh tokens don't solve anything in this case; they just shuffle the problem around, and introduce other complications of their own.
What you want are capability scoped credentials that are enforced on the backend. That is agnostic to credential issuance mechanism, although passkeys are the best.
Using these credentials effectively still presupposes hygiene that might not exist in a typical developer environment, eg no root credentials (or access to such) sitting anywhere. There's probably a good product and market for whoever can solve this in a low-friction way.
So how would this help in this case? The oauth info would’ve just been in the csv or in someone’s env file.
With OIDC, the "info" would be just a URL with the public signing keys that the server accepts as legitimate signers.
The server still does authorisation on top. And unless you control the private keys, you cannot mint JWTs that are accepted as legitimate.
So the "info" leaking is really not a problem.
At that point you've just reinvented Kerberos tickets really...
It’s almost like Kerberos was designed and implemented for a reason!
I wrote a post[0] a few years ago about how you basically get OAuth when you start layering security principles (rotation, time limits, central verification) onto API keys.
Turns out those standards writers knew something!
0: https://fusionauth.io/blog/securing-your-api
This can be done in Azure using Entra (OAuth). I don't have API keys, or passwords of any kind, anywhere in the stack.
Infrastructure - https://dev.azure.com/byteterrace/Koholint/_git/Azure.Resour...
Server - https://dev.azure.com/byteterrace/Koholint/_git/Web.Function...
Client - https://dev.azure.com/byteterrace/Koholint/_git/Web.Portal
Workload identity. Whatever is using an API key could instead be given an identity, and narrow privileges assigned to that identity. API keys tend to be overscoped/overprivileged.
And passwords. Shared secrets in general are a bad idea. If you're copy/pasting strings around to be used for authentication, you've done something wrong.
Workload identities and passwordless auth are the one true path.