Microsoft is paying top dollar for MarkMonitor, aren't they supposed to proactively register obvious typos so this kind of thing doesn't happen to their clients?
My guess is that MarkMonitor is mainly used for their brand-relevant domains (microsoft, office 365, github (main site), etc), as opposed to one that a small subset of a small subset of their users of one service will use - I would imagine that microsoft likely owns hundreds of domain names and doesn't pay MarkMonitor to monitor every single one
That bug is incredibly dumb and obvious. There's been a PR to fix it for over a year with no attention.
I bet there's not a dedicated "github domain names" team, it's probably part of some overworked platform or infrastructure team, and there's no chance in hell any email you send to microsoft or github will end up with that team ever.
You won't have anyone to transfer the names to, you'll just be holding them and paying for them forever.
The best thing you can do if you want to fix this is:
1. Don't make typos.
2. Email github and tell them to reserve typosquat domains, and know it will get ignored, or _maybe_ added to a backlog and ignored for at least the next 15 years
3. Don't make typos.
4. Don't use ghcr for anything, and always mirror public ghcr.io packages using a "bot" github account with only permissions to public repositories to minimize blast radius.
Actually, the best bet to get this fixed is to wait for Microsoft to provide "Email Github Copilot support", hope that they hooked it up so the AI is capable of making purchase decisions, and convince it to purchase about 6000 domain names that might be typoes for security reasons.
What it's funny it's that because tokenization there is a non zero chance a LLM audit may not see anything wrong here, similar to the strawberry problem.
Why does it seem companies hate subdomains so much? Why is this not just registary.github.com or something? It's like they are trying to get people to fall for phishing by creating so many random domains.
It’s best security practice to host user-generated content on a separate domain to opt into browsers’ cross-domain security policies. Hence ghcr.io, githubusercontent.com, fbimg.com, etc.
Not a web programmer, so know cross-domain only for hearsay :(
It does not seem to hinder e.g. Google using google.com, youtube.com, gmail.com, and several (many?) others to collect your data. Do you say security and privacy work differently here?
In those cases, the company controls all of the code running on those sites, so it's desirable for them to share data and cookies in particular. (e.g. any google.com site can read your login cookie)
In the case of user data domains, intentionally in the design of the service or via a security hole, users may be able to execute code and read cookies (e.g. in JavaScript on a page hosted on githubusercontent.com) and that's undesirable.
Sure, I see why as a company you don't want user data in your domain.
But if the different domain name gives good protection / isolation, why does Google still use completely different domains for different services with content controlled by them. I cannot believe they are interested in protecting users from data collection.
Interestingly, the GitHub doco says outright that it superseded docker.pkg.github.com. ; so it was a conscious choice to go with this domain naming scheme instead of that one.
I've noticed this too. Why does amazon have aboutamazon.com and Google have developers.googleblog.com? They literally have their own .google TLD but still choose this weird domain.
Same with local governments. They love something really random like <countyname>proptaxpayment.org instead of treasurer.<countyname>.gov. It's exactly the kind of domain you are told to watch out for, but actually legit.
A common scenario I've seen in the case of local governments is that a department (e.g. the Assessing Department) contracts with a vendor to run the website and has no idea how DNS works, and the vendor defaults to registering new domains for their clients since that's the easiest when dealing with non-technical clients. Texas alone for example has 254 countries, the vast majority of which are very small and have effectively no full time IT department, so when these vendors are engaging new clients, low IT expertise is the norm by volume.
The local government itself may have an IT department, but they may not know how to create a subdomain, or even be aware this contract is being made and the site is being set up until after it's announced to the public.
One reason why you should never think or say ghcr, but always github container register, even if that is longer. You should have enough time for not getting trapped.
Root cause a stupid FLA of course. For several months I thought it means Google whatever register.
Does OAuth reuse tokens across domains? If not, doesn't this just mean it is requesting an auth token for ghrc (the "fake" domain) but it can't access any auth tokens for ghcr (the real domain)?
Blog author (and OCI maintainer) here. The request to get a bearer token sends the password or PAT using the basic auth header, base64 encoded, but otherwise clear-text. That's the request the www-authenticate header is triggering. Once the token is received, the registry uses that to verify access, and that eventually expires. But the attacker isn't getting the token, they are requesting the credentials that would be used to acquire a bearer auth token.
Reminder not to use goofy TLDs, being cute is not worth it when compared to security. There's no guarantees that the process for taking down a malicious domain will be as smooth as a .com.
I'd rather deal with US verisign rather than the British Indian Ocean territory or colombia or anguila
GitHub Container registry does not even support fine-grained tokens, instead it uses classic ones [1], which makes this even more dangerous.
[1] https://docs.github.com/en/packages/working-with-a-github-pa...
Edit: most relevant issues?
https://github.com/orgs/community/discussions/38467
https://github.com/github/roadmap/issues/558
Are there any additional mitigations folks are using for this? This issue is the only reason we can’t turn classic PATs off entirely.
Short lifetime mandatory reauth to enterprise SSO seems to be the best available, but it’s inconvenient for the single Classic PAT we actually need.
Someone near a computer that is feeling generous should buy up all the typo'd domain names and hand them over to Microsoft.
Microsoft should rename the registry. This is a horrible name. I know I've typo'd it before.
Microsoft is paying top dollar for MarkMonitor, aren't they supposed to proactively register obvious typos so this kind of thing doesn't happen to their clients?
My guess is that MarkMonitor is mainly used for their brand-relevant domains (microsoft, office 365, github (main site), etc), as opposed to one that a small subset of a small subset of their users of one service will use - I would imagine that microsoft likely owns hundreds of domain names and doesn't pay MarkMonitor to monitor every single one
Good luck with that.
People over in this github-actions issue are struggling to get github's attention for a 1-line fix to stop hanging jobs forever https://github.com/actions/runner/issues/3792#issuecomment-3...
That bug is incredibly dumb and obvious. There's been a PR to fix it for over a year with no attention.
I bet there's not a dedicated "github domain names" team, it's probably part of some overworked platform or infrastructure team, and there's no chance in hell any email you send to microsoft or github will end up with that team ever.
You won't have anyone to transfer the names to, you'll just be holding them and paying for them forever.
The best thing you can do if you want to fix this is:
1. Don't make typos.
2. Email github and tell them to reserve typosquat domains, and know it will get ignored, or _maybe_ added to a backlog and ignored for at least the next 15 years
3. Don't make typos.
4. Don't use ghcr for anything, and always mirror public ghcr.io packages using a "bot" github account with only permissions to public repositories to minimize blast radius.
Actually, the best bet to get this fixed is to wait for Microsoft to provide "Email Github Copilot support", hope that they hooked it up so the AI is capable of making purchase decisions, and convince it to purchase about 6000 domain names that might be typoes for security reasons.
Fairly compelling attack vector because it took several readings for me to even see the problem with the domain.
You and many others. Including people who retry multiple times, and even reboot their machines.
* https://stackoverflow.com/a/66985424/340790 (Spot the answerer's account name!)
* https://forums.docker.com/t/docker-unable-to-push-to-ghrc-io...
https://github.com/search?q=ghrc.io&type=code
Took the article pointing out that the c and r were transposed for me to even notice there was a problem!
Yep this is the sort of typo error I make probably 10 times a day.
What it's funny it's that because tokenization there is a non zero chance a LLM audit may not see anything wrong here, similar to the strawberry problem.
The problem here is GitHub's terrible domain name.
The container registry has a horrible name.
Why does it seem companies hate subdomains so much? Why is this not just registary.github.com or something? It's like they are trying to get people to fall for phishing by creating so many random domains.
It’s best security practice to host user-generated content on a separate domain to opt into browsers’ cross-domain security policies. Hence ghcr.io, githubusercontent.com, fbimg.com, etc.
https://www.reddit.com/r/webdev/comments/lg9xnm/why_do_some_...
Not a web programmer, so know cross-domain only for hearsay :(
It does not seem to hinder e.g. Google using google.com, youtube.com, gmail.com, and several (many?) others to collect your data. Do you say security and privacy work differently here?
In those cases, the company controls all of the code running on those sites, so it's desirable for them to share data and cookies in particular. (e.g. any google.com site can read your login cookie)
In the case of user data domains, intentionally in the design of the service or via a security hole, users may be able to execute code and read cookies (e.g. in JavaScript on a page hosted on githubusercontent.com) and that's undesirable.
Sure, I see why as a company you don't want user data in your domain.
But if the different domain name gives good protection / isolation, why does Google still use completely different domains for different services with content controlled by them. I cannot believe they are interested in protecting users from data collection.
Interestingly, the GitHub doco says outright that it superseded docker.pkg.github.com. ; so it was a conscious choice to go with this domain naming scheme instead of that one.
* https://docs.github.com/en/packages/working-with-a-github-pa...
I've noticed this too. Why does amazon have aboutamazon.com and Google have developers.googleblog.com? They literally have their own .google TLD but still choose this weird domain.
Same with local governments. They love something really random like <countyname>proptaxpayment.org instead of treasurer.<countyname>.gov. It's exactly the kind of domain you are told to watch out for, but actually legit.
A common scenario I've seen in the case of local governments is that a department (e.g. the Assessing Department) contracts with a vendor to run the website and has no idea how DNS works, and the vendor defaults to registering new domains for their clients since that's the easiest when dealing with non-technical clients. Texas alone for example has 254 countries, the vast majority of which are very small and have effectively no full time IT department, so when these vendors are engaging new clients, low IT expertise is the norm by volume.
The local government itself may have an IT department, but they may not know how to create a subdomain, or even be aware this contract is being made and the site is being set up until after it's announced to the public.
Probably, it's cool, and honored inside an org to operate a separate domain service vs go ask for a permission for a subdomain to another team.
insecurity through obscurity
One reason why you should never think or say ghcr, but always github container register, even if that is longer. You should have enough time for not getting trapped.
Root cause a stupid FLA of course. For several months I thought it means Google whatever register.
whois says it's registered by dynadot, so it's probably worth contacting their abuse email: abuse@dynadot.com
Previously on Hacker News at https://news.ycombinator.com/item?id=44974240 .
There are alot of open source projects using this domain https://github.com/search?q=ghrc.io&type=code
GitHub should a have tool internally to create bulk and send it as a fix
they probably do, they already have one that identified credentials posted to github repos by accident.
That's a fairly impressively sized list.
Is the danger here token replay? It's using Bearer tokens, so it's not sending a password over:
<https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Aut...>
Threats section for Bearer tokens: <https://datatracker.ietf.org/doc/html/rfc6750#section-5.2>
Does OAuth reuse tokens across domains? If not, doesn't this just mean it is requesting an auth token for ghrc (the "fake" domain) but it can't access any auth tokens for ghcr (the real domain)?
Blog author (and OCI maintainer) here. The request to get a bearer token sends the password or PAT using the basic auth header, base64 encoded, but otherwise clear-text. That's the request the www-authenticate header is triggering. Once the token is received, the registry uses that to verify access, and that eventually expires. But the attacker isn't getting the token, they are requesting the credentials that would be used to acquire a bearer auth token.
Damn, this can pick a typo from a CI job and do mean things.
Reminder not to use goofy TLDs, being cute is not worth it when compared to security. There's no guarantees that the process for taking down a malicious domain will be as smooth as a .com.
I'd rather deal with US verisign rather than the British Indian Ocean territory or colombia or anguila
The .io TLD is administered by Afilias which is an American corporation.
Afilias was sold to Ethos Capital and the whole domain is a mess:
https://en.m.wikipedia.org/wiki/.io