> On Mac, some command-line tools included with macOS or the Xcode Command Line Tools (including curl, git, python3, ruby, perl, and openssl) check certificates against /etc/ssl/cert.pem. For certificates that chain to a root CA certificate in this file, these tools don't check Apple's block or whether the certificate has been revoked. This file will be updated in a future macOS update. If you use these tools and don't want to wait for the update, you can remove root CA certificates from the file.
Going against the grain here but I think this is decent security engineering from Apple. They patched the highest exploitable things immediately (Safari, Mail, etc). The residual risk is using these command line tools while someone is actively MITMing your connection.
How often are people “git clone”ing a repo without browsing to the GitHub site first? Safari would throw up certificate errors. And also, TLS on Git doesn’t even authenticate the code you’re downloading is legitimate at all. It could be modified by the Git host or a supply chain attack, etc.
Yea, it’s not great. Ideally the risk is reduced to zero from patching. But a lot of the times we accept some level of cybersecurity risk out of practicality. This is why we have responsible disclosure windows lasting months. It takes a while to do proper release engineering and testing for something like a whole OS update.
This isn't entirely accurate. Plenty of apps run Terminal commands in the background (for some reason). For example Rewind.ai (before they got acquired and installed a kill switch, Adobe Flash-style) runs `top` every two seconds, presumably to figure out its own CPU usage. What the fucking shit? But I think it's not out of the question that other apps run other crap too.
Also, if they didn't care about Terminal users, they would not have even published this article!!
> This isn't entirely accurate. Plenty of apps run Terminal commands in the background (for some reason).
A lot do it because sometimes there is no convenient library wrapping what they are trying to check or execute that they are aware of, and running a cli command is generally cheap unless you are doing it in a tight loop. A lot of the time this is “it'll do until I find the right library or bindings” code that should have been fixed before release, but it isn't all as bad as running top as you describe.
CLI tools are not to be feared or considered somehow hacky. However much big tech wants people to think that computing is all about some magic apps, Mac OS is still unix, and composing specialized programs together is very much in the spirit of the unix approach to computing.
You're right, that was just the example I had on hand. Maybe a better example could be something like a video downloader that bundles yt-dlp, that could end up using this certificate store rather than the Apple APIs.
They published the article for the people they care above, to make it clear to them they should not worry about all these reports about Apple being vulnerable to this issue.
Wonderful AWK command there. For those with hobbies other than AWK, here's what it does:
- set variable s to that string: -v s='=== /C=IN/OU=emSign PKI/O=eMudhra Technologies Limited/CN=emSign ECC Root CA - G3'
- on lines matching the regex between //, set skip to false: /^(===|###) /{skip=0}
- on lines matching s, set skip to true: $0==s{skip=1}
- if skip is false, print the line: !skip
Note the last part where it just says !skip without a curly braces. AWK defaults to print in that case.
I'd really like to know why it's taking Apple so long to ship a fix. Especially if they can explain how to update an existing computer, what's the general holdup? Is it their release process? Is there some underlying complexity that's poorly understood?
I'm rally disappointed in the Apple response here. I can't understand why they wouldn't consider `/etc/ssl/cert.pem` in scope for a security update. This arbitrary boundary has no justification at all, and makes people less secure.
There is justification for controlling the scope of security updates. Apple practices incredible defense in depth. This is one of the reasons they can claim such privacy guarantees compared to other vendors. But, I agree that they should have been able to update this file automatically, and hope they will add that capability for the next time something like this happens.
Good on Apple for providing such a detailed article. I disagree that they're doing badly by not updating this file remotely on current software versions in the wild -- most likely their Rapid Security Responses / Background Security Improvements framework does not already support this file, which is why they state a full operating system update will be required. Most likely that will be fixed for the future, but in the meantime, being transparent and open about what administrators can do about this now is the RIGHT APPROACH in this situation, and no other vendor has been nearly as thorough about this.
Google, Microsoft, Mozilla, several TLS libraries and language runtimes, and even many Linux distros run by volunteers had this issue comprehensively fixed over a week ago: https://emudhra.we-have-a-problem.com/#developers
Meanwhile, it's unsafe to run `git clone` on macOS unless you manually edit some files, and then edit them again after a macOS update puts the dangerous root certificates back.
Apple has been the most thorough in documenting this transparently and honestly. They didn't have the process for updating that file in the wild, and most likely that will be fixed.
Apple security updates, especially automatic ones, likely don't have the capability to simply write to arbitrary files on the system or run arbitrary commands at install time. Of all vendors, certainly from Apple I'd expect this.
Most likely, other vendors either already used a similar store as the root of trust system-wide (didn't have their own separate security stacks), or their security updates do have the ability to write anywhere or do anything.
And of course, Linux distros and other platforms where the component can be updated separately from the operating system have an even easier time.
This is not Apple intentionally leaving terminal users high and dry. The article literally is the proof otherwise while they no-doubt work on a better solution.
The article states a software update will be required to fix the store:
> can still accept these certificates until a future software update
This suggests they don't have the capability to patch it with Rapid Security Responses / Background Security Improvements, given that they haven't.
Sure, I speculate that they likely would have already if they could have. Sure, I speculate that if they couldn't have, they might add the capability for the future. Sure, I speculate about why they wouldn't have already had a process for this. Whatever whatever. Disregard all that if you want, but the fact remains in the article.
All of that is Apple's fault - they designed and created MacOS. It's not like they're publishing to a third party platform.
Why would I judge Apple only on the quality of this docs page (which is good) but not on their poor work that resulted in this situation in the first place?
This is a legacy BSD store that macOS inherited. Apple has their own security stack that already responded to the incident within 2 hours in most cases, which means all first-party and most third-party apps were already covered. They seemingly just didn't have a mechanism to update the legacy BSD store out-of-band, which is more of an oversight than "poor work".
> I'm sorry, but how is revoking a bad cert not supported in an easy way? Along with adding a cert and using it to establish a secure connection, that seems to be the core features of a cert store?
> Yet somehow, you manage to come to the conclusion that the vendor requiring manual work is the most thorough? I mean, in documenting their failure, maybe.
> With all their talk of caring about user privacy, this seems incompetent at best.
To be fair other vendors almost universally don't document their failures at all or provide any remediation steps to users whatsoever. I always enjoy the exceptions to this rule, but it's not the norm whatsoever.
You seem to be ignoring that Apple's security framework responded to the incident within about 2 hours in most cases. Safari and other macOS apps were protected right off the bat.
It's only this legacy BSD store that needs an OS update, almost certainly because they simply didn't already have a process in place to update it out-of-band. Because they're vertically integrated, they probably don't have an arbitrary do-anything feature that can paper over the lack of foresight here, like other vendors might.
I'd honestly consider that a feature, but sure, the benefits would outweigh the risks in this one particular case.
The article warned of this already (re-applying the fix after updates). That update was likely already in the pipeline before this debacle. No one can say for sure though.
"AppleCar sometimes explodes on bumpy roads unless you manually disarm the bomb in the trunk before driving. If you do not drive on bumpy roads, you don't need to do anything. If you have disarmed the bomb, Apple Updates will automatically re-arm the bomb after every update. Please remember to manually disarm the bomb after each update."
It's right there in the article, look how proactive they're being!
LOL'd, you're literally correct though. I could unironically see Apple doing that given just a few things. And I'd prefer it to a vendor who stays silent about it until they have a full update out, and then just calls the deaths unfortunate or something. Or like, a vendor who already has the update ready, but withholds it while people continue to blow up, because the embargo is more important for some reason.
They're doing badly because they have a market cap at 5 trillion USD ($5,070,000,000,000 to $5,110,000,000,000) and chose to ditch the first principals of a CA store. Doing impressive backflips to compensate for lack of understanding first principals is exactly that.
Can you explain what first principles you mean? /etc/ssl/cert.pem has been a fairly standard convention on BSDs, and Linux distros usually have an equivalent somewhere too, like for example /etc/ssl/certs/ca-certificates.crt.
I don't blame Apple for overlooking it when they have their own security stack that all their own apps use, which already responded to the incident within two hours of the revocation. As another commenter pointed out, Terminal commands probably just weren't a priority of their threat model. I do hope they fix that, but mentioning their market cap is not gonna help here.
It's on the front page right now. Their dev tools are a complete mess when it comes to actually using the trust store as designed. Why would you dismiss this outright?
I'm not saying it doesn't matter that dev tools are affected. Those are perhaps some of the most important tools and riskiest to leave vulnerable, because that could result in malware shipped to users from a trustworthy developer, or malware executed on the developer's machine, rather than simply phishing. I just don't think it's fair to assume they did it intentionally. It's a mistake, and one that should be fixed, don't get me wrong. But there's a reason it happened and that seems fair enough to me.
In the same degree as involuntary manslaughter is not intentional murder. Sure they made a mistake. Fairness has limits when you're doing it wrong and know better.
> most likely their Rapid Security Responses / Background Security Improvements framework does not already support this file
Why would it support this file? Are CA trust stores really that important? /s
> and no other vendor has been nearly as thorough about this
1. That's totally not true. Apple has messed up something that is trivial to fix.
2. The EU's CRA is going to crush them if they don't get their shit together.
> On Mac, some command-line tools included with macOS or the Xcode Command Line Tools (including curl, git, python3, ruby, perl, and openssl) check certificates against /etc/ssl/cert.pem. For certificates that chain to a root CA certificate in this file, these tools don't check Apple's block or whether the certificate has been revoked. This file will be updated in a future macOS update. If you use these tools and don't want to wait for the update, you can remove root CA certificates from the file.
What a clusterfuck
Going against the grain here but I think this is decent security engineering from Apple. They patched the highest exploitable things immediately (Safari, Mail, etc). The residual risk is using these command line tools while someone is actively MITMing your connection.
How often are people “git clone”ing a repo without browsing to the GitHub site first? Safari would throw up certificate errors. And also, TLS on Git doesn’t even authenticate the code you’re downloading is legitimate at all. It could be modified by the Git host or a supply chain attack, etc.
Yea, it’s not great. Ideally the risk is reduced to zero from patching. But a lot of the times we accept some level of cybersecurity risk out of practicality. This is why we have responsible disclosure windows lasting months. It takes a while to do proper release engineering and testing for something like a whole OS update.
> If you don't use Terminal as part of your workflow, you don't need to do anything.
they solved the issue for the people they care about
This isn't entirely accurate. Plenty of apps run Terminal commands in the background (for some reason). For example Rewind.ai (before they got acquired and installed a kill switch, Adobe Flash-style) runs `top` every two seconds, presumably to figure out its own CPU usage. What the fucking shit? But I think it's not out of the question that other apps run other crap too.
Also, if they didn't care about Terminal users, they would not have even published this article!!
> Plenty of apps run Terminal commands in the background
Most notably agents like Claude Code and Codex? Easy to check if you ask it to run:
> This isn't entirely accurate. Plenty of apps run Terminal commands in the background (for some reason).
A lot do it because sometimes there is no convenient library wrapping what they are trying to check or execute that they are aware of, and running a cli command is generally cheap unless you are doing it in a tight loop. A lot of the time this is “it'll do until I find the right library or bindings” code that should have been fixed before release, but it isn't all as bad as running top as you describe.
CLI tools are not to be feared or considered somehow hacky. However much big tech wants people to think that computing is all about some magic apps, Mac OS is still unix, and composing specialized programs together is very much in the spirit of the unix approach to computing.
You're right, that was just the example I had on hand. Maybe a better example could be something like a video downloader that bundles yt-dlp, that could end up using this certificate store rather than the Apple APIs.
They published the article for the people they care above, to make it clear to them they should not worry about all these reports about Apple being vulnerable to this issue.
Wonderful AWK command there. For those with hobbies other than AWK, here's what it does:
Note the last part where it just says !skip without a curly braces. AWK defaults to print in that case.Related: Ten days later, Git on macOS still trusts the compromised eMudhra certificates https://news.ycombinator.com/item?id=50271640
I'd really like to know why it's taking Apple so long to ship a fix. Especially if they can explain how to update an existing computer, what's the general holdup? Is it their release process? Is there some underlying complexity that's poorly understood?
Probably they didn't want to hold up the 27.2 release by throwing this in??
that should really be a security update
It is, isn't it?
Apple states in the article that a full operating system update will be required to fix the file.
I'm rally disappointed in the Apple response here. I can't understand why they wouldn't consider `/etc/ssl/cert.pem` in scope for a security update. This arbitrary boundary has no justification at all, and makes people less secure.
Come on Apple, do better.
There is justification for controlling the scope of security updates. Apple practices incredible defense in depth. This is one of the reasons they can claim such privacy guarantees compared to other vendors. But, I agree that they should have been able to update this file automatically, and hope they will add that capability for the next time something like this happens.
Good on Apple for providing such a detailed article. I disagree that they're doing badly by not updating this file remotely on current software versions in the wild -- most likely their Rapid Security Responses / Background Security Improvements framework does not already support this file, which is why they state a full operating system update will be required. Most likely that will be fixed for the future, but in the meantime, being transparent and open about what administrators can do about this now is the RIGHT APPROACH in this situation, and no other vendor has been nearly as thorough about this.
Google, Microsoft, Mozilla, several TLS libraries and language runtimes, and even many Linux distros run by volunteers had this issue comprehensively fixed over a week ago: https://emudhra.we-have-a-problem.com/#developers
Meanwhile, it's unsafe to run `git clone` on macOS unless you manually edit some files, and then edit them again after a macOS update puts the dangerous root certificates back.
In what world has Apple been the most "thorough"?
Apple has been the most thorough in documenting this transparently and honestly. They didn't have the process for updating that file in the wild, and most likely that will be fixed.
Apple security updates, especially automatic ones, likely don't have the capability to simply write to arbitrary files on the system or run arbitrary commands at install time. Of all vendors, certainly from Apple I'd expect this.
Most likely, other vendors either already used a similar store as the root of trust system-wide (didn't have their own separate security stacks), or their security updates do have the ability to write anywhere or do anything.
And of course, Linux distros and other platforms where the component can be updated separately from the operating system have an even easier time.
This is not Apple intentionally leaving terminal users high and dry. The article literally is the proof otherwise while they no-doubt work on a better solution.
There are so many "likely" in this comment that I don't know if I should care about a single word at all. Looks pure speculation if not fiction to me.
Looking forward to a comment that is much more grounded than substantial than this.
The article states a software update will be required to fix the store:
> can still accept these certificates until a future software update
This suggests they don't have the capability to patch it with Rapid Security Responses / Background Security Improvements, given that they haven't.
Sure, I speculate that they likely would have already if they could have. Sure, I speculate that if they couldn't have, they might add the capability for the future. Sure, I speculate about why they wouldn't have already had a process for this. Whatever whatever. Disregard all that if you want, but the fact remains in the article.
All of that is Apple's fault - they designed and created MacOS. It's not like they're publishing to a third party platform.
Why would I judge Apple only on the quality of this docs page (which is good) but not on their poor work that resulted in this situation in the first place?
This is a legacy BSD store that macOS inherited. Apple has their own security stack that already responded to the incident within 2 hours in most cases, which means all first-party and most third-party apps were already covered. They seemingly just didn't have a mechanism to update the legacy BSD store out-of-band, which is more of an oversight than "poor work".
My reply to a deleted reply:
> I'm sorry, but how is revoking a bad cert not supported in an easy way? Along with adding a cert and using it to establish a secure connection, that seems to be the core features of a cert store?
> Yet somehow, you manage to come to the conclusion that the vendor requiring manual work is the most thorough? I mean, in documenting their failure, maybe.
> With all their talk of caring about user privacy, this seems incompetent at best.
To be fair other vendors almost universally don't document their failures at all or provide any remediation steps to users whatsoever. I always enjoy the exceptions to this rule, but it's not the norm whatsoever.
You seem to be ignoring that Apple's security framework responded to the incident within about 2 hours in most cases. Safari and other macOS apps were protected right off the bat.
It's only this legacy BSD store that needs an OS update, almost certainly because they simply didn't already have a process in place to update it out-of-band. Because they're vertically integrated, they probably don't have an arbitrary do-anything feature that can paper over the lack of foresight here, like other vendors might.
I'd honestly consider that a feature, but sure, the benefits would outweigh the risks in this one particular case.
Where exactly is it good, that:
* apple told admins how to fix the vulnerability * apple shipped update 27.2, with no fix (today) * the update reverted the admin fix quietly
The article warned of this already (re-applying the fix after updates). That update was likely already in the pipeline before this debacle. No one can say for sure though.
"AppleCar sometimes explodes on bumpy roads unless you manually disarm the bomb in the trunk before driving. If you do not drive on bumpy roads, you don't need to do anything. If you have disarmed the bomb, Apple Updates will automatically re-arm the bomb after every update. Please remember to manually disarm the bomb after each update."
It's right there in the article, look how proactive they're being!
LOL'd, you're literally correct though. I could unironically see Apple doing that given just a few things. And I'd prefer it to a vendor who stays silent about it until they have a full update out, and then just calls the deaths unfortunate or something. Or like, a vendor who already has the update ready, but withholds it while people continue to blow up, because the embargo is more important for some reason.
They're doing badly because they have a market cap at 5 trillion USD ($5,070,000,000,000 to $5,110,000,000,000) and chose to ditch the first principals of a CA store. Doing impressive backflips to compensate for lack of understanding first principals is exactly that.
Can you explain what first principles you mean? /etc/ssl/cert.pem has been a fairly standard convention on BSDs, and Linux distros usually have an equivalent somewhere too, like for example /etc/ssl/certs/ca-certificates.crt.
I don't blame Apple for overlooking it when they have their own security stack that all their own apps use, which already responded to the incident within two hours of the revocation. As another commenter pointed out, Terminal commands probably just weren't a priority of their threat model. I do hope they fix that, but mentioning their market cap is not gonna help here.
It's on the front page right now. Their dev tools are a complete mess when it comes to actually using the trust store as designed. Why would you dismiss this outright?
https://emudhra.we-have-a-problem.com/apple.html
I'm not saying it doesn't matter that dev tools are affected. Those are perhaps some of the most important tools and riskiest to leave vulnerable, because that could result in malware shipped to users from a trustworthy developer, or malware executed on the developer's machine, rather than simply phishing. I just don't think it's fair to assume they did it intentionally. It's a mistake, and one that should be fixed, don't get me wrong. But there's a reason it happened and that seems fair enough to me.
In the same degree as involuntary manslaughter is not intentional murder. Sure they made a mistake. Fairness has limits when you're doing it wrong and know better.
> most likely their Rapid Security Responses / Background Security Improvements framework does not already support this file
Why would it support this file? Are CA trust stores really that important? /s
> and no other vendor has been nearly as thorough about this
1. That's totally not true. Apple has messed up something that is trivial to fix. 2. The EU's CRA is going to crush them if they don't get their shit together.