It's exhilarating (for the lack of the knowledge of a better word) to see OpenAI and Anthropic starting getting hacked and getting a taste of their own medicine. Brings back hope that actual software engineering is not dead yet.
I took over user accounts in Microsoft Copilot (there are caveats but still), and they asked me to provide additional information of the exploit (they wanted a video) and before I sent it they fixed the vulnerability, didn't pay me a cent. Fair? Idk.
That’s so low. That’s the level of bounty I awarded researchers when I was managing the bug bounty program at a small startup, it’s just ridiculous coming from a large company like OpenAI
Who says they don't? And if they don't, maybe its because others are using their hard-earned money to buy tokens from them to do it, so they don't really have to.
Having bit the bullet, I found OAI's models actually quite helpful and most of their offering half-decent. But their whole German website was incredibly concerning and the only reason I went for it was people claiming they're not far behind Anthropic on coding and generally superior for everyday tasks like coming up with recipes to cook from your leftovers.
Like half of the website was dysfunctiona and translations incredibly bad to the point of being nonsensical. That is if you were able to read them at all, because many just went off screen. Like, that's what your evil competitor does to your website after hijacking the domain, not you yourself, ffs
I find Sam Altman recent comments very "low" as well, neither matters. So in that case I think bounty hunters should just cold shoulder idiocracy and sell these to the underground. Let their customers be abused and complain about how OAI isn't protecting them. Each one of these types of events could cost them thousands of dollars in lost revenue and/or internal support.
2 trillion $ companies should have the money to pay for these exploits. If they don’t, go tucking exploit them. If they don’t give a fuck about stealing knowledge from copyrighted material and private git repos, light em up.
On the contrary, many (including myself) would argue that "the victim's amount of money/power" has absolutely no ethical relevance: "You shall do no injustice in judgment; you shall not be partial to the poor nor defer to the great, but you are to judge your neighbor fairly" (Leviticus 19:15).
I don't think so, businesses like Zerodium bought zero days in plain sight and I don't think they faced legal consequences. They don't operate anymore though. So basically I'd say it depends who you sell it to and how.
Treating people and businesses ethically that don't reciprocate doesn't work, clearly. Those people and businesses cry in their milk while they do exactly the same thing. There are no ethics in play here.
But, hey if you want to defend billionaire narcissists that are destroying our planet and trying to take computing freedoms then I guess: you do you.
Yeah, and some people just want the multi-billion dollar company that is currently destroying the planet while also permanently crippling social connections to just stop.
This isn't like stealing from a business. These are individuals most private conversations, potentially including their bank accounts etc. Poverty is no defence - and yes, I've spent a great deal of my life barely scraping by. You'll find poor people are significantly less tolerant of theft, especially from individuals, than rich. Whether that evidences the theft inherent in most forms of wealth generation I'll leave up to the reader.
Not worth the effort for a company supposedly worth a trillion dollars. The problem is oai can't afford all of its mistakes at the usual rate and they make bad press. A 100k bug could make a headline, but 5k for the same bug requires the average person to be educated enough to understand how bad these issues are.
Yeah... idk, sounds like when I was starting out almost 30 years ago my clients constantly lowballed me saying "but think of the exposition! this project will give you" lol
Were they not right? Just like money giving you some runway but will run out, the exposure is still only a leveler, it's still up to you to take advantage of it.
The "take over any instagram/meta account through their AI password reset chatbot" was going around and sold on shady forums for ~1 month before being patched.
Very few people would even have the knowledge of who, how, or where to "sell" such vulnerabilities. Going through a public bounty program is pretty much always preferable.
Don't worry, as more and more people get put of of work by AI the number of people willing to be felons will increase. Hell, they might even become president.
$5000 is a joke. They pay millions to their top devs and still have more holes than a colander so a nice guy shows you a huge opening in your platform and all you can give him is a pat on the shoulder? Did you fire your top devs and hire that guy for a million bucks plus options? Did you sue the security firm you paid millions to audit the issues?
I really hope they get hacked and crash down burning in flames like the Hindenburg
I think you know that the mentality of a company like this is “security is a risk but not a reward and whoever captures the reward kills everyone else”
Now what about those millions of lines of code? Wasn’t the agent swarm supposed to take care of it?
All jokes aside this is exactly what happens as you build larger monoliths, system complexity and bug surface increases logarithmically… so in all seriousness you want to reduce LOC as much as possible. Maybe this can teach OpenAI a lesson to produce software that’s correct and adhere to established engineering standards over sensationalised “we built so much code” promotional material.
I’m sure exponential is what parent meant, but geometric (edit: polynomial) is what actually happens, not exponential. Code complexity increases linearly when it doesn’t interact with other features, and geometrically when it does. In the worst case, complexity increases as the square of lines of code. That is still very, very bad, but it’s not exponential.
Much easier to reason and make decisions if the bar is "yeah it exists, but is it effective relative to its intended purpose?" This weeds out buggy code as well as under performant code. If it's not effective (broken, too slow, etc), it either needs a refactor or thrown on the scrap pile.
> Now what about those millions of lines of code? Wasn’t the agent swarm supposed to take care of it?
What's the joke? They're literally saying, every day, that this is true to a civilization endangering extent. They're making demands based on this, and politicians are taking up those demands.
And now they're paying bullshit $5K bug bounties for account-stealing exploits on their own platform. They should have paid more just the make the bug seem more grandiose; I don't know who let them get away with this - this is what happens when CEOs run their own PR I guess. It's basically an public announcement that they're total fakes.
i don't think LoC is the metric. sometimes fewer lines of code results in a larger attack surface through type coercion / widening, poor abstractions, hiding or stuffing behavior (modules that do everything), etc. to me, making code easy to understand, test, fuzz, and verify is more important. sometimes that requires more lines of code.
LoC *IS* the metric. It is obviously not perfect tho but still the best one to compare software surface. Including to compare the same solution in different languages and platforms. The less code you need to manage, the less code you need to change and the less code you need to hold in your head to understand it (or in the LLM context nowadays).
If they or their company use email "security" software that "scans" all links it's not necessarily the users fault / they don't in fact click. The mail is auto-confirmed the second it hits the inbox.
We've had to change our registration confirmation mails too recently to not confirm on GET only, but with another form - in hindsight it was stupid before but many services do that.
> We've had to change our registration confirmation mails too recently to not confirm on GET only, but with another form - in hindsight it was stupid before but many services do that.
Security software fetching links will not trigger this behavior. It needs to be done in the context of the actual ChatGPT user so that a JavaScript payload can run.
If merely GET on the link was sufficient to trigger the behaviors, the adversary generating the link would just GET it themselves.
Yes, if it requires the user session (xss related), like the article, there is no problem.
I was replying more generally about the "don't click links" advice and cases of server generated confirmation-token links (not saying this was the case at OpenAI)
I've worked at a company that did link mangling on everything except simulated phishing emails. The effect was that people were trained to trust the mangled links.
Oof. Together with the fact that it's possible to remotely access the file system (!) of computers ever having started a "Work" chat in the ChatGPT desktop app [1], this seems pretty bad.
Claude Code terminal has "remote control" option activated via /rc command. -- somewhat similar in that it still runs the session on the PC/Mac but allows the Claude authenticated user to access the session via internet / app
I am guessing this ChatGPT functionality is a similar Codex feature?
But this session is disconnected when the conversation is /exit ed
For my personal blog [0], I am still writing without LLM assistance. The goal is to test how well I understand the topic I am writing about, as well as to improve my own writing.
I think there was another HN discussion about it re bounty value and as far as I remember the issue is not that big companies are greedy in that case, but rather that over time it’s strategically better to have lower bounties so less people are interested in it.
5 or 50k doesnt matter to them. And this bug in terms of severity is very high imho.
> I'm trying not to have any important data to even lose
This is the way. Respect user privacy and keep as little data as possible, and not only do you become a less attractive target, if you are targeted the outcome won’t be so bad.
only $5000 for this, lmao. It should be $50k out of pure shame. If they don't make these things hurt just a little they're never going to take following best practices seriously enough. There will be 9 more anyways.
Meanwhile the actual user of the account probably has to sign in using a yubikey because they get "CyBeRsEcRuItY aLeRtS" when they try to update node in their project to fix a crash and have to switch to daybreak blue to continue (or attempt to trick the normal models into doing it and risk getting autobanned)
why didnt the black box of bullshit company detect this? for a group that touts every inane 'discovery' with hyperbolic bullshit, they really fall short on delivering anything of actual value, just rehashed bullshit.
Another classic OAuth implementation failure. Mismanaging the state parameter or leaking tokens via referer headers in redirects is Security 101. Wild to see this in a production environment of this scale.
The post seems to be talking about a html injection via a postMessage call where the receiver didn’t check who sent the message, which is a well known vulnerability entry point. Did I miss where OAuth had anything to do with this??
It's exhilarating (for the lack of the knowledge of a better word) to see OpenAI and Anthropic starting getting hacked and getting a taste of their own medicine. Brings back hope that actual software engineering is not dead yet.
> OpenAI confirmed both frontends rejected the reported messages, resolved the issue, and awarded a total bounty of $5,000.
It requires the user to click a link. But I still find this bounty very low. Others may not report them if such a small bounty is given.
I took over user accounts in Microsoft Copilot (there are caveats but still), and they asked me to provide additional information of the exploit (they wanted a video) and before I sent it they fixed the vulnerability, didn't pay me a cent. Fair? Idk.
Typical $MSFT behavior, I can see why NightmareEclipse (ie. YellowKey zero-day exploit) got released without prior fix.
A bounty like this is a polite "thanks asshole, fuck off, don't do it again" rate.
Paying effectively nothing for vulnerabilities just means they don't want to deal with the program.
Wild to think a company supposedly worth a $trillion+ is doing shit like this.
That’s so low. That’s the level of bounty I awarded researchers when I was managing the bug bounty program at a small startup, it’s just ridiculous coming from a large company like OpenAI
But on the other hand, you can use AI now to find those vulnerabilities.
If that is so, why doesn't OAI have a swarm doing this around the clock?
Surely with all their security programs, they would run testing on the ChatGPT implementation.
I speculate that the AI didn't find this and that human effort and intuition for where to look were needed to find this vulnerability.
Who says they don't? And if they don't, maybe its because others are using their hard-earned money to buy tokens from them to do it, so they don't really have to.
Having bit the bullet, I found OAI's models actually quite helpful and most of their offering half-decent. But their whole German website was incredibly concerning and the only reason I went for it was people claiming they're not far behind Anthropic on coding and generally superior for everyday tasks like coming up with recipes to cook from your leftovers.
Like half of the website was dysfunctiona and translations incredibly bad to the point of being nonsensical. That is if you were able to read them at all, because many just went off screen. Like, that's what your evil competitor does to your website after hijacking the domain, not you yourself, ffs
They should have paid in tokens?
A bounty amount is related to the blast radius and overall impact, not the effort required to find the issue
They have just announced free API credits for Pro plan users up to $500 per month. Irony.
To be fair, based on the profit margins indicated in DeepSeek’s reports, a $500 API credit might only cost them $5 to provide.
I find Sam Altman recent comments very "low" as well, neither matters. So in that case I think bounty hunters should just cold shoulder idiocracy and sell these to the underground. Let their customers be abused and complain about how OAI isn't protecting them. Each one of these types of events could cost them thousands of dollars in lost revenue and/or internal support.
2 trillion $ companies should have the money to pay for these exploits. If they don’t, go tucking exploit them. If they don’t give a fuck about stealing knowledge from copyrighted material and private git repos, light em up.
some people don’t want to be criminals as a matter of ethics
but hey you do you
True, however:
Ethics and legal requirements shouldn’t be conflated.
There are times when ethics are a higher standard than what the law requires, and in that case the ethics are no less important.
This isn’t one of those times.
Only if you don't believe the means justify the ends. Not saying I believe that myself
Yes, you have a moral obligation to Sam Altman’s trillion dollar business.
On the contrary, many (including myself) would argue that "the victim's amount of money/power" has absolutely no ethical relevance: "You shall do no injustice in judgment; you shall not be partial to the poor nor defer to the great, but you are to judge your neighbor fairly" (Leviticus 19:15).
Citing a fantasy novel to prove your point doesn't add anything to the discussion.
Is it criminal to sell an exploit?
Facilitation of a crime is a crime too. So I think in the US and most western countries, yes. At least if it gets used illegally later.
I don't think so, businesses like Zerodium bought zero days in plain sight and I don't think they faced legal consequences. They don't operate anymore though. So basically I'd say it depends who you sell it to and how.
Treating people and businesses ethically that don't reciprocate doesn't work, clearly. Those people and businesses cry in their milk while they do exactly the same thing. There are no ethics in play here.
But, hey if you want to defend billionaire narcissists that are destroying our planet and trying to take computing freedoms then I guess: you do you.
Yeah, and some people just want the multi-billion dollar company that is currently destroying the planet while also permanently crippling social connections to just stop.
Ah yes, OpenAI, the poster child for ethics
If only poverty were so simple
This isn't like stealing from a business. These are individuals most private conversations, potentially including their bank accounts etc. Poverty is no defence - and yes, I've spent a great deal of my life barely scraping by. You'll find poor people are significantly less tolerant of theft, especially from individuals, than rich. Whether that evidences the theft inherent in most forms of wealth generation I'll leave up to the reader.
Not worth the effort for a company supposedly worth a trillion dollars. The problem is oai can't afford all of its mistakes at the usual rate and they make bad press. A 100k bug could make a headline, but 5k for the same bug requires the average person to be educated enough to understand how bad these issues are.
Should be given $5-10million under NDA. No headlines.
The bounty payouts must be public in order to serve as an incentive.
That incentive works both ways. For instance employee sneaking in a security hole and then selling it to a mate so they can collect the bounty.
Just imagine what the press would be when somebody breaks that NDA
>should be paid multiple years of a high end vuln researcher salary per single discovery
What? Why would anyone work at a company paying them that much less than they pay the contractors?
OpenAI acknowledging and fixing the bug is worth millions
Yeah... idk, sounds like when I was starting out almost 30 years ago my clients constantly lowballed me saying "but think of the exposition! this project will give you" lol
Were they not right? Just like money giving you some runway but will run out, the exposure is still only a leveler, it's still up to you to take advantage of it.
Also, they were not OpenAI.
This is a pet peeve of tptacek:
> server-side vulnerabilities are worth very little, because they quickly get patched.
The "take over any instagram/meta account through their AI password reset chatbot" was going around and sold on shady forums for ~1 month before being patched.
then server side vulnerabilities should not be reported, instead sell the data, more profitable
Very few people would even have the knowledge of who, how, or where to "sell" such vulnerabilities. Going through a public bounty program is pretty much always preferable.
Most people don't want to be felons.
Don't worry, as more and more people get put of of work by AI the number of people willing to be felons will increase. Hell, they might even become president.
I think people who can hack, also know where they can sell it
Anyone can "hack". It's not really a skill anymore.
Indeed, it's like "here peasant, have this fortune of $5,000, may it change your life". Pathetic.
that wouldn't be the time worth of many people here to make the security report for OAI..
$5000 is a joke. They pay millions to their top devs and still have more holes than a colander so a nice guy shows you a huge opening in your platform and all you can give him is a pat on the shoulder? Did you fire your top devs and hire that guy for a million bucks plus options? Did you sue the security firm you paid millions to audit the issues?
I really hope they get hacked and crash down burning in flames like the Hindenburg
I wonder if the new economics of finding bugs means that they pay the amount they think the tokens cost to find it.
Over the probability that you'll find something.
I think you know that the mentality of a company like this is “security is a risk but not a reward and whoever captures the reward kills everyone else”
Now what about those millions of lines of code? Wasn’t the agent swarm supposed to take care of it? All jokes aside this is exactly what happens as you build larger monoliths, system complexity and bug surface increases logarithmically… so in all seriousness you want to reduce LOC as much as possible. Maybe this can teach OpenAI a lesson to produce software that’s correct and adhere to established engineering standards over sensationalised “we built so much code” promotional material.
Did you mean exponentially? Logarithmic increase doesn't sound too bad.
I’m sure exponential is what parent meant, but geometric (edit: polynomial) is what actually happens, not exponential. Code complexity increases linearly when it doesn’t interact with other features, and geometrically when it does. In the worst case, complexity increases as the square of lines of code. That is still very, very bad, but it’s not exponential.
What's the difference? From what I'm reading online geometric is just the discrete version of exponential
No difference, you’re right. I meant polynomial not geometric. :P
“Square of the lines of code”
Sounds exponential
That’s quadratic, not exponential. 2^n != n^2
And that's why sw eng interviews often have some basic math sprinkled in.
it's not like microservices fare better, when you introduce them, you've traded a hard problem for a hard problem and distributed computing...
Our industry has wasted hundreds of billions of dollars on this shit show called Microservices.
99.9% of shops need a monolith with same monolith having entry points for background workers and crons.
I like LOEC: Lines of Effective Code.
Much easier to reason and make decisions if the bar is "yeah it exists, but is it effective relative to its intended purpose?" This weeds out buggy code as well as under performant code. If it's not effective (broken, too slow, etc), it either needs a refactor or thrown on the scrap pile.
> Now what about those millions of lines of code? Wasn’t the agent swarm supposed to take care of it?
What's the joke? They're literally saying, every day, that this is true to a civilization endangering extent. They're making demands based on this, and politicians are taking up those demands.
And now they're paying bullshit $5K bug bounties for account-stealing exploits on their own platform. They should have paid more just the make the bug seem more grandiose; I don't know who let them get away with this - this is what happens when CEOs run their own PR I guess. It's basically an public announcement that they're total fakes.
Reducing attack surface is a lesson for everyone. Newcomers are impressed by massive piles of code.
Seasoned developers are unimpressed, or even disgusted, by the same massive piles of code.
i don't think LoC is the metric. sometimes fewer lines of code results in a larger attack surface through type coercion / widening, poor abstractions, hiding or stuffing behavior (modules that do everything), etc. to me, making code easy to understand, test, fuzz, and verify is more important. sometimes that requires more lines of code.
LoC *IS* the metric. It is obviously not perfect tho but still the best one to compare software surface. Including to compare the same solution in different languages and platforms. The less code you need to manage, the less code you need to change and the less code you need to hold in your head to understand it (or in the LLM context nowadays).
https://www.folklore.org/Negative_2000_Lines_Of_Code.html
> able to take over anyone's account simply by sending them a link
dont click links people
that'll be $500k for the security training pls
If they or their company use email "security" software that "scans" all links it's not necessarily the users fault / they don't in fact click. The mail is auto-confirmed the second it hits the inbox.
We've had to change our registration confirmation mails too recently to not confirm on GET only, but with another form - in hindsight it was stupid before but many services do that.
> We've had to change our registration confirmation mails too recently to not confirm on GET only, but with another form - in hindsight it was stupid before but many services do that.
This has been a thing since forever lol
Security software fetching links will not trigger this behavior. It needs to be done in the context of the actual ChatGPT user so that a JavaScript payload can run.
If merely GET on the link was sufficient to trigger the behaviors, the adversary generating the link would just GET it themselves.
Yes, if it requires the user session (xss related), like the article, there is no problem.
I was replying more generally about the "don't click links" advice and cases of server generated confirmation-token links (not saying this was the case at OpenAI)
... says IT and then it sets Outlook to mangle link to abominations like [0]. "Check if the preview shows something different or suspicious."
[0]: https://che01.safelinks.protection.outlook.com/?url=https%3A...
I've worked at a company that did link mangling on everything except simulated phishing emails. The effect was that people were trained to trust the mangled links.
God I hate those
I wanted to check your website from your profile, but then I remembered your advice. Thanks!, dodged a potential bullet
Also many emails from tech companies today:
- "sign in via one-time code."
- "Click here to see our latest update".
dont click links
250k refresher fee
Click this link to be told not to click links.
hey, don't dump the market
"Scan this QR code!"
Thanks! Here, click on this link to get paid
Oof. Together with the fact that it's possible to remotely access the file system (!) of computers ever having started a "Work" chat in the ChatGPT desktop app [1], this seems pretty bad.
[1] https://news.ycombinator.com/item?id=50078284
Claude Code terminal has "remote control" option activated via /rc command. -- somewhat similar in that it still runs the session on the PC/Mac but allows the Claude authenticated user to access the session via internet / app
I am guessing this ChatGPT functionality is a similar Codex feature?
But this session is disconnected when the conversation is /exit ed
Does no one write their own blogs anymore?
For my personal blog [0], I am still writing without LLM assistance. The goal is to test how well I understand the topic I am writing about, as well as to improve my own writing.
[0] https://trung.to
I actually still write my own blog [1] :sweat_smile:
Pure html by hand, using vim. No frameworks, no nothing, single stylesheet. I think there are a few others around here that still do this.
[1] https://armincat.com/
[2] https://bookofjoe2.blogspot.com/
This one is particularly bad, like they probably made $7500 (not much tbh) for p1, atleast spend a few hours to write the blog post.
For who?
For the LLMs so they get new training data
5k makes me think the next guy is gonna sell his exploit in the black market instead.
Appreciate the incremental write up building up the scope of the poc.
I like the chain of regular old school things used to achieve it like post message, JSONP, iframe, Img tags.
5k is nothing for the amount of headache you just saved them..
I think there was another HN discussion about it re bounty value and as far as I remember the issue is not that big companies are greedy in that case, but rather that over time it’s strategically better to have lower bounties so less people are interested in it.
5 or 50k doesnt matter to them. And this bug in terms of severity is very high imho.
if only they had some sort of cybersecurity state of the art thing...
we are getting to the point of “why have passwords?” Right? At some point soon we have to move to the honor system of just please don’t login as me?
My latest project emails a magic link.
I'm trying not to have any important data to even lose, when I eventually get hacked.
We can all turn up our noses, like the other commenters in this thread, but every one of them is exposed. Negligence is a matter of degrees.
> I'm trying not to have any important data to even lose
This is the way. Respect user privacy and keep as little data as possible, and not only do you become a less attractive target, if you are targeted the outcome won’t be so bad.
> please don’t login as me
Please don't drain my $6.5 billion in Dogecoin.
Been sayin this for years. Everybody in the world world: just BE COOL. Solves like every internet problem at once.
we all joke but spiritual communities have been saying this is the way for years. Like the law of one stuff.
only $5000 for this, lmao. It should be $50k out of pure shame. If they don't make these things hurt just a little they're never going to take following best practices seriously enough. There will be 9 more anyways.
Meanwhile the actual user of the account probably has to sign in using a yubikey because they get "CyBeRsEcRuItY aLeRtS" when they try to update node in their project to fix a crash and have to switch to daybreak blue to continue (or attempt to trick the normal models into doing it and risk getting autobanned)
Claude or Gemini didn’t find the vulnerability on ChatGPT?
Sigh. With how good Openai are good in programming frontier agents, they still have weak security. Hmmn, i'm short of words.
why didnt the black box of bullshit company detect this? for a group that touts every inane 'discovery' with hyperbolic bullshit, they really fall short on delivering anything of actual value, just rehashed bullshit.
Another classic OAuth implementation failure. Mismanaging the state parameter or leaking tokens via referer headers in redirects is Security 101. Wild to see this in a production environment of this scale.
The post seems to be talking about a html injection via a postMessage call where the receiver didn’t check who sent the message, which is a well known vulnerability entry point. Did I miss where OAuth had anything to do with this??
Production scale is not production grade I guess...