In Google AI Studio, Google documentation encourages to deploy vibecoded apps with an open proxy that allow equivalent AI billing abuse - giving the impression that the API key were secure because it is behind a proxy. Vulnerable apps (all apps deployed from AI studio) are easily found by searching Google, Twitter or Hacker News. https://github.com/qudent/qudent.github.io/blob/master/_post...
> Leaked key blocking. They are defaulting to blocking API keys that are discovered as leaked and used with the Gemini API.
There are no "leaked" keys if google hasn't been calling them a secret.
They should ideally prevent all keys created before Gemini from accessing Gemini. It would be funny(though not surprising) if their leaked key "discovery" has false positives and starts blocking keys from Gemini.
Yeah its tremendously unclear how they can even recover from this. I think the most selective would be: they have to at minimum remove the Generative Language API grant from every API key that was created before it was released. But even that isn't a full fix, because there's definitely keys that were created after that API was released which accidentally got it. They might have to just blanket remove the Generative Language API grant from every API key ever issued.
This is going to break so many applications. No wonder they don't want to admit this is a problem. This is, like, whole-number percentage of Gemini traffic, level of fuck-up.
Jesus, and the keys leak cached context and Gemini uploads. This might be the worst security vulnerability Google has ever pushed to prod.
The Gemini API is not enabled by default, it has to be explicitly enabled for each project.
The problem here is that people create an API key for use X, then enable Gemini on the same project to do something else, not realizing that the old key now allows access to Gemini as well.
Takeaway: GCP projects are free and provide strong security boundaries, so use them liberally and never reuse them for anything public-facing.
Imagine enabling Maps, deploying it on your website, and then enabling Google Drive API and that key immediately providing the ability to store or read files. It didn't work like that for any other service, why should it work that way for Gemini.
Also, for APIs with quotas you have to be careful not to use multiple GCP projects for a single logical application, since those quotas are tracked per application, not per account. It is definitely not Google's intent that you should have one GCP project per service within a single logical application.
Really? I make multiple GCP projects per app. One project for the (eg) Maps API, one for Drive, one for Mail, one for $THING. Internal corp-services might have one project with a few APIs enabled - but for the client-app that we sell, there are many projects with one or two APIs enabled only.
I’m usually client side dev, and am an ex googler and very curious how this happened.
I can somewhat follow this line of thinking, it’s pretty intentional and clear what you’re doing when you flip on APIs in the Google cloud site.
But I can’t wrap my mind around what is an API key. All the Google cloud stuff I’ve done the last couple years involves a lot of security stuff and permissions (namely, using Gemini, of all things. The irony…).
Somewhat infamously, there’s a separate Gemini API specifically to get the easy API key based experience. I don’t understand how the concept of an easy API key leaked into Google Cloud, especially if it is coupled to Gemini access. Why not use that to make the easy dev experience? This must be some sort of overlooked fuckup. You’d either ship this and API keys for Gemini, or neither. Doing it and not using it for an easier dev experience is a head scratcher.
This seems so… obvious? How can a company of this size, with its talent and expertise, not have standardized tests or specs preventing such a blatant flaw?
First of all, Google is a shell of the company it used to be.
That said, I’d actually argue there’s an evolutionary explanation behind this where at a certain size, and more importantly complexity, an oversight like this becomes even more likely, not less.
Seems like they ought to be dedicated security teams monitoring for exactly this: does a key to X give users access to not-X. Even more bizarre is their VDP team not immediately understanding the severity of the issue.
Yeah, lets just start building a house and don't wait for architects to finish the blueprints :) They just slowing us down with all that thinking things through stuff.
I feel it in a smaller but forced growing organization as the combination of atomised responsibilities and confused/overloaded coordination. For - a certian kind of - efficiency people are isolated into their responsibility area that they are able to oversee/comprehend - with accountability - that a manegement layer is supposed to coordinate. If the mangemenet layer is now overloaded or poorly executed - confused in case of evolution and growth and any kind of restructuring - but the atomic responsibility areas are having basically no (other than anecdotic employee chatter) oversight then troubles, even obvious ones, go undetected.
Ohh so that's how that happened. I had noticed (purely for research purposes of course) that some of Google's own keys hardcoded into older Android images were useable for Gemini (some instantly ratelimited so presumably used by many other people already but some still usable) until they all got disabled as leaked like two months ago. They also had over time disabled Gemini API access on some of them over them beforehand.
What's frustrating is that a lot of these keys were generated a long time ago with a small amount of GCP services that they could connect to. (Ex. Firebase remote config, firestore, etc.)
When Gemini came around, rather than that service being disabled by default for those keys, Gemini was enabled, allowing exploiters to easily utilize these keys (Ex. a "public" key stored in an APK file)
Gemini API is not enabled by default, a project owner has to go explicitly enable it.
The problem described here is that developer X creates an API key intended for Maps or something, developer Y turns on Gemini, and now X's key can access Gemini without either X or Y realizing that this is the case.
The solution is to not reuse GCP projects for multiple purposes, especially in prod.
Please see my response to your pasted comment in another thread: for many APIs that you can enable on a GCP project, you are intended to use the same GCP project across the whole application for quota tracking. Google even makes you assert that you are only using one GCP project (or at least list out all GCP projects, which APIs are enabled on them and what their purpose is and why you have more than one) when seeking approval for public facing OAuth.
You are wrong that increasing projects have no cost; many services have project based costs (Cloud Armour rules cannot be used cross project at the base tier), many services (mostly observeability) degrade significantly cross project, the Google Cloud Console _sucks_ cross project.
You are also wrong in saying there are no projects that could reasonably have a safe api key made unsafe by this exploit.
One example, a service that has firebase auth must publish the key (Google's docs recommend). Later, you add gen ai to that service, managing access using IAM/service accounts (the proper way). You've now elevated the Firebase Auth Key to be a Gemini key. Really undeniably poor from Google.
> Last month, a developer on your team enabled the Gemini API for an internal prototype.
> The result: thousands of API keys that were deployed as benign billing tokens are now live Gemini credentials sitting on the public internet.
Benign, deployed openly without any access restrictions whatsoever, billing tokens can be used to bill for a service under the account it is enabled for. That's the intended behavior, literally. Maps API keys are used to give your users access to Google Maps on your credit card.
What's the problem here? Yes, the defaults could have been stricter, but it's not like it costs anything to create a bunch of internal projects that do not have good-for-billing access keys floating around open internet. People moved fast, deployed LLM generated code, broke things and then blame everyone else but themselves?
So even if they fix the issue, it sounds as though you can still shoot itself in the foot by essentially being at to arbitrarily change an existing key from “not a secret” to “is a secret”?
Even if you have a key that you use for maps (not secret) someone could add the generative AI scope to it and make it now necessarily secret (even though it’s probably already publicly available)?
This totally reminds me of SSN use, when initially they were just a number (not secret) to identify a person, and then suddenly people started to use them as a key for authorisation, because someone had a bright idea how to implement things fast/simple/cheap (cheap part comes at expense of others)
Is the implication at the end that Google has not actually fixed this issue yet? This is really bad; a massive oversight, very clearly caused by a rush to get Gemini in customers' hands, and the remediation is in all likelihood going to nuke customer workflows by forcing them to disable keys. Extremely bad look for Google.
> Retroactive Privilege Expansion. You created a Maps key three years ago and embedded it in your website's source code, exactly as Google instructed. Last month, a developer on your team enabled the Gemini API for an internal prototype. Your public Maps key is now a Gemini credential. Anyone who scrapes it can access your uploaded files, cached content, and rack up your AI bill. Nobody told you.
Malpractice/I can't believe they're just rolling forward
They should limit the new features to new API keys that explicitly opt-in instead of fucking over every user who trusted their previous documentation that these keys are public information.
Is there a way to use Google maps apis on the web without exposing the key?
Re host header seems an odd way for Google to do it, surely they would have fixed that by now? I guess not a huge problem as attackers would have to proxy traffic or something to obscure the host headers sent by real clients? Any links on how people exploit this?
It would be helpful if you answer the question about web api usage, most of that is not relevant.
The only suggestion I see there from a quick skim that would avoid the above is for customers to set up a google maps proxy server for every usage with adds security and hides the key. That is completely impractical suggestion for the majority of users of embedded google maps.
Woof. Impedance mismatch outcome from moving fast - the GCP auth model was never designed to work like oAI's API key model; this isn't the only pain point this year, but it's a nasty one. I'm sympathetic, except that dealing with GCP has always been a huge pain in the ass. So I'm a little less sympathetic.
2k feels very small considering the number of business sites that embed Google Maps. I guess a lot of those sites use other website building services that handle the Google API keys for them, and/or they're old and untouched enough that no one enabled Gemini on them.
I had the same thought. I guess a lot of those keys may belong to dormant/deleted accounts and only a % of people who have enabled Gemini (presumably it required user action)
Happened to me recently, I got a warning in Gemini Studio that a key leaked. I was perplexed initially and then realized what had happened. The proper fix is to limit the key to just Maps APIs. Of course even this is not so easy, as there's a long list of APIs with complicated names. It was at least limited to my domain.
They may have used ChatGPT or similar to help with the prose but the technical content (as discussed elsewhere on this page) is good, so does it really matter if they did?
The problem with AI slop (to me) is more that the technical content is not good or is entirely the product of the LLM. At that point, there's no point in me reading it, I can just prompt the question if I'm interested.
This is original research which wasn't public before, so the value is still there and I didn't think whichever combination of a human and LLM that generated it did a bad job.
When you enable the Gemini API (Generative Language API) on a Google Cloud project, existing API keys in that project (including the ones sitting in public JavaScript on your website) can silently gain access to sensitive Gemini endpoints. No warning. No confirmation dialog. No email notification.
Specifically, the last bit - “No warning. No confirmation dialog. No email notification.” Immediately smells like LLM generated text to me. Punchy repetition in a set of 3.
If you scroll through tiktok or instagram you can see the same exact pattern in a lot of LLM generated descriptions.
I think there's a lot more than just that, but I think part of the problem is that you just get an uncanny valley feeling. All of the phrases and rhetorical tricks that these tools use are perfectly valid, but together they feel somehow thin?
That said, some specific things that feel very AI-y are the mostly short, equally-sized paragraphs with occasional punchy one-sentence paragraphs interspersed between them; the use of bold when listing things (and the number of two-element lists); there are a couple of "it's not X, it's Y"-style statements; one paragraph ends with an "they say it's X, but it's actually Y" construct; and even the phrasing of some of the headings.
None of these are necessarily individually tells of AI writing (and I suspect if you look through my own comments and blog posts on various sites, you'd find me using many of the same constructs, because they're all either effective rhetorically, or make the text clearer and easier to understand). But there's something about the concentration of them here that feels like AI - the uncanny valley feeling.
I would put money on this post at least having gone through AI review, if not having been generated by AI from human-written notes. I understand why people do that, but I also think it's a shame that some of the individual colour of people's writing is disappearing from these sorts of blog posts.
Using threes is common in English writing and speaking. It has an optimal balance of expressiveness (three marking a pattern or breadth; creating momentum) without being overwhelming.
It’s not uncommon, as basic writing advice, to use sets of three for emphasis. That isn’t a signifier of LLM generation, in my opinion.
This excerpt is demonstrating the use of a literary technique to write non-literary prose. It's an almost sure sign that an LLM is generating the text.
Of course, how could a writer writing have writing chops and use writing techniques? It boggles the mind that anyone thinks that would ever happens. Must have been aliens.
It's also seemingly the only way ChatGPT knows how to write, while being very uncommon for blogposts beforehand. Of course it's not 100% proof, but it's the most likely explanation.
“The rule of three is a writing principle which suggests that a trio of entities such as events or characters is more satisfying, effective, or humorous than other numbers, hence also more memorable, because it combines both brevity and rhythm with the smallest amount of information needed to create a pattern.”
It’s how I was taught to write, but I understand that my personal experience can’t be generalized to make sweeping statements.
Do you have data that suggests it’s uncommon in human-authored blog posts and more common in LLM-generated text?
Speaking of LLM-writing in general, it seems to greatly overuse certain types of constructions or use them in uncommon contexts. So that probably isn't so much using the rule of threes, but overusing the rule of threes in certain specific ways in certain specific contexts.
I don’t necessarily doubt you or the grand-parent comment, but if it’s ‘obvious to even the most casual of observers’ (as my father would say) then it should be easy to have hard data.
I’m not a native speaker so my level of AI recognition is already low. I find it very interesting what patters people bring up to declare it’s AI. The 3 punchline one for instance is a pattern I use while speaking. Can’t say I would write like this though.
It's not so much the grouping of 3 or way it's supposed to be punchy specifically that's the problem, that is just one example of what gives the article the "LLM Generated" feeling since whatever cheap model people are using for this kind of spam has some common ticks.
I use groupings of 3 and try to make things punchy myself sometimes, especially when I'm writing something intended to sway others. I think the problem with this article is the way it feels like the perfect average of corporate writing. It's sort of like the "written by committee" feel that incredibly generic pop music often has.
When I write things, I often go back and edit and reword parts. Like the brushstrokes in an oil painting, the flow of thought varies between paragraphs and even sentences. LLMs only generate things from left to right (or vice versa in RTL languages, I presume). I think that gives LLM generated text a "smooth" texture that really stands out to anyone who reads a lot.
I completely agree with you. There's something conspicuous about this particular use of the "group of three" device. It's trying but it's goofy and conspicuous. I think it's not human, it's 52 trillion parameters in a trenchcoat.
I'm not a native speaker and my level of AI recognition is higher than 99.999% of native speakers - and I'd be happy to be tested on it for proof.
The biggest factor is simply how long you've been using LLMs to generate text, how often, how much. It's like how an experienced UI designer can instantly tell that something is off by a single pixel off upon first seeing a UI, whereas if you gave me $200 to find it within 10 minutes I might well fail.
Aside from particulars like the set of 3, LLMs add a lot of emotive language which doesn't mean anything or is a repetition of already established points. Since they can't add any actual substance beyond what was in the prompt, the only thing they do is pad the prompt with filler language.
OK I've seen many people make this point on this site over just the last few months, but where do you think LLMs pick up these patterns? How did this rule of threes https://en.wikipedia.org/wiki/Rule_of_three_(writing) get into the LLM so they are so damn recognizable as LLMs and not as humans?
HN Note: Yes the rule of threes is broader than just this particular pattern here, but in my opinion this common writing and communication pattern is a specific example of the rule of threes.
Punchy repetition in a set of 3. Yes. LLMs are able to capably mimic the common patterns that how to write books have suggested for the last 100 years as ways to make your writing more "impactful" and attention-grabbing. So are humans. They learned it from watching us.
I am a little bit worked up on this as I have felt insulted a couple times at having something I've written been accused of being by an LLM, in that case it was because I had written something from the viewpoint of a depressed and tired character and someone thought it had to be an LLM because they seemed detached from humanity! Success!
I too would like to be able to reliably detect when something has been written by an LLM so I can discount it out of hand, but frankly many of the attempts I see people make to detect these things seem poorly reasoned and actively detrimental.
People have learned in classes and from reading how to improve their writing. LLMs have learned from ingesting our output. If something matches a common writing 101 tip it is just as likely to be reasonably competent as it is to be non-human. The solution to escape being labelled an LLM is not to become less competent as a writer.
I have been overly verbose here, as I am somewhat worked up and angry and it is too late in the morning to go back to sleep but really too early to be awake. I know verbosity is also a symptom of being an LLM, but not giving a damn is a symptom of humanity.
>but where do you think LLMs pick up these patterns?
>LLMs are able to capably mimic the common patterns that how to write books have suggested for the last 100 years as ways to make your writing more "impactful" and attention-grabbing. So are humans. They learned it from watching us.
Don't forget that LLMs (at least the "instruct" versions) undergo substantial post-training to align them with the authors' objectives, so they are not a 100% pure reflection of the distribution seen on the internet. For example, it's common for LLMs to respond with "You're absolutely right!" to every second message, which isn't what humans usually do. It's a result of some kind of RLHF: human labelers liked to hear that they're right, so they preferred answers containing such phrases, and those responses became amplified. People recognize LLM-generated writing because LLMs' pattern distribution is different from the actual pattern distribution found in articles written by humans.
It's too well structured and the message is too clear. HN (and the whole internet) is allergic to proper writing. We praise human sloppiness now.
No, I'm not being sarcastic. People have given up em-dash, which is an official punctuation you use in proper writing. And it's all a downhill from there.
> It's too well structured and the message is too clean. HN (and the whole internet) is allergic to proper writing. We praise human sloppiness now.
Yes. And it's only a matter of time that the model companies start to try to train in that "human sloppiness." After all, a lot of their customers want machines that can pass for humans.
> No, I'm not being sarcastic. People have given up em-dash, which is an official punctuation you use in proper writing. And it's all a downhill from there.
I wouldn't be surprised if the internet language of people devolves into a weird constantly-changing mish-mash of slang and linguistic fads. Basically an arms race where people constantly innovate in order to stay distinct from the latest models.
But the end result of that would be probably fragmentation, isolation, and a kind of dark ages. Different communities would have different slang, and that slang would change so fast that old text would quickly become hard to understand.
> TruffleHog will verify whether discovered keys are live and have Gemini access, so you'll know exactly which keys are exposed and active, not just which ones match a regular expression.
I don't know exactly, but I'm sure. The cadence, the clarity, the bolding, the italics, it's all just crisp and clean structured and actionable in a way that a meandering human would not distill it down to.
Yup, it was actually an interesting article but there are a few telltale parts that sound like every AI spam post on /r/webdev and similar. "No warning. No confirmation dialog. No email notification." is another. The three negatives repeated is present in so many AI generated promotional posts.
I don't even have a problem with the content itself, I think frankly the smell is that it's too good. It's just fascinating in the sense that it's one LLM attacking another LLM.
It's too structured and consistent. Imo. Has that AI smell to it, but I guess humans will eventually also start writing more like the AIs they learn from.
> guess humans will eventually also start writing more like the AIs they learn from.
With the AI feedback loop being so fast and tight for some tasks, the focus moves on to delivery than learning. There is no incentive, space or time for learning.
For me personally, both at work and in my free time, I spend _more_ time on writing things _that matter_ since I’ve freed up time by using LLM’s for boilerplate tasks.
My motto is - If it wasn’t worth writing, it won’t be worth reading.
A good example of writing where I’d recommend using LLM’s is product documentation. You pass the diff, the description of the task, and the context (existing documentation) with a prompt ”Update the documentation…”.
Documentation is important but it’s not prose. However, writing a comment on hacker news is.
The fact that according to this reply section most of HN can't tell means that predictably, all hope is lost and there's no point in writing anything by hand any more if you're in it for money/engagement.
While writing this I suddenly realized that marketers and writers probably do a better job at recognizing it than developers and engineers, so maybe all hope isn't.
For those who want to know the tells: overall cadence and frequency of patterns - especially infrequency of patterns - are the biggest ones. And that means that we can't actually give you the best tells, because they're more about what is absent than what is present. What's absent is a single sentence pattern that falls completely out of the LLM go-toes. Anything human written has at most a good mix of both. LLM-written text just entirely lacks it. Humans do use the LLM-preferred patterns, but not for every single sentence. But anyway, here we go.
> Transparently, the initial triage was frustrating; the report was dismissed as "Intended Behavior”. But after providing concrete evidence from Google's own infrastructure, the GCP VDP team took the issue seriously.
^ Fun fact - The ";" would've originally been an em-dash but was either rewritten or a rule was included for this.
> Then Gemini arrived.
^ Dramatic short sentences, a pattern with magnitudes higher LLM-frequency than human frequency, but hasn't reached the public conscious yet a la "not just X but Y".
> No warning. No confirmation dialog. No email notification.
^ Another such pattern. Not just because it's three of them, but also because of the content and repetition. Humans rarely write like that because it again sounds overly dramatic. It's something you see in fiction rather than a technical writeup. In a thriller.
> Retroactive Privilege Expansion. You created a Maps key three years ago and embedded it in your website's source code, exactly as Google instructed. Last month, a developer on your team enabled the Gemini API for an internal prototype. Your public Maps key is now a Gemini credential. Anyone who scrapes it can access your uploaded files, cached content, and rack up your AI bill. Nobody told you.
This style of scenario writing is another one.
> Nobody told you.
Absolute drama queen.
>The UI shows a warning about "unauthorized use," but the architectural default is wide open.
Again.
> The attacker never touches your infrastructure. They just scrape a key from a public webpage.
Again.
> These aren't just hobbyist side projects. The victims included major financial institutions, security companies, global recruiting firms, and, notably, Google itself.
..
> A key that was deployed years ago for a completely benign purpose had silently gained full access to a sensitive API without any developer intervention.
Surprised it hasn't gained consciousness by now. Maybe that's a future plot point.
Here's a great example to train your skills on, because it's rare in that the ratio of "human : straight from LLM" increased gradually as the article goes on: https://www.wallstreetraider.com/story.html
It started at heavy human editing (or just human-written), but less and less towards the end.
The author confirmed this upon pointing it out, FWIW [0].
This is true but also not as new as the author claims. There have been various ways to abuse Google API keys in the past (at least to abuse them financially) and it’s always been very confusing for developers.
In Google AI Studio, Google documentation encourages to deploy vibecoded apps with an open proxy that allow equivalent AI billing abuse - giving the impression that the API key were secure because it is behind a proxy. Vulnerable apps (all apps deployed from AI studio) are easily found by searching Google, Twitter or Hacker News. https://github.com/qudent/qudent.github.io/blob/master/_post...
> Leaked key blocking. They are defaulting to blocking API keys that are discovered as leaked and used with the Gemini API.
There are no "leaked" keys if google hasn't been calling them a secret.
They should ideally prevent all keys created before Gemini from accessing Gemini. It would be funny(though not surprising) if their leaked key "discovery" has false positives and starts blocking keys from Gemini.
Yeah its tremendously unclear how they can even recover from this. I think the most selective would be: they have to at minimum remove the Generative Language API grant from every API key that was created before it was released. But even that isn't a full fix, because there's definitely keys that were created after that API was released which accidentally got it. They might have to just blanket remove the Generative Language API grant from every API key ever issued.
This is going to break so many applications. No wonder they don't want to admit this is a problem. This is, like, whole-number percentage of Gemini traffic, level of fuck-up.
Jesus, and the keys leak cached context and Gemini uploads. This might be the worst security vulnerability Google has ever pushed to prod.
The Gemini API is not enabled by default, it has to be explicitly enabled for each project.
The problem here is that people create an API key for use X, then enable Gemini on the same project to do something else, not realizing that the old key now allows access to Gemini as well.
Takeaway: GCP projects are free and provide strong security boundaries, so use them liberally and never reuse them for anything public-facing.
Imagine enabling Maps, deploying it on your website, and then enabling Google Drive API and that key immediately providing the ability to store or read files. It didn't work like that for any other service, why should it work that way for Gemini.
Also, for APIs with quotas you have to be careful not to use multiple GCP projects for a single logical application, since those quotas are tracked per application, not per account. It is definitely not Google's intent that you should have one GCP project per service within a single logical application.
Really? I make multiple GCP projects per app. One project for the (eg) Maps API, one for Drive, one for Mail, one for $THING. Internal corp-services might have one project with a few APIs enabled - but for the client-app that we sell, there are many projects with one or two APIs enabled only.
Isn't there a limit to the number of projects you can make and then you have to ask support to increase it?
I’m usually client side dev, and am an ex googler and very curious how this happened.
I can somewhat follow this line of thinking, it’s pretty intentional and clear what you’re doing when you flip on APIs in the Google cloud site.
But I can’t wrap my mind around what is an API key. All the Google cloud stuff I’ve done the last couple years involves a lot of security stuff and permissions (namely, using Gemini, of all things. The irony…).
Somewhat infamously, there’s a separate Gemini API specifically to get the easy API key based experience. I don’t understand how the concept of an easy API key leaked into Google Cloud, especially if it is coupled to Gemini access. Why not use that to make the easy dev experience? This must be some sort of overlooked fuckup. You’d either ship this and API keys for Gemini, or neither. Doing it and not using it for an easier dev experience is a head scratcher.
I started replying with a clever approach to layer scopes onto keys… but nope. Doesn’t work.
How did this get past any kind of security review at all? It’s like using usernames as passwords.
I hope Google has a database with the creation timestamp for every API key they issued.
This seems so… obvious? How can a company of this size, with its talent and expertise, not have standardized tests or specs preventing such a blatant flaw?
First of all, Google is a shell of the company it used to be.
That said, I’d actually argue there’s an evolutionary explanation behind this where at a certain size, and more importantly complexity, an oversight like this becomes even more likely, not less.
Seems like they ought to be dedicated security teams monitoring for exactly this: does a key to X give users access to not-X. Even more bizarre is their VDP team not immediately understanding the severity of the issue.
And slow down the time to ship things? The shareholders wouldn't like that.
That's how you slow down development to a crawl
Yeah, lets just start building a house and don't wait for architects to finish the blueprints :) They just slowing us down with all that thinking things through stuff.
I feel it in a smaller but forced growing organization as the combination of atomised responsibilities and confused/overloaded coordination. For - a certian kind of - efficiency people are isolated into their responsibility area that they are able to oversee/comprehend - with accountability - that a manegement layer is supposed to coordinate. If the mangemenet layer is now overloaded or poorly executed - confused in case of evolution and growth and any kind of restructuring - but the atomic responsibility areas are having basically no (other than anecdotic employee chatter) oversight then troubles, even obvious ones, go undetected.
I don’t see it.
Imagine for a moment the there is no oversight. Every intern can ship prod code with their own homemade crypto.
How do you, in a retail business, agree to accept credentials that anyone can mint for free?
I mean obviously it happened. But… this doesn’t even seem like a compliance mistake. It’s a business-level mistake.
If you've never worked in a large corporate environment you don't know how stupid things become. In a perfect bureaucracy nobody thinks.
Stuff like this was proposed to be added to standard interviews, but they were too busy reversing binary trees
Security. The final frontier. Where no developer has ever bothered before.
Their “talent and expertise” is mostly in selling ads.
in a company of this size ... left hand don't know what right hand is doing
They probably used the in house AI tools to build this.
"This seems fine"
Ohh so that's how that happened. I had noticed (purely for research purposes of course) that some of Google's own keys hardcoded into older Android images were useable for Gemini (some instantly ratelimited so presumably used by many other people already but some still usable) until they all got disabled as leaked like two months ago. They also had over time disabled Gemini API access on some of them over them beforehand.
What's frustrating is that a lot of these keys were generated a long time ago with a small amount of GCP services that they could connect to. (Ex. Firebase remote config, firestore, etc.)
When Gemini came around, rather than that service being disabled by default for those keys, Gemini was enabled, allowing exploiters to easily utilize these keys (Ex. a "public" key stored in an APK file)
Gemini API is not enabled by default, a project owner has to go explicitly enable it.
The problem described here is that developer X creates an API key intended for Maps or something, developer Y turns on Gemini, and now X's key can access Gemini without either X or Y realizing that this is the case.
The solution is to not reuse GCP projects for multiple purposes, especially in prod.
Please see my response to your pasted comment in another thread: for many APIs that you can enable on a GCP project, you are intended to use the same GCP project across the whole application for quota tracking. Google even makes you assert that you are only using one GCP project (or at least list out all GCP projects, which APIs are enabled on them and what their purpose is and why you have more than one) when seeking approval for public facing OAuth.
You are wrong that increasing projects have no cost; many services have project based costs (Cloud Armour rules cannot be used cross project at the base tier), many services (mostly observeability) degrade significantly cross project, the Google Cloud Console _sucks_ cross project.
You are also wrong in saying there are no projects that could reasonably have a safe api key made unsafe by this exploit.
One example, a service that has firebase auth must publish the key (Google's docs recommend). Later, you add gen ai to that service, managing access using IAM/service accounts (the proper way). You've now elevated the Firebase Auth Key to be a Gemini key. Really undeniably poor from Google.
The problem is Google explicitly stating that those API keys are not secret and should be public, which indeed was true until Gemini came around.
Or usecase: developer X stopped using Maps/etc N years ago, and is long gone, and then developer Y stumbles into the company's google api console.
Of course, Google is full of smart anti-fraud experts, they just handle 80% of this shit on the back-end, so they don't care about the front-end pain.
Explain It Like I'm Five.
From TFA:
> Last month, a developer on your team enabled the Gemini API for an internal prototype. > The result: thousands of API keys that were deployed as benign billing tokens are now live Gemini credentials sitting on the public internet.
Benign, deployed openly without any access restrictions whatsoever, billing tokens can be used to bill for a service under the account it is enabled for. That's the intended behavior, literally. Maps API keys are used to give your users access to Google Maps on your credit card.
What's the problem here? Yes, the defaults could have been stricter, but it's not like it costs anything to create a bunch of internal projects that do not have good-for-billing access keys floating around open internet. People moved fast, deployed LLM generated code, broke things and then blame everyone else but themselves?
The key didn’t change. What changed is what it can do.
What used to be a simple project identifier can now trigger expensive API calls once Gemini is enabled.
Since nothing in the old code changes, there’s no obvious moment where someone stops and re-evaluates the risk.
So even if they fix the issue, it sounds as though you can still shoot itself in the foot by essentially being at to arbitrarily change an existing key from “not a secret” to “is a secret”?
Even if you have a key that you use for maps (not secret) someone could add the generative AI scope to it and make it now necessarily secret (even though it’s probably already publicly available)?
This totally reminds me of SSN use, when initially they were just a number (not secret) to identify a person, and then suddenly people started to use them as a key for authorisation, because someone had a bright idea how to implement things fast/simple/cheap (cheap part comes at expense of others)
Is the implication at the end that Google has not actually fixed this issue yet? This is really bad; a massive oversight, very clearly caused by a rush to get Gemini in customers' hands, and the remediation is in all likelihood going to nuke customer workflows by forcing them to disable keys. Extremely bad look for Google.
> Retroactive Privilege Expansion. You created a Maps key three years ago and embedded it in your website's source code, exactly as Google instructed. Last month, a developer on your team enabled the Gemini API for an internal prototype. Your public Maps key is now a Gemini credential. Anyone who scrapes it can access your uploaded files, cached content, and rack up your AI bill. Nobody told you.
Malpractice/I can't believe they're just rolling forward
They should limit the new features to new API keys that explicitly opt-in instead of fucking over every user who trusted their previous documentation that these keys are public information.
Maps keys should not be made public otherwise an attacker can steal them and drain your wallet and use it for their own sites.
Maps keys are always public in js on the website (but locked to use on certain domains). That’s how they work.
It is not actually locked to a site is just based off the host header. Which is public information an attacker can use to make the requests.
Is there a way to use Google maps apis on the web without exposing the key?
Re host header seems an odd way for Google to do it, surely they would have fixed that by now? I guess not a huge problem as attackers would have to proxy traffic or something to obscure the host headers sent by real clients? Any links on how people exploit this?
Google's own recommended practices https://developers.google.com/maps/api-security-best-practic...
It would be helpful if you answer the question about web api usage, most of that is not relevant.
The only suggestion I see there from a quick skim that would avoid the above is for customers to set up a google maps proxy server for every usage with adds security and hides the key. That is completely impractical suggestion for the majority of users of embedded google maps.
It’s been years but I thought I recalled having to use the key but then also setting what sites it’d work on.
If an attacker can figure out what sites it can be used on, they can use the API.
This firm is doing great work, I still refer to this post ("Anyone can Access Deleted and Private Repository Data on GitHub"): https://trufflesecurity.com/blog/anyone-can-access-deleted-a...
Can’t wait til someone makes a Gemini prompt to find these public keys and launch a copy of itself using them.
Woof. Impedance mismatch outcome from moving fast - the GCP auth model was never designed to work like oAI's API key model; this isn't the only pain point this year, but it's a nasty one. I'm sympathetic, except that dealing with GCP has always been a huge pain in the ass. So I'm a little less sympathetic.
Separating & importing keys for gpg in --quick-generate-keys prior to git clone in SSH.
Cargo fuzzing the API over local address server yielding r-w-x permissions.
Seems like the kind of bug caused by using Gemini to vibe code the GCP.
You must be right. Do you have inside info?
> 2,863 Live Keys on the Public Internet
It will be more interesting if they scan GitHub code instead. The number terrified me. Though I am not sure how many of that are live.
2k feels very small considering the number of business sites that embed Google Maps. I guess a lot of those sites use other website building services that handle the Google API keys for them, and/or they're old and untouched enough that no one enabled Gemini on them.
I had the same thought. I guess a lot of those keys may belong to dormant/deleted accounts and only a % of people who have enabled Gemini (presumably it required user action)
Happened to me recently, I got a warning in Gemini Studio that a key leaked. I was perplexed initially and then realized what had happened. The proper fix is to limit the key to just Maps APIs. Of course even this is not so easy, as there's a long list of APIs with complicated names. It was at least limited to my domain.
Great write-up. Hilarious situation where no one (except unwieldiness) is the villain.
Private data should not be allowed to be accessed using public keys. That is the core problem. It is not about Google API keys are secret or not.
It was intended for situations where the keyholder is a middleman between Google's API and the end user.
Dang, another obvious reason (among many others) you shouldn't be uploading documents to any LLM client (or use them on anything important).
ChatGPT writing a blog post attacking Gemini security flaws. It's their world now, we're just watching how it plays out.
They may have used ChatGPT or similar to help with the prose but the technical content (as discussed elsewhere on this page) is good, so does it really matter if they did?
The problem with AI slop (to me) is more that the technical content is not good or is entirely the product of the LLM. At that point, there's no point in me reading it, I can just prompt the question if I'm interested.
This is original research which wasn't public before, so the value is still there and I didn't think whichever combination of a human and LLM that generated it did a bad job.
How do you know that this blog post was written by ChatGPT?
It feels generated to me too. It’s this:
Specifically, the last bit - “No warning. No confirmation dialog. No email notification.” Immediately smells like LLM generated text to me. Punchy repetition in a set of 3.If you scroll through tiktok or instagram you can see the same exact pattern in a lot of LLM generated descriptions.
I think there's a lot more than just that, but I think part of the problem is that you just get an uncanny valley feeling. All of the phrases and rhetorical tricks that these tools use are perfectly valid, but together they feel somehow thin?
That said, some specific things that feel very AI-y are the mostly short, equally-sized paragraphs with occasional punchy one-sentence paragraphs interspersed between them; the use of bold when listing things (and the number of two-element lists); there are a couple of "it's not X, it's Y"-style statements; one paragraph ends with an "they say it's X, but it's actually Y" construct; and even the phrasing of some of the headings.
None of these are necessarily individually tells of AI writing (and I suspect if you look through my own comments and blog posts on various sites, you'd find me using many of the same constructs, because they're all either effective rhetorically, or make the text clearer and easier to understand). But there's something about the concentration of them here that feels like AI - the uncanny valley feeling.
I would put money on this post at least having gone through AI review, if not having been generated by AI from human-written notes. I understand why people do that, but I also think it's a shame that some of the individual colour of people's writing is disappearing from these sorts of blog posts.
Using threes is common in English writing and speaking. It has an optimal balance of expressiveness (three marking a pattern or breadth; creating momentum) without being overwhelming.
It’s not uncommon, as basic writing advice, to use sets of three for emphasis. That isn’t a signifier of LLM generation, in my opinion.
This excerpt is demonstrating the use of a literary technique to write non-literary prose. It's an almost sure sign that an LLM is generating the text.
Of course, how could a writer writing have writing chops and use writing techniques? It boggles the mind that anyone thinks that would ever happens. Must have been aliens.
A good writer knows when to use literary techniques.
It's also seemingly the only way ChatGPT knows how to write, while being very uncommon for blogposts beforehand. Of course it's not 100% proof, but it's the most likely explanation.
It has a name. The Rule of Threes. https://en.wikipedia.org/wiki/Rule_of_three_(writing)
“The rule of three is a writing principle which suggests that a trio of entities such as events or characters is more satisfying, effective, or humorous than other numbers, hence also more memorable, because it combines both brevity and rhythm with the smallest amount of information needed to create a pattern.”
It’s how I was taught to write, but I understand that my personal experience can’t be generalized to make sweeping statements.
Do you have data that suggests it’s uncommon in human-authored blog posts and more common in LLM-generated text?
> It has a name. The Rule of Threes. https://en.wikipedia.org/wiki/Rule_of_three_(writing)
I don't think that's exactly it.
Speaking of LLM-writing in general, it seems to greatly overuse certain types of constructions or use them in uncommon contexts. So that probably isn't so much using the rule of threes, but overusing the rule of threes in certain specific ways in certain specific contexts.
I don’t necessarily doubt you or the grand-parent comment, but if it’s ‘obvious to even the most casual of observers’ (as my father would say) then it should be easy to have hard data.
I’m not a native speaker so my level of AI recognition is already low. I find it very interesting what patters people bring up to declare it’s AI. The 3 punchline one for instance is a pattern I use while speaking. Can’t say I would write like this though.
It's not so much the grouping of 3 or way it's supposed to be punchy specifically that's the problem, that is just one example of what gives the article the "LLM Generated" feeling since whatever cheap model people are using for this kind of spam has some common ticks.
I use groupings of 3 and try to make things punchy myself sometimes, especially when I'm writing something intended to sway others. I think the problem with this article is the way it feels like the perfect average of corporate writing. It's sort of like the "written by committee" feel that incredibly generic pop music often has.
When I write things, I often go back and edit and reword parts. Like the brushstrokes in an oil painting, the flow of thought varies between paragraphs and even sentences. LLMs only generate things from left to right (or vice versa in RTL languages, I presume). I think that gives LLM generated text a "smooth" texture that really stands out to anyone who reads a lot.
I completely agree with you. There's something conspicuous about this particular use of the "group of three" device. It's trying but it's goofy and conspicuous. I think it's not human, it's 52 trillion parameters in a trenchcoat.
I'm not a native speaker and my level of AI recognition is higher than 99.999% of native speakers - and I'd be happy to be tested on it for proof.
The biggest factor is simply how long you've been using LLMs to generate text, how often, how much. It's like how an experienced UI designer can instantly tell that something is off by a single pixel off upon first seeing a UI, whereas if you gave me $200 to find it within 10 minutes I might well fail.
Aside from particulars like the set of 3, LLMs add a lot of emotive language which doesn't mean anything or is a repetition of already established points. Since they can't add any actual substance beyond what was in the prompt, the only thing they do is pad the prompt with filler language.
OK I've seen many people make this point on this site over just the last few months, but where do you think LLMs pick up these patterns? How did this rule of threes https://en.wikipedia.org/wiki/Rule_of_three_(writing) get into the LLM so they are so damn recognizable as LLMs and not as humans?
HN Note: Yes the rule of threes is broader than just this particular pattern here, but in my opinion this common writing and communication pattern is a specific example of the rule of threes.
Punchy repetition in a set of 3. Yes. LLMs are able to capably mimic the common patterns that how to write books have suggested for the last 100 years as ways to make your writing more "impactful" and attention-grabbing. So are humans. They learned it from watching us.
I am a little bit worked up on this as I have felt insulted a couple times at having something I've written been accused of being by an LLM, in that case it was because I had written something from the viewpoint of a depressed and tired character and someone thought it had to be an LLM because they seemed detached from humanity! Success!
I too would like to be able to reliably detect when something has been written by an LLM so I can discount it out of hand, but frankly many of the attempts I see people make to detect these things seem poorly reasoned and actively detrimental.
People have learned in classes and from reading how to improve their writing. LLMs have learned from ingesting our output. If something matches a common writing 101 tip it is just as likely to be reasonably competent as it is to be non-human. The solution to escape being labelled an LLM is not to become less competent as a writer.
I have been overly verbose here, as I am somewhat worked up and angry and it is too late in the morning to go back to sleep but really too early to be awake. I know verbosity is also a symptom of being an LLM, but not giving a damn is a symptom of humanity.
>but where do you think LLMs pick up these patterns?
>LLMs are able to capably mimic the common patterns that how to write books have suggested for the last 100 years as ways to make your writing more "impactful" and attention-grabbing. So are humans. They learned it from watching us.
Don't forget that LLMs (at least the "instruct" versions) undergo substantial post-training to align them with the authors' objectives, so they are not a 100% pure reflection of the distribution seen on the internet. For example, it's common for LLMs to respond with "You're absolutely right!" to every second message, which isn't what humans usually do. It's a result of some kind of RLHF: human labelers liked to hear that they're right, so they preferred answers containing such phrases, and those responses became amplified. People recognize LLM-generated writing because LLMs' pattern distribution is different from the actual pattern distribution found in articles written by humans.
It's too well structured and the message is too clear. HN (and the whole internet) is allergic to proper writing. We praise human sloppiness now.
No, I'm not being sarcastic. People have given up em-dash, which is an official punctuation you use in proper writing. And it's all a downhill from there.
> It's too well structured and the message is too clean. HN (and the whole internet) is allergic to proper writing. We praise human sloppiness now.
Yes. And it's only a matter of time that the model companies start to try to train in that "human sloppiness." After all, a lot of their customers want machines that can pass for humans.
> No, I'm not being sarcastic. People have given up em-dash, which is an official punctuation you use in proper writing. And it's all a downhill from there.
I wouldn't be surprised if the internet language of people devolves into a weird constantly-changing mish-mash of slang and linguistic fads. Basically an arms race where people constantly innovate in order to stay distinct from the latest models.
But the end result of that would be probably fragmentation, isolation, and a kind of dark ages. Different communities would have different slang, and that slang would change so fast that old text would quickly become hard to understand.
> The Core Problem
> What You Should Do Right Now
> Bonus: Scan with TruffleHog.
> TruffleHog will verify whether discovered keys are live and have Gemini access, so you'll know exactly which keys are exposed and active, not just which ones match a regular expression.
I don't know exactly, but I'm sure. The cadence, the clarity, the bolding, the italics, it's all just crisp and clean structured and actionable in a way that a meandering human would not distill it down to.
Yup, it was actually an interesting article but there are a few telltale parts that sound like every AI spam post on /r/webdev and similar. "No warning. No confirmation dialog. No email notification." is another. The three negatives repeated is present in so many AI generated promotional posts.
I don't even have a problem with the content itself, I think frankly the smell is that it's too good. It's just fascinating in the sense that it's one LLM attacking another LLM.
It's too structured and consistent. Imo. Has that AI smell to it, but I guess humans will eventually also start writing more like the AIs they learn from.
AI was trained on human writing.
> AI was trained on human writing.
AI output is not varied like real human writing. This is a very distinctive narrowing of style.
And now humans are trained on AI writing.
Like what happens to YouTube videos that go through the compression algorithm 20 times.
> guess humans will eventually also start writing more like the AIs they learn from.
With the AI feedback loop being so fast and tight for some tasks, the focus moves on to delivery than learning. There is no incentive, space or time for learning.
For me personally, both at work and in my free time, I spend _more_ time on writing things _that matter_ since I’ve freed up time by using LLM’s for boilerplate tasks.
My motto is - If it wasn’t worth writing, it won’t be worth reading.
A good example of writing where I’d recommend using LLM’s is product documentation. You pass the diff, the description of the task, and the context (existing documentation) with a prompt ”Update the documentation…”.
Documentation is important but it’s not prose. However, writing a comment on hacker news is.
Won't be well received here, but this is the truth.
The fact that according to this reply section most of HN can't tell means that predictably, all hope is lost and there's no point in writing anything by hand any more if you're in it for money/engagement.
While writing this I suddenly realized that marketers and writers probably do a better job at recognizing it than developers and engineers, so maybe all hope isn't.
For those who want to know the tells: overall cadence and frequency of patterns - especially infrequency of patterns - are the biggest ones. And that means that we can't actually give you the best tells, because they're more about what is absent than what is present. What's absent is a single sentence pattern that falls completely out of the LLM go-toes. Anything human written has at most a good mix of both. LLM-written text just entirely lacks it. Humans do use the LLM-preferred patterns, but not for every single sentence. But anyway, here we go.
> Transparently, the initial triage was frustrating; the report was dismissed as "Intended Behavior”. But after providing concrete evidence from Google's own infrastructure, the GCP VDP team took the issue seriously.
^ Fun fact - The ";" would've originally been an em-dash but was either rewritten or a rule was included for this.
> Then Gemini arrived.
^ Dramatic short sentences, a pattern with magnitudes higher LLM-frequency than human frequency, but hasn't reached the public conscious yet a la "not just X but Y".
> No warning. No confirmation dialog. No email notification.
^ Another such pattern. Not just because it's three of them, but also because of the content and repetition. Humans rarely write like that because it again sounds overly dramatic. It's something you see in fiction rather than a technical writeup. In a thriller.
> Retroactive Privilege Expansion. You created a Maps key three years ago and embedded it in your website's source code, exactly as Google instructed. Last month, a developer on your team enabled the Gemini API for an internal prototype. Your public Maps key is now a Gemini credential. Anyone who scrapes it can access your uploaded files, cached content, and rack up your AI bill. Nobody told you.
This style of scenario writing is another one.
> Nobody told you.
Absolute drama queen.
>The UI shows a warning about "unauthorized use," but the architectural default is wide open.
Again.
> The attacker never touches your infrastructure. They just scrape a key from a public webpage.
Again.
> These aren't just hobbyist side projects. The victims included major financial institutions, security companies, global recruiting firms, and, notably, Google itself.
..
> A key that was deployed years ago for a completely benign purpose had silently gained full access to a sensitive API without any developer intervention.
Surprised it hasn't gained consciousness by now. Maybe that's a future plot point.
Here's a great example to train your skills on, because it's rare in that the ratio of "human : straight from LLM" increased gradually as the article goes on: https://www.wallstreetraider.com/story.html
It started at heavy human editing (or just human-written), but less and less towards the end.
The author confirmed this upon pointing it out, FWIW [0].
[0] https://news.ycombinator.com/item?id=47013150
They don't. Many of these claims are due to illiteracy.
Someone is complaining that
> it's all just crisp and clean structured and actionable in a way that a meandering human would not distill it down to.
but this is a security report ... people intentionally write such things carefully and crisply with multiple edits and reviews.
This is true but also not as new as the author claims. There have been various ways to abuse Google API keys in the past (at least to abuse them financially) and it’s always been very confusing for developers.