Though I'm in the camp "people should really know to sandbox by now and be careful", I'd say we should also be mindful of how far from everyone has deep knowledge of the systems and tools they use. This behaviour of a tool is just malicious. You have to take into account the human factor, of how people likely end up using a system. And in this case, the consequences of exfiltrating so many secrets this way are really quite unacceptable.
macOS largely _does_ bake this into the OS, and it is annoying. They also provide a way to turn it off for specific applications (including, for example, Terminal.app).
I've found that I can usually write apps that respect it. MacOS is a free-love hippie, compared to iOS. In many cases, we have no choice.
It's annoying, and often rather infuriating, but I understand that one of the motivations for people buying Apple stuff, is for that very reason, so I'm really sawing off the branch that I'm sitting on, by trying to work around it.
Not to mention the very wide push to "Use AI NOW, for EVERYTHING!" in marketing ans many companies, with hardly any though given to safety or where does all the data end up.
Not gonna argue about the utility of AI, but isn't the statement "AI is not here to help people" completely meaningless? AI itself is not "here" for anything; the problem is big tech doing big tech shit as always, not the current technology they're doing it with.
AI is created by big tech stealing other people's data. Yes, this has everything to do with AI - stealing data is a foundational feature of the technology.
True, but it isn't here to not help people, either.
It's a spanner. Who wields the spanner, makes all the difference.
We've spent the last couple of decades, cultivating a huge crop of ultimate scumbag billionaires, with comically exaggerated sociopathy, and that has filtered down to almost every level of society. They are treated as gods, these days (they certainly think of themselves that way).
It still shocks me (but really shouldn't), on a daily basis, to encounter regular folks, interacting in stores and restaurants, or driving on roads, that mirror the values systems exemplified by our billionaires. Our politicians act that way, and one of their biggest selling points, is normalizing sociopathy (not just the US, either).
The first clue should have been when all the Silicon Valley CEO’s lined up to kiss the ring after the second Trump victory. Remember it wasn’t that long before that “woke” tech companies were derided and accused by the same factions they suddenly found themselves in good standing with. There were entire pushes regarding section 230, etc. to go against social media companies, constant complaints about “Facebook” jails and shadow banning. Now they’re all buddy buddy.
Anyway, ghouls like Thiel are now a well known name among populist left and right as an enemy, so maybe some good may come from this.
(an aside but the majority of the US population didn't elect Trump. He is in office because of the electoral college. Might seem like a distinction without a difference but I think it matters when we're implying personal culpability)
If the agent you are running really wanted to it could easily find a way to mount the windows folders and read them all. WSL isn't a security boundary, you are only barely more protected than people running grok in their home directory.
I never thought about it in that extreme, but just today the google results ai gave me conflicting information from one search to another. Maybe I should start.
P.S. the conflicting information was Keith Richard's age. One search said he was 37 Dec 18 1981, the other said he was 37 in 1980.
My first thought would be their server side extentions, code excecutoon sandboxes and document RAG search, being on by default? Probably should be an opt-in instead of an opt-out.
This wasn't the LLM, it was Grok CLI preemptively uploading the entire CWD, regardless of where that CWD is, to its own server.
I don't think it is reasonable to expect every user (including those just starting out with the tools - maybe experimenting, maybe younger/less experienced in general) to think that the tool they're running for the very first time is going to automatically exfiltrate all of their data.
It's a pretty serious fuck-up. This guy tweeted about it, who knows how many didn't even notice. It should have been opt-in, it should give user an indication that it's about to do this, etc.
The grok-cli is on github[0] there is nothing that I can see in the code that is activily looping ~/ and uploading everything.
My two guesses would be one the LLM decided it needed these files for the task or two the user simple asked grok to do it so they could post the tool calls on twitter.
That is not the Grok CLI being discussed. That's an open source, third party CLI. https://x.ai/cli is the official Grok CLI being discussed, and it is not open source.
I think there are arguments on both sides. People should look for guidance on how to use complex tools, but we know people will not.
Whose fault is it if someone drives a car without learning how to and injures themselves? On the other hand if the manufacturer has promoted it as one you can drive without learning how to, then whose fault is it?
A lot of users are fine with everything being uploaded. Most people's primary computing device is now a phone that backs up everything to cloud and using apps that are thin front ends over cloud services.
If your immediate reaction to a new piece of software siphoning up someone’s entire system full of highly personal data is, “you’re holding it wrong”, it might help to take a beat and remember that software was developed by a multi-trillion dollar company’s entire business model revolves around siphoning up as much highly personal data as possible
The point is more that you should not blame the user (why didn't you set up sandbox instead of directly using the tool of big corp) if a tool does something unexpected. If your Dropbox client would suddenly just upload your home directory instead of it's folder you configured you'd also not blame the user that they use Dropbox, you'd blame Dropbox for not doing their job correctly or being user hostile.
Agreed. You can still encourage people to use defense in depth without actively blaming them for not having the deepest moat imaginable. Software creators still have some responsibility
Maybe possibly with --dangerously-skip-permissions. I've been using auto mode and enjoy how it blocks every tool use that could've allowed something potentially sensitive into context. Burns extra tokens though.
But Claude Code, arguably one of the most famous ones, is not. And recently got some heat about sending meta data that wasn't so obvious. Just as a counter-example.
This sort of stuff is precisely why running local models has to be the future. It's absolutely insane that we just send our code to the cloud like this, and we basically have to trust these companies with it.
https://gist.github.com/cereblab/dc9a40bc26120f4540e4e09b75ffb547
Elon did this horrible thing, so I made grok build available for omp with it's own endpoint; Without sending your private repos and secret keys to them.
-
oh-my-pi-plugin-grok-build
Standalone oh-my-pi extension for the xAI Grok Build subscription provider. It adds OAuth login, authoritative model discovery, and OpenAI Responses streaming with the request identity expected by Grok Build.
Install (No-spywares):
omp plugin install oh-my-pi-plugin-grok-build
-
https://github.com/metaphorics/oh-my-pi-plugin-grok-build
Star me if you like it or if you hate spywares, lol.
Is the Grok CLI a 2 terabyte install? Did Elon dropship you an 8U rack of B200s?
No?
Well the model weights, the GPUs, and the context obviously all have to be in the same place, so “sending your project to them” is literally the only thing that could possibly happen, unless you think agents work by fucking magic.
This is the biggest case of PEBKAC in history, maybe ever.
This is the kind of confusion that Charles Babbage could not rightly comprehend, except at those politicians at least had the excuse that computers had only been invented five minutes prior.
If you decide that your entire home directory is the project, as the OP did by setting the repo_path to ~/, then, well… I mean, if you ask me, I don’t recommend it, but it’s your computer and your free will.
Then that should be considered a 'nuke button' and nuke buttons don't find themselves into well-designed software for the simple fact that an end user shouldn't have to worry about shooting themselves in the foot. Again, with good software.
It's fascinating how many people in this conversation think that LLMs need to have all of the files in your $CWD on the model provider's servers to be able to do anything.
Though I'm in the camp "people should really know to sandbox by now and be careful", I'd say we should also be mindful of how far from everyone has deep knowledge of the systems and tools they use. This behaviour of a tool is just malicious. You have to take into account the human factor, of how people likely end up using a system. And in this case, the consequences of exfiltrating so many secrets this way are really quite unacceptable.
These tools are explicitly marketed as a way for non-technical people to code. If we expect those same people to understand sandboxing we're dreaming.
True, but that's a fantasy happy path. It will never happen, for most people. Only HN people will do that.
It needs to be baked into the OS.
At that point, HN users start screeching about it, so it's lose/lose, really.
macOS largely _does_ bake this into the OS, and it is annoying. They also provide a way to turn it off for specific applications (including, for example, Terminal.app).
I've found that I can usually write apps that respect it. MacOS is a free-love hippie, compared to iOS. In many cases, we have no choice.
It's annoying, and often rather infuriating, but I understand that one of the motivations for people buying Apple stuff, is for that very reason, so I'm really sawing off the branch that I'm sitting on, by trying to work around it.
We should also be mindful of how much these tools break down the "be careful and thoughtful" barriers in favor of more and more convenience.
Not to mention the very wide push to "Use AI NOW, for EVERYTHING!" in marketing ans many companies, with hardly any though given to safety or where does all the data end up.
So is X going to claim the user disabled something the second before everything went south? That's what the owner's other company does.
I am genuinely fascinated by this.
I don’t like piling on especially with security vulnerabilities, but man how many red flags do you need to ignore?
They won’t stop abusing us until we stop using their products.
> They won’t stop abusing us until we stop using their products.
I don't use AI at all in my daily life.
Work however will demand you use it.
AI is not here to help people.
Not gonna argue about the utility of AI, but isn't the statement "AI is not here to help people" completely meaningless? AI itself is not "here" for anything; the problem is big tech doing big tech shit as always, not the current technology they're doing it with.
Nothing about this has anything to do with AI. It has to do with Musk's ethical and engineering standards, or the lack thereof.
AI is created by big tech stealing other people's data. Yes, this has everything to do with AI - stealing data is a foundational feature of the technology.
> AI is not here to help people.
True, but it isn't here to not help people, either.
It's a spanner. Who wields the spanner, makes all the difference.
We've spent the last couple of decades, cultivating a huge crop of ultimate scumbag billionaires, with comically exaggerated sociopathy, and that has filtered down to almost every level of society. They are treated as gods, these days (they certainly think of themselves that way).
It still shocks me (but really shouldn't), on a daily basis, to encounter regular folks, interacting in stores and restaurants, or driving on roads, that mirror the values systems exemplified by our billionaires. Our politicians act that way, and one of their biggest selling points, is normalizing sociopathy (not just the US, either).
I think my first clue was when their CEO hired a bunch of teenage hackers to sack the government and exfiltrate all our data.
I didn't really need a second clue.
My first clue was when he libelled the diver during the Thailand thing in 2018; it was all downhill from there.
...it was quite the sting because I bought a Tesla car only 2 weeks prior to that.
It's not a subtle pattern.
Most people don’t know about that because they live in an information bubble handcrafted by the oligarchs.
The first clue should have been when all the Silicon Valley CEO’s lined up to kiss the ring after the second Trump victory. Remember it wasn’t that long before that “woke” tech companies were derided and accused by the same factions they suddenly found themselves in good standing with. There were entire pushes regarding section 230, etc. to go against social media companies, constant complaints about “Facebook” jails and shadow banning. Now they’re all buddy buddy.
Anyway, ghouls like Thiel are now a well known name among populist left and right as an enemy, so maybe some good may come from this.
I mean, do people expect companies to protect them from a tyrant they themselves elected?
Not necessarily speaking of the present. This seems to be the general sentiment.
If the tyrant goes against established law, yes.
(an aside but the majority of the US population didn't elect Trump. He is in office because of the electoral college. Might seem like a distinction without a difference but I think it matters when we're implying personal culpability)
Well, it looks like he was running the agent in his home directory to begin with considering the `repo_path` field is exactly that.
Why do people run and install these agents locally? No container nothing. I am running Opencode in WSL2 with the windows mounts disabled.
If the agent you are running really wanted to it could easily find a way to mount the windows folders and read them all. WSL isn't a security boundary, you are only barely more protected than people running grok in their home directory.
So the idea is that these should be treated as programs in an extremely low trust environment, akin to running malware in a VM?
yes, this is basically experimental tech, if used with open source harnesses. if used with proprietary harnesses, treat as actual malware.
Technically open source harness like opencode is more "malwarey" than claude code for example because its default permissions are very open.
I never thought about it in that extreme, but just today the google results ai gave me conflicting information from one search to another. Maybe I should start.
P.S. the conflicting information was Keith Richard's age. One search said he was 37 Dec 18 1981, the other said he was 37 in 1980.
I treat all proprietary software as malware, though of course the risk surface varies
Yes, I think if agents as interns with enough smarts to be dangerous.
Same reason one uses VSCode locally
My first thought would be their server side extentions, code excecutoon sandboxes and document RAG search, being on by default? Probably should be an opt-in instead of an opt-out.
why do people give these LLMs full access to everything and then complain when it does somethign stupid? that is what sandboxes are for.
This wasn't the LLM, it was Grok CLI preemptively uploading the entire CWD, regardless of where that CWD is, to its own server.
I don't think it is reasonable to expect every user (including those just starting out with the tools - maybe experimenting, maybe younger/less experienced in general) to think that the tool they're running for the very first time is going to automatically exfiltrate all of their data.
It's a pretty serious fuck-up. This guy tweeted about it, who knows how many didn't even notice. It should have been opt-in, it should give user an indication that it's about to do this, etc.
The grok-cli is on github[0] there is nothing that I can see in the code that is activily looping ~/ and uploading everything.
My two guesses would be one the LLM decided it needed these files for the task or two the user simple asked grok to do it so they could post the tool calls on twitter.
[0] https://github.com/superagent-ai/grok-cli
That is not the Grok CLI being discussed. That's an open source, third party CLI. https://x.ai/cli is the official Grok CLI being discussed, and it is not open source.
thanks for the correction
I think there are arguments on both sides. People should look for guidance on how to use complex tools, but we know people will not.
Whose fault is it if someone drives a car without learning how to and injures themselves? On the other hand if the manufacturer has promoted it as one you can drive without learning how to, then whose fault is it?
A lot of users are fine with everything being uploaded. Most people's primary computing device is now a phone that backs up everything to cloud and using apps that are thin front ends over cloud services.
Every driver needs to learn how to drive, that's why it's called a "driver's license"
If your immediate reaction to a new piece of software siphoning up someone’s entire system full of highly personal data is, “you’re holding it wrong”, it might help to take a beat and remember that software was developed by a multi-trillion dollar company’s entire business model revolves around siphoning up as much highly personal data as possible
Well said. I hope one day it becomes possible for users who choose to install and run said software to also be able to remember this.
When I give my text editor or file browser access to everything I wouldn't expect it to exfiltrate data without asking.
Isn’t a file browser running locally, while Grok is running on someone else’s server?
The point is more that you should not blame the user (why didn't you set up sandbox instead of directly using the tool of big corp) if a tool does something unexpected. If your Dropbox client would suddenly just upload your home directory instead of it's folder you configured you'd also not blame the user that they use Dropbox, you'd blame Dropbox for not doing their job correctly or being user hostile.
Agreed. You can still encourage people to use defense in depth without actively blaming them for not having the deepest moat imaginable. Software creators still have some responsibility
We are speedrunning the various phases of learning about victim-blaming, as a community.
It’s kind of wild to watch in real time, instead of over the decades it took society for SA.
Is it 'unexpected' when we've been hearing stories like this every week for 2 years now?
Not every Anthropic user follows HN or random X posts about these issues.
Stories about copilot messing things up made into regular newspapers...
Do Anthropic users consider themselves clever enough to not make the same mistakes as Microsoft?
Other ones aren't this invasive with user data.
not true, Claude code on its own often create artifacts and straight up upload private stuff to Anthropic, without asking for it.
Then show us the example of Claude uploading a home directory to Anthropic because we have an example of Grok uploading a home directory to X.
Maybe possibly with --dangerously-skip-permissions. I've been using auto mode and enjoy how it blocks every tool use that could've allowed something potentially sensitive into context. Burns extra tokens though.
Auto mode will upload artifacts to Anthropic without asking at least sometimes, for example it did upload my slide decks.
Are we sure about that?
Codex is opensource, there are other opensource harnesses.
But Claude Code, arguably one of the most famous ones, is not. And recently got some heat about sending meta data that wasn't so obvious. Just as a counter-example.
https://xcancel.com/a_green_being/status/2076598897779020159
Posting a complaint about Elon on Elon's platform and tagging him is ballsy. He tends to limit visibility of accounts who do that.
ballsy only if you care about participating in that shithole of a platform.
Why is that page not there anymore?
Weird, right? Especially on a website with an owner who so ardently supports free speech. </heavy sarcasm>
https://x.com/a_green_being/status/2076598897779020159
the post has been deleted!! So much for freedom of speech.
The post is still there: https://x.com/a_green_being/status/2076598897779020159
Not sure which weirdness happened here
Dupe: https://news.ycombinator.com/item?id=48892512
Fully agentic development (supervised)
This is particularly the whole reason why I created https://github.com/VibePod/vibepod-cli
If someone wants to contribute the Grok CLI I'm happy to support it.
Closed source coding agents are just complete info stealing malware. Both Claude and Grok were caught stealing info from your own machines.
This is why it is important to use open source harnesses instead of shady closed ones.
Run your agents in podman containers.
Or don't use closed source harnesses at all.
This sort of stuff is precisely why running local models has to be the future. It's absolutely insane that we just send our code to the cloud like this, and we basically have to trust these companies with it.
screenshot of the tweet, back it up :)
https://ibb.co/ycs6K4c9
Copied this from discord:
Some more discussion: https://news.ycombinator.com/item?id=48892512
Rules don't apply to certain CEOs.
You have to put them into a RULES.md of course!
That's the only file on your computer the AI won't read.
Alex Karp was right, AI Compagnies are stealing people code while making them pay for unproductive tokens
Is the Grok CLI a 2 terabyte install? Did Elon dropship you an 8U rack of B200s?
No?
Well the model weights, the GPUs, and the context obviously all have to be in the same place, so “sending your project to them” is literally the only thing that could possibly happen, unless you think agents work by fucking magic.
This is the biggest case of PEBKAC in history, maybe ever.
This is the kind of confusion that Charles Babbage could not rightly comprehend, except at those politicians at least had the excuse that computers had only been invented five minutes prior.
Haha so just send over your entire home directory including password managers and home videos every time you need some python code rewritten.
Only a buffoon would be confused by the straightforward logic.
If you decide that your entire home directory is the project, as the OP did by setting the repo_path to ~/, then, well… I mean, if you ask me, I don’t recommend it, but it’s your computer and your free will.
Then that should be considered a 'nuke button' and nuke buttons don't find themselves into well-designed software for the simple fact that an end user shouldn't have to worry about shooting themselves in the foot. Again, with good software.
You’re telling me LLMs aren’t actually magic?
That's not how any other coding harness/agent works. Why confidently comment on stuff you know nothing about?
It's fascinating how many people in this conversation think that LLMs need to have all of the files in your $CWD on the model provider's servers to be able to do anything.