I was one of the “lucky” few to witness the school of slur-fish.
Being in security I laughed because of how egregious it was but also because I knew someone on HN with some actual time on their hands to help properly would be along soon.
I also appreciate this post mortem. Vibe-coded anything in prod is a lot of my work load in IR these days but it was nice to see such a low stakes project properly documented.
People will be quick to jump on the "it was vibe coding's fault" but at least two of the issues are pretty common even in designed systems without AI - leaving in a "test admin" access and verifying tokens but not cross-checking them.
This is pretty reductive of the actual problem people typically complain about with vibe coding - It produces very workable prototypes fairly quickly and without a lot of hassle. Great! The problem is, and this is a great example (of many) where someone mistook the working prototype with a system that was ready for production. The JWT thing in particular is not really a mistake many people who work on that kind of thing would make.
People need more understanding of the risks of vibe coding and YOLOing to prod with these tools. They are powerful, but like all powerful tools, can be wielded irresponsibly.
most in-use LLMs prompted with a simple "You're in charge of infrastructure security, let's review possible problem points" would have uncovered this.
I wouldn't fault a compiler for erring when someone left out a period; i'd tell the person to start including it -- but for some reason the expectation for LLMs is hands-off work ; I guess we're just in that phase of the hype at the moment.
I thought checking a token against the cert is actually called verifying or is noawadys verifying just if it looks like a token it maybe a valid token?
I think it's pretty reasonable to expect AI to produce systems with issues "pretty common even in designed systems without AI" because that's what AI was trained on.
I expect these AI and LLM to be, basically, a middle of the bell-curve type producer of code. Just like their other output. Not terrible, not exceptional, just what a Mid could do - only faster.
Not sure what's being marketed, but I expect mediocre.
It does sound condescending. I think the sentiment is important though. Asking someone to be specific can help them think clearly. What’s a nicer way to do that?
I think "Who, specifically, claims that [...]?" comes off as less condescending than "Who claims that [...]? Be specific." just by virtue of the latter using imperative language, which triggers a reflexive "you're not the boss of me" reaction.
The message is clear in both cases. It's easier to put aside these irrational reflexive reactions and think about
whatever worth can be derived from the message than it is to carefully manage the emotions of varied readers whom you don't know. This is different from bring overtly inflammatory, although the lines for this are subjective.
Ultimately it's probably not a productive use of time to be commenting here at all from a strict EV perspective. Meaning that if you're posting here, you're probably getting something else out of it. The value of that "something else" determines how you should approach the problem of managing the gut reactions of your readers.
If someone asks for a better way to word something to reduce reader hostility to their point, I assume that they will be better off for knowing the answer to that question, and can decide for themselves whether they want to change their writing style or not - and, whether they do or do not, the effects of their writing will be more intentional.
I actually did see one, while the site was #1 and well before the overnight excitement. (Good grief, even at this late age I have something to do with my Friday evenings...)
I don't really know what you want me to tell you about it. The swastika per se as I recall had to be drawn backwards, because there is no meaningful overlap between its outline and that of a fish, so unlike the penis case this is very easy for the classifier. It wasn't clever and it wasn't funny. Several people reported it and it quickly disappeared, whereupon apparently someone decided we shouldn't have nice things, or not for a little while at least.
> But if you had the displeasure of viewing my website between the hours of 2AM (20 minutes after I went to sleep) and 8AM (when I woke up) EST on Aug 3, then you would have seen chaos. Every single username was transformed to a heinous slur, many unsavory fish had made it into the fishtank, and many beautiful fish were gone.
At some point, asking for more starts to seem like rubbernecking at a car crash, you know?
Great post-mortem, especially since it's a vibe-coded app.
Curious if you were inspired by Lego's build-a-fish* exhibit at the Lego House? I visited recently and it is ridiculously addictive to see a fish you create swim with others :)
Oh! I hadn't seen this. I was more inspired by the St Louis aquarium (where you color a fish and it swims) and Google's Quickdraw (a memory from like 2016)
I remember reading about 5 or so years ago that the FBI was doing exactly this to counter the EternalBlue exploit (I might be getting the exploit name wrong)
Got a killer original idea, got it built, learned a ton, is embarrassed about a few failures because he’s actually a solid professional. Well done, mate. If it didn’t cost you $100k, take down the network or lose you your job, you’re ahead and will laugh in a year.
I love the fact that you did a post mortem on a vibe-coded website
Honestly - i think often so many people take tech very seriously so seeing this is quite refreshing and genuinely interesting from small side coded project point of view.
That's actually intentional desgin - I think you can like a fish a little or like a fish a lot, and therefore should be able to upvote/downvote to your hearts content :)
The only potential risk I see with anonymous voting is some 4chaner is apt to create a bunch of fish with horrific names and use distributed IPs to upvote them to the top. I guess regular moderation could filter that out.
The first case would be an issue yes, although this app doesn't exactly strike me as something that an insane amount of people would be connecting to from corporate networks
It can work okay for apps that don't go to massive scale and where users aren't concentrated in the same place. The chance of another simultaneous user on your block/building/CGNAT isn't extremely likely.
> I used the JWT to authorize login, but never confirmed that the JWT token belonged to the userId / email associated with it in the admin actions. So you could log in with my username and password, grab the JWT, and then send that along with your request.
IANAWD: What is more appropriate than an admin token being able to authenticate admin actions?
I've noticed with nearly all of these "Vibe Code" security fatalities, they're nearly ALWAYS using Firebase as a backend. I get it, I've used Firebase for a number of enterprise and personal projects, its convenient and easy to setup.
But even before LLM coding, I had team members walk into its numerous footguns - especially around public buckets and bad firestore rules. How many of these stories are really to be blamed on the AI tooling, and how many could be blamed on the very poor default settings of Firebase?
But if you show HN something... and it makes HN-reading KF users think "OH EXPLOITABLE!!!111!" so they post it to KF... they're basically going to test if your moderation works or not. If you only have some automated rules or even "AI" as the moderation, not humans, then they're very quickly going to work out what gets past your automated filter and what doesn't, faster than 4chan can make Microsoft's Tay say "Hitler did nothing wrong".
Then the KF users will gloat about the slur-ridden fish they've made.
Then KF-reading Sharty users will learn about it, the rest of the Sharty will join in the game.
Then they'll get angry that your "normie" userbase has given a Palestine flag fish named "River to the Sea" the most upvotes, and an Israel flag fish the most downvotes, they'll want to cheat and hack their own creations to the top. So they'll start looking for an admin interface.
When they find one, but find it needs credentials, the first place they'll go looking for password clues is all your socials, all your previous forums, basically your whole identity. And if they hit paydirt, of course they'll post whatever they find, because for them, doxxing people is fun too.
Just scroll to the bottom of this page and look at the idiots bragging about making swastika fishes. HN nurtures this community of mentally ill edgelords that we could very well do without.
At my prior work we had this garage where we could wash our cars. But once the sponge was in the wrong place and the responsible foreman shut the whole thing down. "This is why we can't have nice things" he said.
Just ignore the trolls and wait for the fish mods to ban them.
Here in Zurich there's a mural of maybe twenty dinosaurs (not accurate but something that looks like it would be in a children's book). One day someone drew a dick on every single dinosaur. Even the flying pterodactyl had a big dick hanging off of him. It was so puerile and primitive it cracked everyone up that saw it. No tags. No football club graffiti. Just dicks everywhere. Thankfully the mural was repaired pretty quickly.
There's a screen shot of a certain site in the footnotes of the blog and if you visit it, you can probably find some screenshots in the 30-something page thread about OP's game.
Edit: yes, the site too terrible to name as evidenced by the reply to this comment becoming dead within minutes l-o-l
I am sure someone somewhere works on making LLMs commit code. Aside from that it was great witnessing the site in action and reading the postmortem. I wonder how the "hacker" made the connection to the user acount on neopets.com but maybe they just tried something like "ahallak"?
To be fair, it says the attention was unexpected, and this was just a coding exercise... And the port-mortem shows what I'd hope to see: digging in and figuring out root causes. So I'm not judging OP poorly over this.
But still. Launching a vibe-coded app that accepts input from anonymous users is just asking for trouble. I'm frankly surprised it ran as long as it did without such problems. (Although I did see a few weenies swimming around even before the problem hours.)
The lesson I'd pull from this is that if you are not the type of dev who could put together a post-mortem along these lines... don't launch a vibe-coded app.
Ultimately an app like this caused more joy than harm. I'm all for people vibe coding fun little things like this when the stakes are low. Would prefer to see more non coders with diverse ideas feeling empowered to start a project rather than them seeing a huge wall to climb and never starting. The web needs more silly apps from silly people.
I was proud of my ability to have an extremely penis looking fish get past filters. I feel like when presented with censorship my instinct is to always test its limits
I read the HN thread first, and the first comment I saw from the author was about AI nazi symbol detector they put in... I wonder how many orders of magnitude that comment increased interest in making offensive fish?
Later I saw images of the attacked site posted elsewhere and thought they were both predictable and hilarious.
A better group psychological approach might have been to only name the penis filter, ye. Then the transgressions are on the vandal who can't rationalize it is sticking it to the man, but that he is just that guy drawing swastikas.
I was one of the “lucky” few to witness the school of slur-fish.
Being in security I laughed because of how egregious it was but also because I knew someone on HN with some actual time on their hands to help properly would be along soon.
I also appreciate this post mortem. Vibe-coded anything in prod is a lot of my work load in IR these days but it was nice to see such a low stakes project properly documented.
People will be quick to jump on the "it was vibe coding's fault" but at least two of the issues are pretty common even in designed systems without AI - leaving in a "test admin" access and verifying tokens but not cross-checking them.
This is pretty reductive of the actual problem people typically complain about with vibe coding - It produces very workable prototypes fairly quickly and without a lot of hassle. Great! The problem is, and this is a great example (of many) where someone mistook the working prototype with a system that was ready for production. The JWT thing in particular is not really a mistake many people who work on that kind of thing would make.
People need more understanding of the risks of vibe coding and YOLOing to prod with these tools. They are powerful, but like all powerful tools, can be wielded irresponsibly.
it's just incompleteness -- a human issue.
most in-use LLMs prompted with a simple "You're in charge of infrastructure security, let's review possible problem points" would have uncovered this.
I wouldn't fault a compiler for erring when someone left out a period; i'd tell the person to start including it -- but for some reason the expectation for LLMs is hands-off work ; I guess we're just in that phase of the hype at the moment.
> I wouldn't fault a compiler for erring when someone left out a period
I'd fault it if it silently injected multiple serious vulnerabilities.
I thought checking a token against the cert is actually called verifying or is noawadys verifying just if it looks like a token it maybe a valid token?
I think it's pretty reasonable to expect AI to produce systems with issues "pretty common even in designed systems without AI" because that's what AI was trained on.
But that isn’t the expectation or what is being marketed
I expect these AI and LLM to be, basically, a middle of the bell-curve type producer of code. Just like their other output. Not terrible, not exceptional, just what a Mid could do - only faster.
Not sure what's being marketed, but I expect mediocre.
Being marketed by who? Be specific.
“Be specific.”
Am I the only one that feels like it’s really condescending when people say this on the internet?
It sounds like something you would see on a community college writing assignment
It does sound condescending. I think the sentiment is important though. Asking someone to be specific can help them think clearly. What’s a nicer way to do that?
I think "Who, specifically, claims that [...]?" comes off as less condescending than "Who claims that [...]? Be specific." just by virtue of the latter using imperative language, which triggers a reflexive "you're not the boss of me" reaction.
The message is clear in both cases. It's easier to put aside these irrational reflexive reactions and think about whatever worth can be derived from the message than it is to carefully manage the emotions of varied readers whom you don't know. This is different from bring overtly inflammatory, although the lines for this are subjective.
Ultimately it's probably not a productive use of time to be commenting here at all from a strict EV perspective. Meaning that if you're posting here, you're probably getting something else out of it. The value of that "something else" determines how you should approach the problem of managing the gut reactions of your readers.
If someone asks for a better way to word something to reduce reader hostility to their point, I assume that they will be better off for knowing the answer to that question, and can decide for themselves whether they want to change their writing style or not - and, whether they do or do not, the effects of their writing will be more intentional.
Dario Amodei: https://www.windowscentral.com/software-apps/work-productivi...
'By whom.'
By Brian Sheltzer, 302 Main St, Chicago, Illinois.
Not even a screenshot, I really wanted to see the swastikarp.
I actually did see one, while the site was #1 and well before the overnight excitement. (Good grief, even at this late age I have something to do with my Friday evenings...)
I don't really know what you want me to tell you about it. The swastika per se as I recall had to be drawn backwards, because there is no meaningful overlap between its outline and that of a fish, so unlike the penis case this is very easy for the classifier. It wasn't clever and it wasn't funny. Several people reported it and it quickly disappeared, whereupon apparently someone decided we shouldn't have nice things, or not for a little while at least.
Yes but what was the incident like, in general? Was it just swastikas, or words too, or what? When I saw it, it was just normal fish.
Quoting from the article's third paragraph:
> But if you had the displeasure of viewing my website between the hours of 2AM (20 minutes after I went to sleep) and 8AM (when I woke up) EST on Aug 3, then you would have seen chaos. Every single username was transformed to a heinous slur, many unsavory fish had made it into the fishtank, and many beautiful fish were gone.
At some point, asking for more starts to seem like rubbernecking at a car crash, you know?
Great post-mortem, especially since it's a vibe-coded app.
Curious if you were inspired by Lego's build-a-fish* exhibit at the Lego House? I visited recently and it is ridiculously addictive to see a fish you create swim with others :)
https://www.youtube.com/watch?v=KYs3ne0HCwM
Oh! I hadn't seen this. I was more inspired by the St Louis aquarium (where you color a fish and it swims) and Google's Quickdraw (a memory from like 2016)
TeamLabs also has this in both of their main Tokyo art installations (Borderless & Planets).
Wild that some random used a security hole to try and counter the malicious actions actively lol
This has happened a number of times that I remember - one was a worm/exploit that would patch the hole.
I remember reading about 5 or so years ago that the FBI was doing exactly this to counter the EternalBlue exploit (I might be getting the exploit name wrong)
I've seen multiple articles where Microsoft does this by taking over C&C domains with court orders
Relevant link maybe: https://www.justice.gov/archives/opa/pr/justice-department-a...
this was awesome. people are surprising
There's a long history of this. A defense against the Morris worm made use of this as well.
The "S" in "vibe-coded" stands for security.
The S is for snake... errr.. dragon...
I said more competent Vs!
*consummate Vs
Got a killer original idea, got it built, learned a ton, is embarrassed about a few failures because he’s actually a solid professional. Well done, mate. If it didn’t cost you $100k, take down the network or lose you your job, you’re ahead and will laugh in a year.
There's a swastika fish on there right now. Folks get through the filter by putting it inside a fish shape:
https://drawafish.com/rank.html?userId=1754341779700_log2xle...
Edit: it's been deleted.
That fish bears the touch of the Buddha himself.
I love the fact that you did a post mortem on a vibe-coded website
Honestly - i think often so many people take tech very seriously so seeing this is quite refreshing and genuinely interesting from small side coded project point of view.
You can also upvote any fish without auth, limit is 20 votes per minute per IP
POST https://fishes-be-571679687712.northamerica-northeast1.run.a... {"fishId":"xxxx","vote":"up"}
That's actually intentional desgin - I think you can like a fish a little or like a fish a lot, and therefore should be able to upvote/downvote to your hearts content :)
The only potential risk I see with anonymous voting is some 4chaner is apt to create a bunch of fish with horrific names and use distributed IPs to upvote them to the top. I guess regular moderation could filter that out.
IP based breaks users with CGNAT and people connecting from corporate networks
In those cases you'd be denied votes since someone else used them up
The first case would be an issue yes, although this app doesn't exactly strike me as something that an insane amount of people would be connecting to from corporate networks
It can work okay for apps that don't go to massive scale and where users aren't concentrated in the same place. The chance of another simultaneous user on your block/building/CGNAT isn't extremely likely.
CGNAT is used in places other than corporate networks.
I'm aware of that - I spoke to the corporate network part specifically as the post I was replying to specifically listed that in addition to CGNAT
Any button a human can touch is a vector for abuse...
You write with such whimsy, absolutely delightful :)
Great project and a good post too, you write well and are funny. Would like to see more for my rss reader :-)
…if you had RSS…
I was 15 years old when they killed google reader, so can you blame me for forgetting about RSS?
Added :) https://aldenhallak.com/blog/rss.xml
Ha! Subscribed, and now the pressure is on to write something ;-)
> I used the JWT to authorize login, but never confirmed that the JWT token belonged to the userId / email associated with it in the admin actions. So you could log in with my username and password, grab the JWT, and then send that along with your request.
IANAWD: What is more appropriate than an admin token being able to authenticate admin actions?
I've noticed with nearly all of these "Vibe Code" security fatalities, they're nearly ALWAYS using Firebase as a backend. I get it, I've used Firebase for a number of enterprise and personal projects, its convenient and easy to setup.
But even before LLM coding, I had team members walk into its numerous footguns - especially around public buckets and bad firestore rules. How many of these stories are really to be blamed on the AI tooling, and how many could be blamed on the very poor default settings of Firebase?
If you are the dev think you need to add an option to mirror the fish drawing.
I can draw a fish facing left, but for some reason it's very difficult to draw one facing right.
Can't you just draw facing left and then flip it horizontally?
Sorry, that's what i meant, i think it needs an option to flip it horizontally on the page, not mirror it.
That is mirroring?
That was quite something. Thanks for all the fish. Also for posting this.
> You may have heard differently, especially if you saw my doxxing[4b] on the unsavory website.
Is it pretty common to get doxxed for getting to the top of HN?
It is not.
But if you show HN something... and it makes HN-reading KF users think "OH EXPLOITABLE!!!111!" so they post it to KF... they're basically going to test if your moderation works or not. If you only have some automated rules or even "AI" as the moderation, not humans, then they're very quickly going to work out what gets past your automated filter and what doesn't, faster than 4chan can make Microsoft's Tay say "Hitler did nothing wrong".
Then the KF users will gloat about the slur-ridden fish they've made.
Then KF-reading Sharty users will learn about it, the rest of the Sharty will join in the game.
Then they'll get angry that your "normie" userbase has given a Palestine flag fish named "River to the Sea" the most upvotes, and an Israel flag fish the most downvotes, they'll want to cheat and hack their own creations to the top. So they'll start looking for an admin interface.
When they find one, but find it needs credentials, the first place they'll go looking for password clues is all your socials, all your previous forums, basically your whole identity. And if they hit paydirt, of course they'll post whatever they find, because for them, doxxing people is fun too.
This comedy bit explains the thinking of such folk: https://www.youtube.com/watch?v=eibWo9t6pos#t=10m
Imagine that, hackers reading Hacker News.
Just scroll to the bottom of this page and look at the idiots bragging about making swastika fishes. HN nurtures this community of mentally ill edgelords that we could very well do without.
At my prior work we had this garage where we could wash our cars. But once the sponge was in the wrong place and the responsible foreman shut the whole thing down. "This is why we can't have nice things" he said.
Just ignore the trolls and wait for the fish mods to ban them.
I wish there some screenshots of the vandalism.
Here in Zurich there's a mural of maybe twenty dinosaurs (not accurate but something that looks like it would be in a children's book). One day someone drew a dick on every single dinosaur. Even the flying pterodactyl had a big dick hanging off of him. It was so puerile and primitive it cracked everyone up that saw it. No tags. No football club graffiti. Just dicks everywhere. Thankfully the mural was repaired pretty quickly.
There's a screen shot of a certain site in the footnotes of the blog and if you visit it, you can probably find some screenshots in the 30-something page thread about OP's game.
Edit: yes, the site too terrible to name as evidenced by the reply to this comment becoming dead within minutes l-o-l
I've never understood why everybody gets upset over New Zealand farmers.
That was a fun read, nice work
This was so fun. Hope to see more like this
On the allure of vide coding the author says,
> It is really fun to just have high velocity, and it is really fun to not do code reviews and to just push stuff.
Was slurfish fun?
Looks like if you don't like doing deep and thorough code reviews, LLM-generated code is not for you.
As the author concludes, "...LLMs are a tool. They let you generate a lot of code really fast...it is up to you to review it"
>Was slurfish fun?
Yes.
>Was slurfish fun?
It was for someone!
I am sure someone somewhere works on making LLMs commit code. Aside from that it was great witnessing the site in action and reading the postmortem. I wonder how the "hacker" made the connection to the user acount on neopets.com but maybe they just tried something like "ahallak"?
> I vibe coded...
Well, there ya go.
To be fair, it says the attention was unexpected, and this was just a coding exercise... And the port-mortem shows what I'd hope to see: digging in and figuring out root causes. So I'm not judging OP poorly over this.
But still. Launching a vibe-coded app that accepts input from anonymous users is just asking for trouble. I'm frankly surprised it ran as long as it did without such problems. (Although I did see a few weenies swimming around even before the problem hours.)
The lesson I'd pull from this is that if you are not the type of dev who could put together a post-mortem along these lines... don't launch a vibe-coded app.
Ultimately an app like this caused more joy than harm. I'm all for people vibe coding fun little things like this when the stakes are low. Would prefer to see more non coders with diverse ideas feeling empowered to start a project rather than them seeing a huge wall to climb and never starting. The web needs more silly apps from silly people.
So long and thanks for all the fish.
When i tried it, it almost blew my cpu.
this is why we cant have anything nice.
nice one, great hobby project! great postmortem!
Do you ship stickers to Canada?
I was proud of my ability to have an extremely penis looking fish get past filters. I feel like when presented with censorship my instinct is to always test its limits
I read the HN thread first, and the first comment I saw from the author was about AI nazi symbol detector they put in... I wonder how many orders of magnitude that comment increased interest in making offensive fish?
Later I saw images of the attacked site posted elsewhere and thought they were both predictable and hilarious.
A better group psychological approach might have been to only name the penis filter, ye. Then the transgressions are on the vandal who can't rationalize it is sticking it to the man, but that he is just that guy drawing swastikas.