I've started looking at profiles of the non-technical "AI at any cost" people on Reddit and noticed a trend towards AI generated NSFW posts and anime. Unsurprisingly they tend to have zero tolerance towards accepting legitimate criticisms or concerns.
After reading about the blackmail a school system in the UK faced because criminals took a public post of 10 year old soccer players and generated sexual child abuse materials with the real girls faces it's in everyones best interests to not post their children's images until or unless we can regulate this behavior.
"By November 2025, IWF reports of AI-generated CSAM had more than doubled year over year, rising from 199 to 426. Girls accounted for 94% of the victims. Reported cases included children ranging from newborns to two-year-olds, according to the organization.
The ecosystem around these tools is industrial. In April 2025, a researcher found an exposed AWS S3 bucket belonging to South Korean “nudify” app GenNomis containing 93,485 AI-generated images alongside the prompts that produced them."
What more regulation needs to happen? Real or synthetic CSAM is already illegal. Adding another law that is effectively, “Same thing as the previous ones, but this time the generator is a LLM, not Photoshop”.
You'd have to have a model that has never seen children, which is perfectly fine, but I can't see any of those with the resources to train such a model form scratch actually doing so.
Guys, it's always the victims fault. Grift Economy 101. No way to prevent this says Grift Economists for the 928401st time. Better write another check to big business.
> Joi AI is an online platform that includes AI-generated avatars, voice interactions, and personalized chat experiences built around companionship and intimacy.
Companionship and intimacy, two things people desperately want and two things AI cannot deliver.
This stuff is deeply dystopian and I struggle to believe good faith on the part of people selling this stuff.
I disagree in part and I say that really sadly, but I genuinely think many people will be completely ok with having "AI friends" if let say it was absolutely or close to 1:1 for a human (which should be the case someday) as in good-looking humanoids, you might want to see how a very old person (>80 years old) behave when you put ElevenLabs in a flawless manner, it can be hours of convo daily without stopping, the "joy" they feel by this companion is quite real, at least it's felt.
We might even reach the point where having AI companions somehow will become essential and primary, even with an already fulfilled social life.
I've had "real friends" completely online, never met IRL, those were actual friends that I still talk to a decade later, and had countless evening of fun and random talks, in the end, if they happened to be AI (and let say deepfaked to the point you can't even know), I don't think it would have changed anything, it's still felt the same way, it's still helpful the same way, it's still a "bond".
I can't really make up my mind about it, but I'm certain we will have AI stuff behaving 1:1 with humans, including the graphics and flaws.
> but I genuinely think many people will be completely ok with having "AI friends" if let say it was absolutely or close to 1:1 for a human
I disagree, mostly because I don’t think AI friends will ever be “1:1 for a human”. Part of the reason people seek out AI friends is because they are sycophantic. You cannot experience friendship with entities that just tell you how great you are. And I don’t think AI friends that actually push back like human friends would be popular.
Users think otherwise. I agree it is dystopian, but to me it is the same bucket as sex work. There are people that won't be able to get a sex partner in a "normal" way. The only way for them is sex workers.
I assume the same is true here. A lot of people have real trouble to get romantic partner, and AI filla the void. It may look bad from the outside, but I don't think normal people can judge romantically deprived.
What would be even more dystopian than AI-partner society is not only to blame incels for each and any issue, but also to disallow them even right to use AI for romantic feelings.
I’m not blaming incels. I’m blaming the people building companies with the express goal of extracting money from them.
There is no moral imperative to allow predatory behavior. We can recognize the difficulty many experience with romantic relationships without blessing others to take advantage of them.
I also very much doubt that people struggling with romantic relationships actually show any measure of mental health improvement after engaging with these AI bots. I imagine the pattern is one of addiction, where participants feel better while actively using the product but then worse at baseline because none of their actual problems have improved.
That is a sort of a nightmare. On demand sex work for extremely low cost with 0 risk of disease...
We are slowly stripping everyone of the will to build meaningful lives by fulfilling all wants with soulless instant gratification. No perfection of AI/sex work or whatever can match actual intimacy with someone.
That is at the root of most of our current societal ills I believe. Bad economy, Bad job market, etc. are all a smokescreen over the fact that people can escape their problems. You can live a numb half-life and avoid any crisis that might trigger real change.
If a bad economic outlook was all it took to wipe out the birth rate we would never have survived the great depression or other 10,000 hard times we have endured as a species.
> You can live a numb half-life and avoid any crisis that might trigger real change.
Changes go both ways and people on the bottom don't have a good track record regarding changes they experienced in their lives. Is it worth taking a risk for 1 that gets better while statistically 99 will get even worse?
Should we rely on letting crisis happen to improve thigs?
Having an actual romantic partner provides physical and emotional intimacy.
Hiring a sex worker provides physical intimacy and no emotional intimacy (unless you're paying for that as well, I suppose), but lonely people can project onto the SW and mistake physical intimacy for emotional intimacy. That can be unhealthy.
With this, there's no physical intimacy and a simulated level of barebones interaction, meaning that you're getting the physical release but no actual intimacy and none of the other stuff that comes with any sort of relationship, let alone a romantic one, let alone a healthy and rewarding romantic one.
There's a time and a place for this, but it's a digital sex toy and that's all. AI isn't filling the void of a romantic partner any more than eating a bacon double cheeseburger every day is filling the void of a healthy balanced diet. It keeps the body going, but it doesn't deliver what you and your body need to thrive.
(All of this outside of the context of asexual/aromantic individuals, who may or may not need or want or be comfortable with any level of any of this and for whom this might be the perfect product for all I know.)
I would agree with this if the AI was actually a reasonable partner. But the AIs are heavily slanted towards agreeing with the user and thus very prone to leading them down rabbit holes rather than keeping them out of rabbit holes.
And the problem with incels is that they have been lead down the rabbit hole of hate. Which is, unfortunately, quite understandable given how much disdain the romantically challenged get.
Providing an AI "companion" to those who are lonely is kind of like selling drugs to addicts. Yes, it's what they are seeking, and yes, they are suffering. But what you're giving them causes them to suffer more, not less, and it's evil to profit by giving people something that is harming them even if they ask for it.
How exactly do you think it is harming them? (not teenagers and so-on), curious of your take. Sometimes treatments are useful even if you can't get out of it. AI companions will never disappear, that's sure, so it's not like giving something temporarily and then removing it.
I don't really use any form of AI as companion, but I do use it for introspection, I wouldn't say it's harmful, I think it depends on the depth of it.
I think that computers can't be a meaningful companion, because they aren't actually sentient. They can't truly sympathize with you, nudge you to grow when needed, and so on. Moreover, I think that treating a machine as a friend (or a lover) is going to stunt one's ability to connect with actual humans.
It's kind of like (though not exactly the same as) guys who watch crazy amounts of porn and jerk off all the time. Those guys have reported that when they are with a real woman, they sometimes can't even get hard (because a real person can't match the stimulation of the artificial thing designed to be as stimulating as possible), or can't have an orgasm (because a vagina isn't as tight as the grip they use when jerking it). In those cases, the outlet those guys were using to fill that need has ruined their ability to enjoy the healthier alternative. I believe that AI "companionship" is going to prove to have a similar effect.
You’re right, we should kick them while they are down instead, starving them of all “companionship” (otherwise, how could we uphold our current values as a society)? We need to keep producing a regular supply of [redacted] to rally people against in order to maintain the illusion of morale.
I didn't say that, any more than I said we should round up all drug addicts and shoot them. What I am saying is that giving a struggling person something which soothes his pain in the short term, while worsening his problem over the long term, is not compassionate treatment. It is preying on the less fortunate, and we should not tolerate it as a society.
> Companionship and intimacy, two things people desperately want and two things AI cannot deliver.
For a lot of people no human can or is willing to deliver that either. So I'd say alternatives are worth exploring. We already exploit other species to satisfy some of those needs partially.
The set of people who can’t find companionship or intimacy is not nearly as large as the black pill community would lead one to believe. Certainly it’s smaller than the target market for these sorts of services.
Maybe for young people, but for older, increasingly significant percentage of people end up without companionship and intimacy. Even still being in formal relationships.
> This stuff is deeply dystopian and I struggle to believe good faith on the part of people selling this stuff.
I agree, but there's something I also kinda respect about saying the quiet part out loud.
Other AIs are executed just slyly enough for anyone with legitimate criticism to be given a hard time by a specious and silver-tongued communications departments, or a eager outside sycophant. Sure, $HEAD of $AI_VENTURE probably does has a genuine good faith idea or two about AI. They might not always be lying in the press. But the companies still act in dystopian bad faith.
I've started looking at profiles of the non-technical "AI at any cost" people on Reddit and noticed a trend towards AI generated NSFW posts and anime. Unsurprisingly they tend to have zero tolerance towards accepting legitimate criticisms or concerns. After reading about the blackmail a school system in the UK faced because criminals took a public post of 10 year old soccer players and generated sexual child abuse materials with the real girls faces it's in everyones best interests to not post their children's images until or unless we can regulate this behavior.
https://www.malwarebytes.com/blog/family-and-parenting/2026/...
"By November 2025, IWF reports of AI-generated CSAM had more than doubled year over year, rising from 199 to 426. Girls accounted for 94% of the victims. Reported cases included children ranging from newborns to two-year-olds, according to the organization.
The ecosystem around these tools is industrial. In April 2025, a researcher found an exposed AWS S3 bucket belonging to South Korean “nudify” app GenNomis containing 93,485 AI-generated images alongside the prompts that produced them."
What more regulation needs to happen? Real or synthetic CSAM is already illegal. Adding another law that is effectively, “Same thing as the previous ones, but this time the generator is a LLM, not Photoshop”.
They want the regulation to be that Ai models cannot produce such images at all, not that producing them is illegal
You'd have to have a model that has never seen children, which is perfectly fine, but I can't see any of those with the resources to train such a model form scratch actually doing so.
Wouldn’t help. You just ask for a short adult with slightly different proportions.
Guys, it's always the victims fault. Grift Economy 101. No way to prevent this says Grift Economists for the 928401st time. Better write another check to big business.
they're not wrong though
I am looking for signs we are at 'peak AI', is this it?
If it's not, I'm struggling to come up with what would be.
The models are nearly Climaxed
The Models are nearly Climaxed
Yes. CSAM [x]. Propaganda [x]. Useless, time consuming "help" [x].
Sounds like UBI to me
Just when you thought Juicero was the bottom of the silly valley barrel... a startup called Joi offers JOI by AI
Can I order a Monroebot?
This is another job H1Bs will do for half the price.
> Joi AI is an online platform that includes AI-generated avatars, voice interactions, and personalized chat experiences built around companionship and intimacy.
Companionship and intimacy, two things people desperately want and two things AI cannot deliver.
This stuff is deeply dystopian and I struggle to believe good faith on the part of people selling this stuff.
I disagree in part and I say that really sadly, but I genuinely think many people will be completely ok with having "AI friends" if let say it was absolutely or close to 1:1 for a human (which should be the case someday) as in good-looking humanoids, you might want to see how a very old person (>80 years old) behave when you put ElevenLabs in a flawless manner, it can be hours of convo daily without stopping, the "joy" they feel by this companion is quite real, at least it's felt.
We might even reach the point where having AI companions somehow will become essential and primary, even with an already fulfilled social life.
I've had "real friends" completely online, never met IRL, those were actual friends that I still talk to a decade later, and had countless evening of fun and random talks, in the end, if they happened to be AI (and let say deepfaked to the point you can't even know), I don't think it would have changed anything, it's still felt the same way, it's still helpful the same way, it's still a "bond".
I can't really make up my mind about it, but I'm certain we will have AI stuff behaving 1:1 with humans, including the graphics and flaws.
> but I genuinely think many people will be completely ok with having "AI friends" if let say it was absolutely or close to 1:1 for a human
I disagree, mostly because I don’t think AI friends will ever be “1:1 for a human”. Part of the reason people seek out AI friends is because they are sycophantic. You cannot experience friendship with entities that just tell you how great you are. And I don’t think AI friends that actually push back like human friends would be popular.
Sycophancy hasn't been that big of a problem in years.
And yes, some people can experience their version of friendship. You don't get to decide what's friendship for someone else.
Very bad, yes, I agree with that.
[dead]
Users think otherwise. I agree it is dystopian, but to me it is the same bucket as sex work. There are people that won't be able to get a sex partner in a "normal" way. The only way for them is sex workers.
I assume the same is true here. A lot of people have real trouble to get romantic partner, and AI filla the void. It may look bad from the outside, but I don't think normal people can judge romantically deprived.
What would be even more dystopian than AI-partner society is not only to blame incels for each and any issue, but also to disallow them even right to use AI for romantic feelings.
I’m not blaming incels. I’m blaming the people building companies with the express goal of extracting money from them.
There is no moral imperative to allow predatory behavior. We can recognize the difficulty many experience with romantic relationships without blessing others to take advantage of them.
I also very much doubt that people struggling with romantic relationships actually show any measure of mental health improvement after engaging with these AI bots. I imagine the pattern is one of addiction, where participants feel better while actively using the product but then worse at baseline because none of their actual problems have improved.
That is a sort of a nightmare. On demand sex work for extremely low cost with 0 risk of disease...
We are slowly stripping everyone of the will to build meaningful lives by fulfilling all wants with soulless instant gratification. No perfection of AI/sex work or whatever can match actual intimacy with someone.
That is at the root of most of our current societal ills I believe. Bad economy, Bad job market, etc. are all a smokescreen over the fact that people can escape their problems. You can live a numb half-life and avoid any crisis that might trigger real change.
If a bad economic outlook was all it took to wipe out the birth rate we would never have survived the great depression or other 10,000 hard times we have endured as a species.
> You can live a numb half-life and avoid any crisis that might trigger real change.
Changes go both ways and people on the bottom don't have a good track record regarding changes they experienced in their lives. Is it worth taking a risk for 1 that gets better while statistically 99 will get even worse?
Should we rely on letting crisis happen to improve thigs?
I mean, this seems like another step further.
Having an actual romantic partner provides physical and emotional intimacy.
Hiring a sex worker provides physical intimacy and no emotional intimacy (unless you're paying for that as well, I suppose), but lonely people can project onto the SW and mistake physical intimacy for emotional intimacy. That can be unhealthy.
With this, there's no physical intimacy and a simulated level of barebones interaction, meaning that you're getting the physical release but no actual intimacy and none of the other stuff that comes with any sort of relationship, let alone a romantic one, let alone a healthy and rewarding romantic one.
There's a time and a place for this, but it's a digital sex toy and that's all. AI isn't filling the void of a romantic partner any more than eating a bacon double cheeseburger every day is filling the void of a healthy balanced diet. It keeps the body going, but it doesn't deliver what you and your body need to thrive.
(All of this outside of the context of asexual/aromantic individuals, who may or may not need or want or be comfortable with any level of any of this and for whom this might be the perfect product for all I know.)
I would agree with this if the AI was actually a reasonable partner. But the AIs are heavily slanted towards agreeing with the user and thus very prone to leading them down rabbit holes rather than keeping them out of rabbit holes.
And the problem with incels is that they have been lead down the rabbit hole of hate. Which is, unfortunately, quite understandable given how much disdain the romantically challenged get.
Providing an AI "companion" to those who are lonely is kind of like selling drugs to addicts. Yes, it's what they are seeking, and yes, they are suffering. But what you're giving them causes them to suffer more, not less, and it's evil to profit by giving people something that is harming them even if they ask for it.
How exactly do you think it is harming them? (not teenagers and so-on), curious of your take. Sometimes treatments are useful even if you can't get out of it. AI companions will never disappear, that's sure, so it's not like giving something temporarily and then removing it.
I don't really use any form of AI as companion, but I do use it for introspection, I wouldn't say it's harmful, I think it depends on the depth of it.
I think that computers can't be a meaningful companion, because they aren't actually sentient. They can't truly sympathize with you, nudge you to grow when needed, and so on. Moreover, I think that treating a machine as a friend (or a lover) is going to stunt one's ability to connect with actual humans.
It's kind of like (though not exactly the same as) guys who watch crazy amounts of porn and jerk off all the time. Those guys have reported that when they are with a real woman, they sometimes can't even get hard (because a real person can't match the stimulation of the artificial thing designed to be as stimulating as possible), or can't have an orgasm (because a vagina isn't as tight as the grip they use when jerking it). In those cases, the outlet those guys were using to fill that need has ruined their ability to enjoy the healthier alternative. I believe that AI "companionship" is going to prove to have a similar effect.
You’re right, we should kick them while they are down instead, starving them of all “companionship” (otherwise, how could we uphold our current values as a society)? We need to keep producing a regular supply of [redacted] to rally people against in order to maintain the illusion of morale.
I didn't say that, any more than I said we should round up all drug addicts and shoot them. What I am saying is that giving a struggling person something which soothes his pain in the short term, while worsening his problem over the long term, is not compassionate treatment. It is preying on the less fortunate, and we should not tolerate it as a society.
Harm reduction programs where you give addicts narcotics safely have an amazing track record
> Companionship and intimacy, two things people desperately want and two things AI cannot deliver.
For a lot of people no human can or is willing to deliver that either. So I'd say alternatives are worth exploring. We already exploit other species to satisfy some of those needs partially.
The set of people who can’t find companionship or intimacy is not nearly as large as the black pill community would lead one to believe. Certainly it’s smaller than the target market for these sorts of services.
Maybe for young people, but for older, increasingly significant percentage of people end up without companionship and intimacy. Even still being in formal relationships.
I wonder what the age distribution of users of this sort of product looks like.
Probably somewhere between casual gamers and romance novels and movies enthusiasts. So I'd guess about 45 on average. Midlife crisis age.
> This stuff is deeply dystopian and I struggle to believe good faith on the part of people selling this stuff.
I agree, but there's something I also kinda respect about saying the quiet part out loud.
Other AIs are executed just slyly enough for anyone with legitimate criticism to be given a hard time by a specious and silver-tongued communications departments, or a eager outside sycophant. Sure, $HEAD of $AI_VENTURE probably does has a genuine good faith idea or two about AI. They might not always be lying in the press. But the companies still act in dystopian bad faith.
[dead]
[flagged]