I've started looking at profiles of the non-technical "AI at any cost" people on Reddit and noticed a trend towards AI generated NSFW posts and anime. Unsurprisingly they tend to have zero tolerance towards accepting legitimate criticisms or concerns.
After reading about the blackmail a school system in the UK faced because criminals took a public post of 10 year old soccer players and generated sexual child abuse materials with the real girls faces it's in everyones best interests to not post their children's images until or unless we can regulate this behavior.
"By November 2025, IWF reports of AI-generated CSAM had more than doubled year over year, rising from 199 to 426. Girls accounted for 94% of the victims. Reported cases included children ranging from newborns to two-year-olds, according to the organization.
The ecosystem around these tools is industrial. In April 2025, a researcher found an exposed AWS S3 bucket belonging to South Korean “nudify” app GenNomis containing 93,485 AI-generated images alongside the prompts that produced them."
What more regulation needs to happen? Real or synthetic CSAM is already illegal. Adding another law that is effectively, “Same thing as the previous ones, but this time the generator is a LLM, not Photoshop”.
You'd have to have a model that has never seen children, which is perfectly fine, but I can't see any of those with the resources to train such a model form scratch actually doing so.
> Joi AI is an online platform that includes AI-generated avatars, voice interactions, and personalized chat experiences built around companionship and intimacy.
Companionship and intimacy, two things people desperately want and two things AI cannot deliver.
This stuff is deeply dystopian and I struggle to believe good faith on the part of people selling this stuff.
I disagree in part and I say that really sadly, but I genuinely think many people will be completely ok with having "AI friends" if let say it was absolutely or close to 1:1 for a human (which should be the case someday) as in good-looking humanoids, you might want to see how a very old person (>80 years old) behave when you put ElevenLabs in a flawless manner, it can be hours of convo daily without stopping, the "joy" they feel by this companion is quite real, at least it's felt.
We might even reach the point where having AI companions somehow will become essential and primary, even with an already fulfilled social life.
I've had "real friends" completely online, never met IRL, those were actual friends that I still talk to a decade later, and had countless evening of fun and random talks, in the end, if they happened to be AI (and let say deepfaked to the point you can't even know), I don't think it would have changed anything, it's still felt the same way, it's still helpful the same way, it's still a "bond".
I can't really make up my mind about it, but I'm certain we will have AI stuff behaving 1:1 with humans, including the graphics and flaws.
> but I genuinely think many people will be completely ok with having "AI friends" if let say it was absolutely or close to 1:1 for a human
I disagree, mostly because I don’t think AI friends will ever be “1:1 for a human”. Part of the reason people seek out AI friends is because they are sycophantic. You cannot experience friendship with entities that just tell you how great you are. And I don’t think AI friends that actually push back like human friends would be popular.
Users think otherwise. I agree it is dystopian, but to me it is the same bucket as sex work. There are people that won't be able to get a sex partner in a "normal" way. The only way for them is sex workers.
I assume the same is true here. A lot of people have real trouble to get romantic partner, and AI filla the void. It may look bad from the outside, but I don't think normal people can judge romantically deprived.
What would be even more dystopian than AI-partner society is not only to blame incels for each and any issue, but also to disallow them even right to use AI for romantic feelings.
I’m not blaming incels. I’m blaming the people building companies with the express goal of extracting money from them.
There is no moral imperative to allow predatory behavior. We can recognize the difficulty many experience with romantic relationships without blessing others to take advantage of them.
I also very much doubt that people struggling with romantic relationships actually show any measure of mental health improvement after engaging with these AI bots. I imagine the pattern is one of addiction, where participants feel better while actively using the product but then worse at baseline because none of their actual problems have improved.
That is a sort of a nightmare. On demand sex work for extremely low cost with 0 risk of disease...
We are slowly stripping everyone of the will to build meaningful lives by fulfilling all wants with soulless instant gratification. No perfection of AI/sex work or whatever can match actual intimacy with someone.
That is at the root of most of our current societal ills I believe. Bad economy, Bad job market, etc. are all a smokescreen over the fact that people can escape their problems. You can live a numb half-life and avoid any crisis that might trigger real change.
If a bad economic outlook was all it took to wipe out the birth rate we would never have survived the great depression or other 10,000 hard times we have endured as a species.
Having an actual romantic partner provides physical and emotional intimacy.
Hiring a sex worker provides physical intimacy and no emotional intimacy (unless you're paying for that as well, I suppose), but lonely people can project onto the SW and mistake physical intimacy for emotional intimacy. That can be unhealthy.
With this, there's no physical intimacy and a simulated level of barebones interaction, meaning that you're getting the physical release but no actual intimacy and none of the other stuff that comes with any sort of relationship, let alone a romantic one, let alone a healthy and rewarding romantic one.
There's a time and a place for this, but it's a digital sex toy and that's all. AI isn't filling the void of a romantic partner any more than eating a bacon double cheeseburger every day is filling the void of a healthy balanced diet. It keeps the body going, but it doesn't deliver what you and your body need to thrive.
(All of this outside of the context of asexual/aromantic individuals, who may or may not need or want or be comfortable with any level of any of this and for whom this might be the perfect product for all I know.)
Providing an AI "companion" to those who are lonely is kind of like selling drugs to addicts. Yes, it's what they are seeking, and yes, they are suffering. But what you're giving them causes them to suffer more, not less, and it's evil to profit by giving people something that is harming them even if they ask for it.
How exactly do you think it is harming them? (not teenagers and so-on), curious of your take. Sometimes treatments are useful even if you can't get out of it. AI companions will never disappear, that's sure, so it's not like giving something temporarily and then removing it.
I don't really use any form of AI as companion, but I do use it for introspection, I wouldn't say it's harmful, I think it depends on the depth of it.
I think that computers can't be a meaningful companion, because they aren't actually sentient. They can't truly sympathize with you, nudge you to grow when needed, and so on. Moreover, I think that treating a machine as a friend (or a lover) is going to stunt one's ability to connect with actual humans.
It's kind of like (though not exactly the same as) guys who watch crazy amounts of porn and jerk off all the time. Those guys have reported that when they are with a real woman, they sometimes can't even get hard (because a real person can't match the stimulation of the artificial thing designed to be as stimulating as possible), or can't have an orgasm (because a vagina isn't as tight as the grip they use when jerking it). In those cases, the outlet those guys were using to fill that need has ruined their ability to enjoy the healthier alternative. I believe that AI "companionship" is going to prove to have a similar effect.
You’re right, we should kick them while they are down instead, starving them of all “companionship” (otherwise, how could we uphold our current values as a society)? We need to keep producing a regular supply of [redacted] to rally people against in order to maintain the illusion of morale.
I didn't say that, any more than I said we should round up all drug addicts and shoot them. What I am saying is that giving a struggling person something which soothes his pain in the short term, while worsening his problem over the long term, is not compassionate treatment. It is preying on the less fortunate, and we should not tolerate it as a society.
I've started looking at profiles of the non-technical "AI at any cost" people on Reddit and noticed a trend towards AI generated NSFW posts and anime. Unsurprisingly they tend to have zero tolerance towards accepting legitimate criticisms or concerns. After reading about the blackmail a school system in the UK faced because criminals took a public post of 10 year old soccer players and generated sexual child abuse materials with the real girls faces it's in everyones best interests to not post their children's images until or unless we can regulate this behavior.
https://www.malwarebytes.com/blog/family-and-parenting/2026/...
"By November 2025, IWF reports of AI-generated CSAM had more than doubled year over year, rising from 199 to 426. Girls accounted for 94% of the victims. Reported cases included children ranging from newborns to two-year-olds, according to the organization.
The ecosystem around these tools is industrial. In April 2025, a researcher found an exposed AWS S3 bucket belonging to South Korean “nudify” app GenNomis containing 93,485 AI-generated images alongside the prompts that produced them."
What more regulation needs to happen? Real or synthetic CSAM is already illegal. Adding another law that is effectively, “Same thing as the previous ones, but this time the generator is a LLM, not Photoshop”.
They want the regulation to be that Ai models cannot produce such images at all, not that producing them is illegal
You'd have to have a model that has never seen children, which is perfectly fine, but I can't see any of those with the resources to train such a model form scratch actually doing so.
Wouldn’t help. You just ask for a short adult with slightly different proportions.
I am looking for signs we are at 'peak AI', is this it?
If it's not, I'm struggling to come up with what would be.
The Models are nearly Climaxed
The models are nearly Climaxed
Sounds like UBI to me
Just when you thought Juicero was the bottom of the silly valley barrel... a startup called Joi offers JOI by AI
> Joi AI is an online platform that includes AI-generated avatars, voice interactions, and personalized chat experiences built around companionship and intimacy.
Companionship and intimacy, two things people desperately want and two things AI cannot deliver.
This stuff is deeply dystopian and I struggle to believe good faith on the part of people selling this stuff.
I disagree in part and I say that really sadly, but I genuinely think many people will be completely ok with having "AI friends" if let say it was absolutely or close to 1:1 for a human (which should be the case someday) as in good-looking humanoids, you might want to see how a very old person (>80 years old) behave when you put ElevenLabs in a flawless manner, it can be hours of convo daily without stopping, the "joy" they feel by this companion is quite real, at least it's felt.
We might even reach the point where having AI companions somehow will become essential and primary, even with an already fulfilled social life.
I've had "real friends" completely online, never met IRL, those were actual friends that I still talk to a decade later, and had countless evening of fun and random talks, in the end, if they happened to be AI (and let say deepfaked to the point you can't even know), I don't think it would have changed anything, it's still felt the same way, it's still helpful the same way, it's still a "bond".
I can't really make up my mind about it, but I'm certain we will have AI stuff behaving 1:1 with humans, including the graphics and flaws.
> but I genuinely think many people will be completely ok with having "AI friends" if let say it was absolutely or close to 1:1 for a human
I disagree, mostly because I don’t think AI friends will ever be “1:1 for a human”. Part of the reason people seek out AI friends is because they are sycophantic. You cannot experience friendship with entities that just tell you how great you are. And I don’t think AI friends that actually push back like human friends would be popular.
Users think otherwise. I agree it is dystopian, but to me it is the same bucket as sex work. There are people that won't be able to get a sex partner in a "normal" way. The only way for them is sex workers.
I assume the same is true here. A lot of people have real trouble to get romantic partner, and AI filla the void. It may look bad from the outside, but I don't think normal people can judge romantically deprived.
What would be even more dystopian than AI-partner society is not only to blame incels for each and any issue, but also to disallow them even right to use AI for romantic feelings.
I’m not blaming incels. I’m blaming the people building companies with the express goal of extracting money from them.
There is no moral imperative to allow predatory behavior. We can recognize the difficulty many experience with romantic relationships without blessing others to take advantage of them.
I also very much doubt that people struggling with romantic relationships actually show any measure of mental health improvement after engaging with these AI bots. I imagine the pattern is one of addiction, where participants feel better while actively using the product but then worse at baseline because none of their actual problems have improved.
That is a sort of a nightmare. On demand sex work for extremely low cost with 0 risk of disease...
We are slowly stripping everyone of the will to build meaningful lives by fulfilling all wants with soulless instant gratification. No perfection of AI/sex work or whatever can match actual intimacy with someone.
That is at the root of most of our current societal ills I believe. Bad economy, Bad job market, etc. are all a smokescreen over the fact that people can escape their problems. You can live a numb half-life and avoid any crisis that might trigger real change.
If a bad economic outlook was all it took to wipe out the birth rate we would never have survived the great depression or other 10,000 hard times we have endured as a species.
I mean, this seems like another step further.
Having an actual romantic partner provides physical and emotional intimacy.
Hiring a sex worker provides physical intimacy and no emotional intimacy (unless you're paying for that as well, I suppose), but lonely people can project onto the SW and mistake physical intimacy for emotional intimacy. That can be unhealthy.
With this, there's no physical intimacy and a simulated level of barebones interaction, meaning that you're getting the physical release but no actual intimacy and none of the other stuff that comes with any sort of relationship, let alone a romantic one, let alone a healthy and rewarding romantic one.
There's a time and a place for this, but it's a digital sex toy and that's all. AI isn't filling the void of a romantic partner any more than eating a bacon double cheeseburger every day is filling the void of a healthy balanced diet. It keeps the body going, but it doesn't deliver what you and your body need to thrive.
(All of this outside of the context of asexual/aromantic individuals, who may or may not need or want or be comfortable with any level of any of this and for whom this might be the perfect product for all I know.)
Providing an AI "companion" to those who are lonely is kind of like selling drugs to addicts. Yes, it's what they are seeking, and yes, they are suffering. But what you're giving them causes them to suffer more, not less, and it's evil to profit by giving people something that is harming them even if they ask for it.
How exactly do you think it is harming them? (not teenagers and so-on), curious of your take. Sometimes treatments are useful even if you can't get out of it. AI companions will never disappear, that's sure, so it's not like giving something temporarily and then removing it.
I don't really use any form of AI as companion, but I do use it for introspection, I wouldn't say it's harmful, I think it depends on the depth of it.
I think that computers can't be a meaningful companion, because they aren't actually sentient. They can't truly sympathize with you, nudge you to grow when needed, and so on. Moreover, I think that treating a machine as a friend (or a lover) is going to stunt one's ability to connect with actual humans.
It's kind of like (though not exactly the same as) guys who watch crazy amounts of porn and jerk off all the time. Those guys have reported that when they are with a real woman, they sometimes can't even get hard (because a real person can't match the stimulation of the artificial thing designed to be as stimulating as possible), or can't have an orgasm (because a vagina isn't as tight as the grip they use when jerking it). In those cases, the outlet those guys were using to fill that need has ruined their ability to enjoy the healthier alternative. I believe that AI "companionship" is going to prove to have a similar effect.
You’re right, we should kick them while they are down instead, starving them of all “companionship” (otherwise, how could we uphold our current values as a society)? We need to keep producing a regular supply of [redacted] to rally people against in order to maintain the illusion of morale.
I didn't say that, any more than I said we should round up all drug addicts and shoot them. What I am saying is that giving a struggling person something which soothes his pain in the short term, while worsening his problem over the long term, is not compassionate treatment. It is preying on the less fortunate, and we should not tolerate it as a society.
This is another job H1Bs will do for half the price.