This kind of behavior should be met with swift and strong reaction from both the school authorities and the law enforcement. Kids responsible should be put on trial and face serious consequences including jail time for most egregious cases. Let the news cover it nationwide. "This is what will happen if you even think about creating and circulating nude pics of anyone" should be the message sent. This is the only way to prevent this behavior - not more censorship, not technical solutions
That's what we need. More children in prison. Now two lives are ruined. Just for the sake of retribution.
Redeemable children can make horrific mistakes. The job of parents, and pastoral adults is to maintain an environment where kids can exist without making irreparable, life-destroying mistakes. Let them learn, but limit the damage they can accidentally cajole themselves into doing.
That means limiting access to social media, phones and cameras. Turning the school clock back to 1995.
Immature kids - and kids are by definition immature - make dumb decisions.
Certainly there needs to be repercussions, the kids need to learn, and need to be warned in advance, but kids will do dumb things.
You can talk about news coverage, but you can't force 12 year olds to watch the news, or understand how their actions have consequences. When the consequences do come, the focus needs to be primary on rehabilitation and restitution for the first instance.
It used to be that if you wanted to see people naked you’d just use your imagination. If you really wanted you’d draw or use photoshop, but both actually required skill. AI made it too easy. Teenagers wanting to see each other naked is normal and part of the human experience.
I don’t agree with the nanny state approach. America is already too obsessed with violence being ok and any kind of sex/nudity being completely evil. I don’t have an actual answer but having a moral panic over it is not the solution.
let me parse what you're saying: you're saying there are no issues with male teenagers generating deepfakes of their female friends. You're saying that's is " normal and part of the human experience."
I wonder if this kind of problem happens a lot less in societies that treat public nudity as normal (e.g. nude saunas, topless beaches in European/Scandinavian countries).
Worth considering that this isn’t “any kind”, it’s a very specific kind. It’s nudity of minors they personally know, shared in their community. It’s on a different level than, say, consensual nudity from a stranger in a Playboy magazine.
Back in my father's day you'd get executed for drawing a dick on Stalin's forehead. It's nice to see people still care more about symbolic violence than actual violence.
Nevertheless there is almost a moral equivocation between images of sex and actual sex by some people. We are talking about restraining someone physically inside a 2 by 3 meter box for multiple years as a retribution for what is after all just pixels on a screen.
(And by the way the implicit comparison here between an image of stalin and an image of a teenage girl is more apt than most would care to admit given the ideological function fulfilled by sexualized images of young women in contemporary American society)
Yeah, 'pixel on a screen'. I will send deepfake video of your daughter (or sister, or mother) having sex to you, let's see how they appreciate it. It's only pixels after all. Or picture of you having sex to your friends, SO and parents, that's probably fine.
I don't have sex in front of my friends and family for a reason, and I would appreciate my privacy protected by the state. And yes, privacy breach of this magnitude is probably worth 2-3 years (which basically means nothing for the first offense, let's be realistic, but makes the second offense way more consequencial).
Isn't it true? You are reacting to me with a fervor that lends its intensity from the moral values around actual physical sex with actual minors, but nothing of that kind occured here.
Yet the consequences for the victims may last a lifetime and be both intrinsic and extrinsic. Sexually forcing themselves on a drunk colleague at a party or taking their dad’s gun and shooting up their classmates also only takes a few minutes, but can scar a community for years.
I’m not going to opine on what the punishment should be, though it seems clear there should be one and more than a slap on the wrist. The community where this took place (i.e. those who are living through it) should probably have a say in it. There should also probably be an investigation of how this came to pass, i.e. has this kid always been an asshole with a pattern of behaviour or have they recently been indoctrinated by manosphere bullshit and should get counselling to understand their actions and that women are people too and deserving of respect?
I don't know how society will function when it's as easy to fake a video or image as it is to draw a stick figure. Hopefully it at least means that people view what happened here as akin to scribbling boobs on someone's photo, but we're not there yet apparently.
Perhaps there is reason to be optimistic. If we believe nothing online, then digital inauthenticity may cause us to yearn more for authentic, in-person interactions.
The modern web has not been without its warts. It's led society to some very unhealthy behaviors. It's telling that a lot of people think the internet is society, rather than a slice of it. Perhaps its decline is the correction society needed.
It's easy to feel like a doomer if you're thinking "it's sad that the web is being torn apart" and your framing of the web is "the magical place of promise it used to be 25 years ago."
Funny, I've been thinking in the same lines as of lately. The speed-up of the lost of trust (and interest) in everything online will make us care more for the real world around us. Perhaps we will see a big shift in values.
But then again, perhaps we won't and things will just get worse.
The older generation (gen-x) and before has been trained to believe traditional media (and by extension, anything that is published on a "website" and/or forwarded on Whatsapp). Images, pictures etc. add to the authenticity. Of course, it's easier to forward than to investigate and refute.
The younger generation is less influenced by this but also care less about generating and forwarding these. Most of these things are "just a joke". In a weird way, pushing out non-consensual nude of someone is the same as editing someone's face in a group photo with dog ears and sending it to a circle of friends.
I don't know what the solution is but this kind of thing erodes trust. Photographs used to be evidence that can be used to establish trust and we've been culturally conditioned to accept them that way. This takes a sledgehammer to that and it's not easy to untrain a whole society away from a deeply conditioned feeling so easily.
> I don't know what the solution is but this kind of thing erodes trust. Photographs used to be evidence that can be used to establish trust and we've been culturally conditioned to accept them that way
I'm just waiting (with some dread) for the day that you can sync up multiple cameras with AI to fake from different angles.
I've been leaning to more analog (and "old") stuff these days. Not sure if it's because I'm tired of all the crap online, or its a coincidence of getting older and missing the old times. Maybe a bit of both. But I really enjoy sitting in my living room and do nothing except listen to my record and cd player.
It's nice to have analog/real world hobbies to reset. My kids and I have picked up whittling. A few blocks of wood, a sharp knife, some sheets of sandpaper and time just slows down. Many others too. It's nice to be completely offline for an extended period of time.
Yeah, that's true, they are digital, but still feel "old". I was quite amazed while playing a vinyl record, accidentally lowered the volume down to 0, and could still literally hear a bit of sound coming out, not from the speakers, from the needle touching the record. Then I remembered that it's the actual sound waves that are engraved in the record. Amazing stuff
I was blown away when I made a crystal radio and could hear (barely) the station with no power source but the radio signal itself! Digital broadcast makes this impossible, alas.
Except you can actually “scan” a digital photo with a Polaroid quite effectively and it will look pretty much identical to a real Polaroid, so even that is security theatre.
true! I'm talking about watching someone click the shutter and hand me the actual photo. Though who knows what could happen with in-camera processing or a cloud-connected camera.
Your comment sounds like you're suggesting that those Epstein files photos are fake. I don't know why you'd want to suggest that unless you're trying to make excuses for paedophile rapists and traffickers and discredit the various people whose lives have been ruined by Epstein and his pals.
In the context of the comment they’re replying to, I think they’re saying the opposite: That the photos were released when they could plausibly imply they were fake, thus complying with the mandate while also giving their base the ammunition to discrediting them despite them being factual.
Hopefully we’ll get back to the time back when people didn’t have to worry about someone filming them all the time.
Cameras cryptographically signing all photos and videos should become the standard. Media that cannot be traced back to real camera hardware should a priori be considered fake or tampered with.
Still, deepfakes are partly a social problem and, as should be well known, social problems can rarely be solved by technological solutions.
This is another example of treating normal behaviour as criminal. Like banning VPNs or age verification.
How about we undo the radical individualism that has rotted out any compassion and empathy from society in the pursuit of greed? Not as easy to do granted, but it's a social technical debt that can't be quick fixed.
I'm not sure how deepfakes are an example of "compassion and empathy rotted out from society in the pursuit of greed". They're just your standard issue human condition, if it's possible to grief a person then there's always someone willing to do just that. Doesn't mean we have to suffer that.
In a compassionate society with community mindedness people are less likely to use tools like this for mindless hurt. Our current culture isnt aligned with compassion due to individualism and greed.
You sound like you've given up on nurturing humanity and instead want to enforce good behaviour as if the base human condition is to hurt others by default.
These are often young boys behaving in ways that are misogynistic and heartless. Usually following the examples of heartless parents and peers. We can do better. They're kids learning the wrongs lessons about the world.
Nothing is ever foolproof, but also, public-key crypto signatures are as secure as can be. Sure, you might be able to eventually extract a private key from camera hardware, but that's on the level of intelligence agencies, completely beyond the current situation of deepfakes being possible to make by anyone with an internet connection and no skills.
I think you meant to say: a single eastern European teenager who gets bored one weekend. These systems have been around for decades, and they've continually been hacked for decades too.
it functioned just fine for the past 50 years. digital manipulation of media had always been possible. americans simply didn't have a society-wide psychosis during the early days of personal computing and Internet.
It took some skill and time before to make anything passable as realistic. Enough skill and time to defeat a teenagers motivation. That's the difference.
"make the person in this photo naked. No mistakes"
Prompts lowered the bar. Even if you do have to install a local text to image model.
very soon no one will takes these things seriously, And That's A Good Thing.
there are, for example, websites that host (among a myriad other such things) style-accurate pictures of every Disney princess being fucked in every way imaginable. 70 years ago something like this might have caused a similar moral panic, but now no one (rightfully) gives a shit. you see a picture of Cinderella with a dick up her ass, you know it wasn't drawn by Disney.
Bizarrely, Cinderella probably has better protection/rights than real people do. Disney could easily assign a fleet of lawyers to protect their intellectual property and get websites taken down.
Even though they fluffed the messaging I think the school was right, in the absence of any actual
deepfakes and even after the police department had scraped all the boys phones, to be wary of accusing anyone in particular of doing anything AI related. Kids are weird and one boy saying he “dropped $250” on making deepfakes of “that hoe” sounds as much like a brag about money than it could be about making fake videos. There’s clearly enough of a text message trail between these boys to give them a significant reprimand for misogyny.
The PD’s PR team changing their narrative seems to lack competence. The school board focusing on being light touch on a specific case instead of cracking down in general seems like another error of judgment. This whole article says more about the effectiveness and competency of cops and school boards as it does about generative imagery.
Reading comments demanding harsh punishments or preaching morality in response to these kinds of incidents always leaves me feeling profoundly grim.
Speaking as a man, never once in my entire life have I consciously willed myself to feel a sexual impulse, thinking, "Alright, I am going to get aroused now." It is always an involuntary release of hormones or whatever; all I can do is hope that my frontal lobe manages the impulse properly so I don't end up doing something society punishes. Is it different for other people? Do people frame this as a failure of willpower because they are somehow generating their sexual urges voluntarily?
Aren't sex offenders, fundamentally, individuals who struggle to take appropriate action because their impulses are overwhelmingly stronger, or their executive function is lower, compared to "normal" people? Much like gambling addiction or obesity. What they really need isn't harsh punishment or moral grandstanding, but rather medical and social support.
The same applies to teenagers. We arbitrarily categorize them as "children" based purely on a number, but physical and neurological development varies wildly from person to person. Confining them in enclosed spaces with attractive peers for hours on end, and then completely derailing the lives of those whose biological makeup couldn't resist the resulting impulses—leading to criminalized behavior—feels like an incredibly cruel system.
Perhaps if enough generations pass, individuals with these genetic predispositions will be weeded out and the number of victims will decrease. But surely we can implement more humane and effective measures? If society essentially prohibits them from passing on their genes, shouldn't there at least be some form of compensation or alternative support?
They chose to do make content about one of their fellow classmates they knew would be harmful and shared it with others. It doesn't matter how aroused they were. If you're aroused, there's petabytes of porn literally all over the Internet.
I have kids. A significant part of raising them is showing/teaching them that feelings are natural, but only some outlets for those feelings are acceptable. My kids are younger than high school.
Society is built around not destroying everything when you're angry. Hopefully that is easier when you're adult, but it's certainly the norm to expect it from children as well.
What would you do if your child suffered brain damage from an illness or injury and lost the ability to regulate their emotions? And what if the severity of that impairment was right on the borderline—just enough to be penalized by society, but not enough to qualify for any support?
I used to be an instructor at a karate dojo, and several parents brought their kids to me, asking me to "do something" about their violent behavior. One of the children had been diagnosed with autism, but the rest hadn't even been evaluated by a doctor. The parents likely expected me to discipline them through fear and (possibly) corporal punishment, but I chose to instruct them relying solely on words until the very end.
However, it was futile. When I taught them that they must not hurt others or lash out, they would reply, "I understand." But the reality was that they had merely learned to say "I understand" whenever spoken to by an intimidating opponent they knew they couldn't beat.
Ultimately, every single case ended in disappointment. Consumed by a profound sense of powerlessness with the last child, I gave up teaching kids altogether.
You and your children are simply fortunate to possess the cognitive capacity to understand that making deepfakes is wrong.
> What would you do if your child suffered brain damage from an illness or injury and lost the ability to regulate their emotions? And what if the severity of that impairment was right on the borderline—just enough to be penalized by society, but not enough to qualify for any support?
I would approve of depriving them of the opportunity to hurt others, but keep loving them and supporting them for as long as I live.
The alternative would be covering for them while other kids suffer the consequences. I love my kids over everything else, but I would fail them and hopefully they would feel the same way if I put their enjoyment/freedoms over other peoples suffering. I wish society veered more towards balancing with responsibility and not simply maximizing individual freedoms. We need both.
> You and your children are simply fortunate to possess the cognitive capacity to understand that making deepfakes is wrong.
Possibly, maybe probably, but I've only met one kid like that yet, and his parents who I've interacted with many times definitely inflicted that behavior. But I can't know what I don't know, so I'll defer to your experience.
> Throughout the night, whenever one of the speakers brought up holding Big Tech accountable, and specifically Apple and Google’s app stores, the crowd applauded.
The framing bothers me. It's shifting responsibility away from kids and the community. Deepfakes didn't do anything: kids made and used deepfakes in destructive ways.
Big tech sucks, you get no argument from me there. But, how do people so quickly get over the fact that a kid is sexually harrassing someone?
This is like when a mass shooting happens and everyone only talks about guns. Well, yeah, if you didn't have guns a mass shooting couldn't happen, but isn't the underlying issue that someone decided that hurting a bunch of people is the best thing to do?
The more people dislike a crime, the less likely they are to talk about the perpetrator's inner life, but, come on, these are kids. How do people get so side-tracked?
This has nothing to do with “deepfakes”. These girls could have been attacked by aliens and be received with more credulity. There has to be a point where you understand that this behavior towards girls and women is simply being allowed.
One of the mothers says this quite explicitly:
> Calling the event rumors and speculation when a crime occurred was almost worse than the crime
The article intentionally obfuscates the difference between images depicting sexual abuse that did not happen, and actual sexual abuse. Ironically this makes it harder to address the problem on the level where the actual abuse happens i.e. bullying, lack of empathy, teenage body image issues in the properly fucked up american high school environment.
For example here the word "abuse" technically refers to the making and spreading of these images and attendant bullying, but obviously has connotations of actual sexual abuse:
>Despite all this, Radnor’s administration failed students in the days and weeks after it learned about the abuse,
This is maybe subtle but later the article gets more explicit about it:
>this technology [...] bears little difference from the non-consensual intimate imagery that’s plagued young girls and teenagers since the invention of the camera.
(i.e. a straightforward equation of images of nudity that did not happen, and images produced by actually pointing a camera at someone who is physically present with their nude body)
>Because the images aren’t “real,” authorities grapple with how to handle them.
Putting "real" in scare-quotes, very nice.
Dorfman is also quoted doing some nice sleight of hand, going in one sentence from "It was about the creation of the videos" to "and then share them with others as if the girls were something to be passed around." - i.e. equating the making of videos depicting unreal events with something that almost sounds like gang rape.
Anyway enough examples, article is full of them. I'm not saying there's no problem here, just that these people seem to have some kind of ideology of the sanctity of the image, i.e. a realistic image always depicts real events, and should always be interpreted as if it does. If it does not, clearly that's a problem to be corrected by reinforcing the connection between images and reality by making it impossible to produce realistic images of non-real things. It strikes me as exactly the wrong way to go about it, that can only make the bullying and harrassment (the actual problem which should be solved on the social level) worse by making its ammunition stronger.
(Slight aside but I do think the closing quote by Woelfel makes a good point)
This is sadly becoming very common and they are worringly good enough for people to assume they are real. I had a friend who had this issue, she has tried to ride it off as no clue who made them or shared them but I know it can't be easy for her. There needs to be appropriate laws for this the same way there is for revenge porn etc.
Didn't get past the paywall but, recently a friend had their primary school targeted by blackmailers who took public photos of children and made (I'm told) extremely explicit images with them, threatening to release them if not paid.
What's the defense? Intelligent screening of incoming messages so that the threat never reaches the blackmail target? I imagine they'll find an unprotected channel.
Don't post innocuous images of children ever? Seems like losing.
Don't post innocuous images of children ever? Seems like losing.
I do have to say that I find it disturbing how liberally parents seem to post pictures of their children in public though. Respect the privacy of your children, they’re not your pets.
I'm more thinking, the winning school sports team, a choir performing, etc. Things that would be posted in normal institutional participation and give children positive affirmation and recognition
Certainly in my lifetime we never posted such things publicly across the globe, neither for myself nor my peers, nor for my children and most of their peers at school.
Such things were printed and handed out and rarely made it past that years parents and students and the school archive (physical visit required).
Some schools will (try to) do this, but under European privacy and portrait right laws, they have to get explicit permissions from parents first.
I think a school news letter / yearbook is cute, but it should be private and only for the students and parents. A physical version would be best, you can put those in the archive instead of realize 20 years later that the digital versions are gone.
It's nice to see pictures of people. I like when parents share photos of my kids at events where they're having a great time but I couldn't make it. It's nice when the local paper shows a picture of happy graduating 8th graders.
Saying "you shouldn't do this thing that was basically harmless from the birth of photography until the ~2010's" is ignoring a lot of history and context.
Broadcasting private photos of children to strangers and corporations was common until 2010s? Were you posting them on classified section on newspapers in 1999?
> What's the defense? Intelligent screening of incoming messages so that the threat never reaches the blackmail target? I imagine they'll find an unprotected channel.
Same defenses that are used against fraud and other crime.
criminal prosecution of the blackmailers AND the services used to generate the pictures.
This is effectively child porn... so penalties would be pretty harsh.
There are extradition treaties to most of the world, so unless the blackmailers are in China/Russia they will end up in jail.
That same thing played out with piracy with people extradited to the US from various countries
I just looked it up and according to Wikipedia distributing child porn is a crime in both, possession is a crime in China. Blackmail is a crime in both too. So even if they do not extradite the blackmailers they are likely to face jail under the their own laws.
Although I agree this is necessary, I don't think it is sufficient.
Hackers and blackmailers on the internet can be anywhere from trivial to impossible to identify.
Even harder to prosecute when they're in a different jurisdiction, as not everywhere has extradition treaties to everywhere else. I wouldn't be overly surprised if e.g. the North Korean government runs some schemes like this (though not necessarily actually this) to bring in money.
But yes, definitely do the easy things first, like trying to stop e.g. grok from doing it and also going after users of e.g. grok who try to get around such efforts. A lot more crime happens than can be prosecuted, so raising the minimum competence threshold to commit crimes in the first place is very necessary.
Sure, but the reality is that if it happens abroad there is no legal recourse. Laws are not global, but the internet is. The only real option there is to close the internet.
Or to elect me as god-emperor and unify the world under my rule.
I'd already realised by about 2010 that the internet was making a mockery of national sovereignty; I absolutely expect nations to demand some way to effectively close the borders of the internet without breaking everything as it presently would.
That is propagation of CSAM. We already have way of dealing with people transmitting CSAM, that is to prosecute them heavily.
You won't prevent people stabbing others with knife by banning knife or asking people to wear knife-proof vest going outside. You deter them by making everybody know that the consequence of harming somebody else is going to be a very unpleasant experience.
Why a school, in this day and age, is posting pictures of children is beyond me. At my kid’s school we refused to sign the photo release clause, and any time they might be taking pictures they had to email us to ask if it was OK if our child was included. We agreed to it for the yearbook, but refused everything else.
Prosecuting every one of those blackmailers. If this kind of crime starts going into the "you're gonna get caught and jailed 100%" category, less people will try it.
And I think it would be an interesting question of discovery to determine if the models were or weren't trained on CSAM. (Diffusors were trained on both pelicans and bicycles, you know...)
I still think there's a lot of legal and IP landmines lying in wait from the Hoover-esque pre-training eras
That is a clear-cut case of extortion. The defence is having the police do their job and apprehend the criminals in question. If you screen incoming messages and stop them from reaching their target, extortionists will switch to publicly releasing one risque image with threats of releasing more explicit ones. There is always an unprotected channel as you noted.
As an aside, please do not use the b-mail word. It is insensitive towards BIPOCs.
Yes, but statistics are very clear here [0]. Speaking as a dad of both, we have tried to raise our kids to treat everyone as a human first, but there are very obviously ways for boys to fall into different behaviors than girls, both from culture (probably mainly) but also from testosterone when older. Girls can absolutely be cruel in other ways, though, and of course incidents diverging from statistical patterns, as with most things.
I think sharing these images is sexual harassment and it's pretty clearly wrong and deserves punishment. Other than that, I don't think it should ever be possible to commit a crime by providing inputs to an air-gapped "clean" computer and then view the output. But it seems like that's not the case.
This kind of behavior should be met with swift and strong reaction from both the school authorities and the law enforcement. Kids responsible should be put on trial and face serious consequences including jail time for most egregious cases. Let the news cover it nationwide. "This is what will happen if you even think about creating and circulating nude pics of anyone" should be the message sent. This is the only way to prevent this behavior - not more censorship, not technical solutions
That's what we need. More children in prison. Now two lives are ruined. Just for the sake of retribution.
Redeemable children can make horrific mistakes. The job of parents, and pastoral adults is to maintain an environment where kids can exist without making irreparable, life-destroying mistakes. Let them learn, but limit the damage they can accidentally cajole themselves into doing.
That means limiting access to social media, phones and cameras. Turning the school clock back to 1995.
Immature kids - and kids are by definition immature - make dumb decisions.
Certainly there needs to be repercussions, the kids need to learn, and need to be warned in advance, but kids will do dumb things.
You can talk about news coverage, but you can't force 12 year olds to watch the news, or understand how their actions have consequences. When the consequences do come, the focus needs to be primary on rehabilitation and restitution for the first instance.
It used to be that if you wanted to see people naked you’d just use your imagination. If you really wanted you’d draw or use photoshop, but both actually required skill. AI made it too easy. Teenagers wanting to see each other naked is normal and part of the human experience.
I don’t agree with the nanny state approach. America is already too obsessed with violence being ok and any kind of sex/nudity being completely evil. I don’t have an actual answer but having a moral panic over it is not the solution.
let me parse what you're saying: you're saying there are no issues with male teenagers generating deepfakes of their female friends. You're saying that's is " normal and part of the human experience."
Am I getting this wrong?
To me the nudity isn't an issue, it's the bullying. Then again I'm not American.
I wonder if this kind of problem happens a lot less in societies that treat public nudity as normal (e.g. nude saunas, topless beaches in European/Scandinavian countries).
If you think European kids aren't making deep fakes of other kids...
Surprise.
> any kind of sex/nudity
Worth considering that this isn’t “any kind”, it’s a very specific kind. It’s nudity of minors they personally know, shared in their community. It’s on a different level than, say, consensual nudity from a stranger in a Playboy magazine.
Back in my father's day you'd get executed for drawing a dick on Stalin's forehead. It's nice to see people still care more about symbolic violence than actual violence.
These are teenage girls, not adult men. This isn’t graffiti, but highly realistic sexual depictions.
There’s a massive difference.
Nevertheless there is almost a moral equivocation between images of sex and actual sex by some people. We are talking about restraining someone physically inside a 2 by 3 meter box for multiple years as a retribution for what is after all just pixels on a screen.
(And by the way the implicit comparison here between an image of stalin and an image of a teenage girl is more apt than most would care to admit given the ideological function fulfilled by sexualized images of young women in contemporary American society)
Yeah, 'pixel on a screen'. I will send deepfake video of your daughter (or sister, or mother) having sex to you, let's see how they appreciate it. It's only pixels after all. Or picture of you having sex to your friends, SO and parents, that's probably fine.
I don't have sex in front of my friends and family for a reason, and I would appreciate my privacy protected by the state. And yes, privacy breach of this magnitude is probably worth 2-3 years (which basically means nothing for the first offense, let's be realistic, but makes the second offense way more consequencial).
Go ahead.
It's not like I can stop you from imagining it.
Adding a GPU to the mix does make it any worse given that you already thought about it.
Not sure why you are being downvoted would a normal reaction to someone just saying it is a "pixel on a screen"
“Just pixels on a screen”
What is wrong with you?
Isn't it true? You are reacting to me with a fervor that lends its intensity from the moral values around actual physical sex with actual minors, but nothing of that kind occured here.
To you, it might be pixels on a screen, to someone else it's their life and dignity.
It wouldn't be the first time a high-schooler killed themself over bullying.
And to a good Stalinist it was the dignity of the USSR.
Something millions died to protect in the great patriotic war.
Again, symbolic violence is worse than real violence to a lot of people.
[dead]
I do agree that shit behavour should be met with direct reaction but lets be honest here the time and effort of doing this is 5 minutes.
A proper society should educate people about things they can't get rid of anymore.
And no you can't just put teenager, which have not fully formed brains yet, into prison for fake nudity
I agree here. People are too quick to levy adult punishment and morality against children.
Teach them compassion and empathy. Teach them why their actions hurt.
There are a lot more things we punish people for that take less than 5 minutes.
> the time and effort of doing this is 5 minutes.
Yet the consequences for the victims may last a lifetime and be both intrinsic and extrinsic. Sexually forcing themselves on a drunk colleague at a party or taking their dad’s gun and shooting up their classmates also only takes a few minutes, but can scar a community for years.
I’m not going to opine on what the punishment should be, though it seems clear there should be one and more than a slap on the wrist. The community where this took place (i.e. those who are living through it) should probably have a say in it. There should also probably be an investigation of how this came to pass, i.e. has this kid always been an asshole with a pattern of behaviour or have they recently been indoctrinated by manosphere bullshit and should get counselling to understand their actions and that women are people too and deserving of respect?
I don't know how society will function when it's as easy to fake a video or image as it is to draw a stick figure. Hopefully it at least means that people view what happened here as akin to scribbling boobs on someone's photo, but we're not there yet apparently.
Perhaps there is reason to be optimistic. If we believe nothing online, then digital inauthenticity may cause us to yearn more for authentic, in-person interactions.
The modern web has not been without its warts. It's led society to some very unhealthy behaviors. It's telling that a lot of people think the internet is society, rather than a slice of it. Perhaps its decline is the correction society needed.
It's easy to feel like a doomer if you're thinking "it's sad that the web is being torn apart" and your framing of the web is "the magical place of promise it used to be 25 years ago."
Funny, I've been thinking in the same lines as of lately. The speed-up of the lost of trust (and interest) in everything online will make us care more for the real world around us. Perhaps we will see a big shift in values.
But then again, perhaps we won't and things will just get worse.
I think there are generational aspects to this.
The older generation (gen-x) and before has been trained to believe traditional media (and by extension, anything that is published on a "website" and/or forwarded on Whatsapp). Images, pictures etc. add to the authenticity. Of course, it's easier to forward than to investigate and refute.
The younger generation is less influenced by this but also care less about generating and forwarding these. Most of these things are "just a joke". In a weird way, pushing out non-consensual nude of someone is the same as editing someone's face in a group photo with dog ears and sending it to a circle of friends.
I don't know what the solution is but this kind of thing erodes trust. Photographs used to be evidence that can be used to establish trust and we've been culturally conditioned to accept them that way. This takes a sledgehammer to that and it's not easy to untrain a whole society away from a deeply conditioned feeling so easily.
> I don't know what the solution is but this kind of thing erodes trust. Photographs used to be evidence that can be used to establish trust and we've been culturally conditioned to accept them that way
I'm just waiting (with some dread) for the day that you can sync up multiple cameras with AI to fake from different angles.
Suddenly a polaroid feels better than any digital photo.
I've been leaning to more analog (and "old") stuff these days. Not sure if it's because I'm tired of all the crap online, or its a coincidence of getting older and missing the old times. Maybe a bit of both. But I really enjoy sitting in my living room and do nothing except listen to my record and cd player.
It's nice to have analog/real world hobbies to reset. My kids and I have picked up whittling. A few blocks of wood, a sharp knife, some sheets of sandpaper and time just slows down. Many others too. It's nice to be completely offline for an extended period of time.
I feel a little like a rebel every time I pay with cash.
Funny that cd's are comparatively analog. (I get your meaning though).
Yeah, that's true, they are digital, but still feel "old". I was quite amazed while playing a vinyl record, accidentally lowered the volume down to 0, and could still literally hear a bit of sound coming out, not from the speakers, from the needle touching the record. Then I remembered that it's the actual sound waves that are engraved in the record. Amazing stuff
I was blown away when I made a crystal radio and could hear (barely) the station with no power source but the radio signal itself! Digital broadcast makes this impossible, alas.
Except you can actually “scan” a digital photo with a Polaroid quite effectively and it will look pretty much identical to a real Polaroid, so even that is security theatre.
true! I'm talking about watching someone click the shutter and hand me the actual photo. Though who knows what could happen with in-camera processing or a cloud-connected camera.
A major problem with a culture of distrust is that bad actors doing bad things get impunity by telling everyone that what they did was fake.
It’s already been tried, when deepfakes weren’t even that good.
https://www.theguardian.com/technology/2023/apr/27/elon-musk...
Certainly. Funny that we start getting photos from the Epstein files as soon as it's easy to make fakes.
Your comment sounds like you're suggesting that those Epstein files photos are fake. I don't know why you'd want to suggest that unless you're trying to make excuses for paedophile rapists and traffickers and discredit the various people whose lives have been ruined by Epstein and his pals.
In the context of the comment they’re replying to, I think they’re saying the opposite: That the photos were released when they could plausibly imply they were fake, thus complying with the mandate while also giving their base the ammunition to discrediting them despite them being factual.
https://en.wikipedia.org/wiki/The_Right_to_Read
You will have no compute and you will like it. The top comment here is about arresting the children for distributing child porn.
We really should have kept the normies out.
Hopefully we’ll get back to the time back when people didn’t have to worry about someone filming them all the time.
Cameras cryptographically signing all photos and videos should become the standard. Media that cannot be traced back to real camera hardware should a priori be considered fake or tampered with.
Still, deepfakes are partly a social problem and, as should be well known, social problems can rarely be solved by technological solutions.
This is another example of treating normal behaviour as criminal. Like banning VPNs or age verification.
How about we undo the radical individualism that has rotted out any compassion and empathy from society in the pursuit of greed? Not as easy to do granted, but it's a social technical debt that can't be quick fixed.
I'm not sure how deepfakes are an example of "compassion and empathy rotted out from society in the pursuit of greed". They're just your standard issue human condition, if it's possible to grief a person then there's always someone willing to do just that. Doesn't mean we have to suffer that.
In a compassionate society with community mindedness people are less likely to use tools like this for mindless hurt. Our current culture isnt aligned with compassion due to individualism and greed.
You sound like you've given up on nurturing humanity and instead want to enforce good behaviour as if the base human condition is to hurt others by default.
These are often young boys behaving in ways that are misogynistic and heartless. Usually following the examples of heartless parents and peers. We can do better. They're kids learning the wrongs lessons about the world.
I can't imagine that governments will end their zeal for surveillance.
That won't work, a camera can also photograph another screen or a fake printed photo. And also cryptographic modules can be cracked.
Nothing is ever foolproof, but also, public-key crypto signatures are as secure as can be. Sure, you might be able to eventually extract a private key from camera hardware, but that's on the level of intelligence agencies, completely beyond the current situation of deepfakes being possible to make by anyone with an internet connection and no skills.
> that's on the level of intelligence agencies
I think you meant to say: a single eastern European teenager who gets bored one weekend. These systems have been around for decades, and they've continually been hacked for decades too.
I would have thought most iPhones absolutely know if they're photographing a scene or a flat surface
Yes, iPhone has some clever depth technologies, but that doesn't mean it can't be tampered with.
We need a way to prove what's real.
Just put your phone down.
it functioned just fine for the past 50 years. digital manipulation of media had always been possible. americans simply didn't have a society-wide psychosis during the early days of personal computing and Internet.
It took some skill and time before to make anything passable as realistic. Enough skill and time to defeat a teenagers motivation. That's the difference.
"make the person in this photo naked. No mistakes"
Prompts lowered the bar. Even if you do have to install a local text to image model.
very soon no one will takes these things seriously, And That's A Good Thing.
there are, for example, websites that host (among a myriad other such things) style-accurate pictures of every Disney princess being fucked in every way imaginable. 70 years ago something like this might have caused a similar moral panic, but now no one (rightfully) gives a shit. you see a picture of Cinderella with a dick up her ass, you know it wasn't drawn by Disney.
Cinderella isn't going to complain because she's not a real person. The problem here is making and distributing fakes of specific real people.
Bizarrely, Cinderella probably has better protection/rights than real people do. Disney could easily assign a fleet of lawyers to protect their intellectual property and get websites taken down.
A fictional character isn't a real teenage girl who feels her life is falling apart because idiots make fake nudes of her to giggle at.
Theyre not the same thing and you can't just hand wave away people's pain. You're part of the problem.
cool.
[dead]
https://archive.is/neN8H
Even though they fluffed the messaging I think the school was right, in the absence of any actual deepfakes and even after the police department had scraped all the boys phones, to be wary of accusing anyone in particular of doing anything AI related. Kids are weird and one boy saying he “dropped $250” on making deepfakes of “that hoe” sounds as much like a brag about money than it could be about making fake videos. There’s clearly enough of a text message trail between these boys to give them a significant reprimand for misogyny.
The PD’s PR team changing their narrative seems to lack competence. The school board focusing on being light touch on a specific case instead of cracking down in general seems like another error of judgment. This whole article says more about the effectiveness and competency of cops and school boards as it does about generative imagery.
(translated by LLM)
Reading comments demanding harsh punishments or preaching morality in response to these kinds of incidents always leaves me feeling profoundly grim.
Speaking as a man, never once in my entire life have I consciously willed myself to feel a sexual impulse, thinking, "Alright, I am going to get aroused now." It is always an involuntary release of hormones or whatever; all I can do is hope that my frontal lobe manages the impulse properly so I don't end up doing something society punishes. Is it different for other people? Do people frame this as a failure of willpower because they are somehow generating their sexual urges voluntarily?
Aren't sex offenders, fundamentally, individuals who struggle to take appropriate action because their impulses are overwhelmingly stronger, or their executive function is lower, compared to "normal" people? Much like gambling addiction or obesity. What they really need isn't harsh punishment or moral grandstanding, but rather medical and social support.
The same applies to teenagers. We arbitrarily categorize them as "children" based purely on a number, but physical and neurological development varies wildly from person to person. Confining them in enclosed spaces with attractive peers for hours on end, and then completely derailing the lives of those whose biological makeup couldn't resist the resulting impulses—leading to criminalized behavior—feels like an incredibly cruel system.
Perhaps if enough generations pass, individuals with these genetic predispositions will be weeded out and the number of victims will decrease. But surely we can implement more humane and effective measures? If society essentially prohibits them from passing on their genes, shouldn't there at least be some form of compensation or alternative support?
They chose to do make content about one of their fellow classmates they knew would be harmful and shared it with others. It doesn't matter how aroused they were. If you're aroused, there's petabytes of porn literally all over the Internet.
I have kids. A significant part of raising them is showing/teaching them that feelings are natural, but only some outlets for those feelings are acceptable. My kids are younger than high school.
Society is built around not destroying everything when you're angry. Hopefully that is easier when you're adult, but it's certainly the norm to expect it from children as well.
What would you do if your child suffered brain damage from an illness or injury and lost the ability to regulate their emotions? And what if the severity of that impairment was right on the borderline—just enough to be penalized by society, but not enough to qualify for any support?
I used to be an instructor at a karate dojo, and several parents brought their kids to me, asking me to "do something" about their violent behavior. One of the children had been diagnosed with autism, but the rest hadn't even been evaluated by a doctor. The parents likely expected me to discipline them through fear and (possibly) corporal punishment, but I chose to instruct them relying solely on words until the very end.
However, it was futile. When I taught them that they must not hurt others or lash out, they would reply, "I understand." But the reality was that they had merely learned to say "I understand" whenever spoken to by an intimidating opponent they knew they couldn't beat.
Ultimately, every single case ended in disappointment. Consumed by a profound sense of powerlessness with the last child, I gave up teaching kids altogether.
You and your children are simply fortunate to possess the cognitive capacity to understand that making deepfakes is wrong.
> What would you do if your child suffered brain damage from an illness or injury and lost the ability to regulate their emotions? And what if the severity of that impairment was right on the borderline—just enough to be penalized by society, but not enough to qualify for any support?
I would approve of depriving them of the opportunity to hurt others, but keep loving them and supporting them for as long as I live.
The alternative would be covering for them while other kids suffer the consequences. I love my kids over everything else, but I would fail them and hopefully they would feel the same way if I put their enjoyment/freedoms over other peoples suffering. I wish society veered more towards balancing with responsibility and not simply maximizing individual freedoms. We need both.
> You and your children are simply fortunate to possess the cognitive capacity to understand that making deepfakes is wrong.
Possibly, maybe probably, but I've only met one kid like that yet, and his parents who I've interacted with many times definitely inflicted that behavior. But I can't know what I don't know, so I'll defer to your experience.
https://archive.is/neN8H
> Throughout the night, whenever one of the speakers brought up holding Big Tech accountable, and specifically Apple and Google’s app stores, the crowd applauded.
The framing bothers me. It's shifting responsibility away from kids and the community. Deepfakes didn't do anything: kids made and used deepfakes in destructive ways.
Big tech sucks, you get no argument from me there. But, how do people so quickly get over the fact that a kid is sexually harrassing someone?
This is like when a mass shooting happens and everyone only talks about guns. Well, yeah, if you didn't have guns a mass shooting couldn't happen, but isn't the underlying issue that someone decided that hurting a bunch of people is the best thing to do?
The more people dislike a crime, the less likely they are to talk about the perpetrator's inner life, but, come on, these are kids. How do people get so side-tracked?
That's crazy talk. You actually want the village to raise the children and have everyone well integrated and healthy?
You know, that is crazy. Disregard what I said. Down with big tech!
This has nothing to do with “deepfakes”. These girls could have been attacked by aliens and be received with more credulity. There has to be a point where you understand that this behavior towards girls and women is simply being allowed.
One of the mothers says this quite explicitly:
> Calling the event rumors and speculation when a crime occurred was almost worse than the crime
The article intentionally obfuscates the difference between images depicting sexual abuse that did not happen, and actual sexual abuse. Ironically this makes it harder to address the problem on the level where the actual abuse happens i.e. bullying, lack of empathy, teenage body image issues in the properly fucked up american high school environment.
For example here the word "abuse" technically refers to the making and spreading of these images and attendant bullying, but obviously has connotations of actual sexual abuse:
>Despite all this, Radnor’s administration failed students in the days and weeks after it learned about the abuse,
This is maybe subtle but later the article gets more explicit about it:
>this technology [...] bears little difference from the non-consensual intimate imagery that’s plagued young girls and teenagers since the invention of the camera.
(i.e. a straightforward equation of images of nudity that did not happen, and images produced by actually pointing a camera at someone who is physically present with their nude body)
>Because the images aren’t “real,” authorities grapple with how to handle them.
Putting "real" in scare-quotes, very nice.
Dorfman is also quoted doing some nice sleight of hand, going in one sentence from "It was about the creation of the videos" to "and then share them with others as if the girls were something to be passed around." - i.e. equating the making of videos depicting unreal events with something that almost sounds like gang rape.
Anyway enough examples, article is full of them. I'm not saying there's no problem here, just that these people seem to have some kind of ideology of the sanctity of the image, i.e. a realistic image always depicts real events, and should always be interpreted as if it does. If it does not, clearly that's a problem to be corrected by reinforcing the connection between images and reality by making it impossible to produce realistic images of non-real things. It strikes me as exactly the wrong way to go about it, that can only make the bullying and harrassment (the actual problem which should be solved on the social level) worse by making its ammunition stronger.
(Slight aside but I do think the closing quote by Woelfel makes a good point)
This is sadly becoming very common and they are worringly good enough for people to assume they are real. I had a friend who had this issue, she has tried to ride it off as no clue who made them or shared them but I know it can't be easy for her. There needs to be appropriate laws for this the same way there is for revenge porn etc.
Surely this comes under defamation law or something similar, as if they photoshopped the girl's heads onto porn photos.
Didn't get past the paywall but, recently a friend had their primary school targeted by blackmailers who took public photos of children and made (I'm told) extremely explicit images with them, threatening to release them if not paid.
What's the defense? Intelligent screening of incoming messages so that the threat never reaches the blackmail target? I imagine they'll find an unprotected channel.
Don't post innocuous images of children ever? Seems like losing.
Don't post innocuous images of children ever? Seems like losing.
I do have to say that I find it disturbing how liberally parents seem to post pictures of their children in public though. Respect the privacy of your children, they’re not your pets.
I'm more thinking, the winning school sports team, a choir performing, etc. Things that would be posted in normal institutional participation and give children positive affirmation and recognition
Certainly in my lifetime we never posted such things publicly across the globe, neither for myself nor my peers, nor for my children and most of their peers at school.
Such things were printed and handed out and rarely made it past that years parents and students and the school archive (physical visit required).
Getting in the newspaper or on TV for winning an award or competition or something has been around for a long time.
For the very rare few, yes, a fair while.
For the Dux's and general high performers, less time than I've been alive.
For entire classrooms to be posted up via live streams, Insta, tok's, etc ... barely a decade or so.
It's not something that was always normal, and just because it's become "normalised" doesn't mean it must remain that way.
Some schools will (try to) do this, but under European privacy and portrait right laws, they have to get explicit permissions from parents first.
I think a school news letter / yearbook is cute, but it should be private and only for the students and parents. A physical version would be best, you can put those in the archive instead of realize 20 years later that the digital versions are gone.
It's nice to see pictures of people. I like when parents share photos of my kids at events where they're having a great time but I couldn't make it. It's nice when the local paper shows a picture of happy graduating 8th graders.
Saying "you shouldn't do this thing that was basically harmless from the birth of photography until the ~2010's" is ignoring a lot of history and context.
Broadcasting private photos of children to strangers and corporations was common until 2010s? Were you posting them on classified section on newspapers in 1999?
> What's the defense? Intelligent screening of incoming messages so that the threat never reaches the blackmail target? I imagine they'll find an unprotected channel.
Same defenses that are used against fraud and other crime.
criminal prosecution of the blackmailers AND the services used to generate the pictures.
This is effectively child porn... so penalties would be pretty harsh.
There are extradition treaties to most of the world, so unless the blackmailers are in China/Russia they will end up in jail.
That same thing played out with piracy with people extradited to the US from various countries
I just looked it up and according to Wikipedia distributing child porn is a crime in both, possession is a crime in China. Blackmail is a crime in both too. So even if they do not extradite the blackmailers they are likely to face jail under the their own laws.
> What's the defense?
Prosecution of AI operators for making indecent images of children?
Although I agree this is necessary, I don't think it is sufficient.
Hackers and blackmailers on the internet can be anywhere from trivial to impossible to identify.
Even harder to prosecute when they're in a different jurisdiction, as not everywhere has extradition treaties to everywhere else. I wouldn't be overly surprised if e.g. the North Korean government runs some schemes like this (though not necessarily actually this) to bring in money.
But yes, definitely do the easy things first, like trying to stop e.g. grok from doing it and also going after users of e.g. grok who try to get around such efforts. A lot more crime happens than can be prosecuted, so raising the minimum competence threshold to commit crimes in the first place is very necessary.
Sure, but the reality is that if it happens abroad there is no legal recourse. Laws are not global, but the internet is. The only real option there is to close the internet.
Or to elect me as god-emperor and unify the world under my rule.
I'd already realised by about 2010 that the internet was making a mockery of national sovereignty; I absolutely expect nations to demand some way to effectively close the borders of the internet without breaking everything as it presently would.
That is propagation of CSAM. We already have way of dealing with people transmitting CSAM, that is to prosecute them heavily.
You won't prevent people stabbing others with knife by banning knife or asking people to wear knife-proof vest going outside. You deter them by making everybody know that the consequence of harming somebody else is going to be a very unpleasant experience.
Why a school, in this day and age, is posting pictures of children is beyond me. At my kid’s school we refused to sign the photo release clause, and any time they might be taking pictures they had to email us to ask if it was OK if our child was included. We agreed to it for the yearbook, but refused everything else.
Not caring.
> What's the defense?
Prosecuting every one of those blackmailers. If this kind of crime starts going into the "you're gonna get caught and jailed 100%" category, less people will try it.
What if the blackmailer is anonymous and abroad?
That's not a defence, that's a remedy.
In the US possession of CSAM is a strict-liability crime. If the models contain the data to produce it, the modelmakers should be liable.
Models don't have to contain imagery of a thing to produce imagery of that thing - the canonical example is the pelican riding a bicycle.
Model makers, arguably, are like pencil manufacturers but in a World of good artists.
I think you can hold people to account for what they help to create, but not what they have potential to do.
Now, if models were trained on any csam, of course, the model's owners should be held you account.
And I think it would be an interesting question of discovery to determine if the models were or weren't trained on CSAM. (Diffusors were trained on both pelicans and bicycles, you know...)
I still think there's a lot of legal and IP landmines lying in wait from the Hoover-esque pre-training eras
That is a clear-cut case of extortion. The defence is having the police do their job and apprehend the criminals in question. If you screen incoming messages and stop them from reaching their target, extortionists will switch to publicly releasing one risque image with threats of releasing more explicit ones. There is always an unprotected channel as you noted.
As an aside, please do not use the b-mail word. It is insensitive towards BIPOCs.
> As an aside, please do not use the b-mail word. It is insensitive towards BIPOCs.
No it isn't. If anything needs to stop its the ever escalating orwellian censorship of words that you're proposing.
> Don't post innocuous images of children ever?
Don’t raise boys who abuse girls. That’d be a big step in the right direction.
Because girls can never be abusive?
Yes, but statistics are very clear here [0]. Speaking as a dad of both, we have tried to raise our kids to treat everyone as a human first, but there are very obviously ways for boys to fall into different behaviors than girls, both from culture (probably mainly) but also from testosterone when older. Girls can absolutely be cruel in other ways, though, and of course incidents diverging from statistical patterns, as with most things.
[0] https://www.abs.gov.au/media-centre/media-releases/97-cent-s...
Well TFA exclusively discusses instances of boys abusing girls.
How did he not get expelled for that? Jesus.
I think sharing these images is sexual harassment and it's pretty clearly wrong and deserves punishment. Other than that, I don't think it should ever be possible to commit a crime by providing inputs to an air-gapped "clean" computer and then view the output. But it seems like that's not the case.