For her safety, Doe has opted to receive alerts from the US Department of Justice Victim Notification System any time she may be a victim in a new criminal investigation. Although she has received countless alerts, she was shocked when the CCCP notified her that it had identified AI-generated CSAM on xAI that depicted her. This re-traumatized Doe, whose complaint alleged that messages were found on online forums “between offenders chatting about creating AI generated CSAM of Plaintiff and other similarly situated known, legacy, victims of CSAM.”
Now, Doe fears that xAI has not only made it easier to make more violative images of the most distressing time in her life, but also that xAI allegedly has stored the images that Grok generates and uses those outputs to further train Grok. Because of this, she believes that Grok has been trained on both the initial set of images that have haunted her for more than 20 years and the more recent AI-generated ones.
This is the first case to accuse xAI of training on CSAM, and the complaint does not go into great detail on that claim. Previously, Ars reported on a controversial dataset that was later scrubbed after researchers found CSAM in the training data, but there’s no indication xAI trained on that data. In a press release from lawyers representing Doe, it explained that Doe’s images were included in a CSAM Hash List maintained by NCMEC, and “that same material” allegedly “was part of the dataset xAI used to build Grok’s image and video generating capabilities.” The complaint similarly only alleged that “CSAM depicting Plaintiff with its longstanding well-known hash values has been used as a part of the dataset used by xAI.”
This should make the entire model illegal. Train it on illegal stuff: delete the whole model.
In all seriousness, there are some very interesting legal questions that will be raised if this case makes it that far.
The problem is that there’s no existing law that would effect this on its own. To my knowledge, no country in the world has a law on the books specifically dealing with AI models trained on CSAM. So the question, under existing laws, would turn on whether the data stored within the model itself would constitute CSAM.
The problem, in no small part, is that we have serious gaps in our public consensus knowledge about how LLMs actually work.
There’s a case that, AFAIK, is still being argued in Germany pushing the theory that LLMs actually do, in effect, store a copy of all their training data, just in a compressed form. This certainly seems to hold some water given both the tests they relied on, and the situation with this Jane Doe where the model produced images so alike to real images of her that they tripped hash detections.
The German case argues that this is analogous to the difference between an MP3 and a WAV, or a JPEG and a PNG. That sharing a lossy copy of a work is no less infringing just because it’s imperfect.
If the underlying claim - that LLMs function as a form of lossy compression - can be substantiated then there would be a real argument that the model itself would constitute CSAM. Since there would be no realistic method that I’m aware of for removing the offending material from the model - and presumably SpaceX would have to somehow prove that they’ve done so - that would make the entire model contraband. They’d have to retrain on a clean dataset.
Of course I said “if the case makes it that far” at the top because I don’t think it will. SpaceX will do anything and everything to avoid handing over meaningful discovery in this case, including, I suspect, outright destruction of evidence. If there is anything that actually proves that they used CSAM in the training data then they are so far beyond fucked that there’s simply no downside to further illegality in pursuit of concealing their crimes. They have the world’s wealthiest asshole in a position to throw literal billions at making this go away. I genuinely wouldn’t be surprised if people turn up dead off the back of this if that’s what it takes.
including, I suspect, outright destruction of evidence
If it was trained on it, that means they are in possession of it, which that right there is straight to jail. I have a feeling they’re scrubbing everything they can right now as we chat
Must suck to be a rich pedophile and not even get an invite to Epstein island.
If it’s using csam images to produce new ones, is there any way to guarantee that any given nude it makes didn’t source csam? Is the whole thing poisoned at this point?
I mean, this is the argument for artist’s copywritten works. There is crossover here. The fights over AI are going to be endless.
To answer your question, yes it’s poisoned.
That’s why I don’t understand why anyone with money to invest in ai doesn’t have a lawyer or lawyers waiving red flags about these issues. It’s copyrighted materials being illegally used, output being machine derived and not able to be copyrighted, the use of csam for training models that output pornography, and on and on. Yet billions of dollars are dumped into what looks like a black market with no concern for how these issues will be settled.
Anything it makes is derived from the whole of its training data.
All billionaires are pedophiles
Backward comments like this are why the conversation doesn’t move.
Can you explain why you think it’s backwards? It may be hyperbolic and facetious, but I think recent history has shown a statistically significant correlation between the ultra-wealthy and pedophilia.
Correlation doesn’t imply causation does it?
I think that’s another generalization based on indeed totally true recent news. But he is right that this is not at all helping the reasonning, hence “backward”
I don’t think I was suggesting that being a billionaire causes you to be pedophile, or the other way around.
Do all statements that do not forward some sort of effort to reason through things count as backwards?
If all of them are pedophile then yes it means being a billionaire means you are pedophile. Hence causation either one way or another.
Now backward is a bit strong but I agree we should avoid these kind of comments that only make everyone more extrem compared to the more nuanced answered you gave about what we are currently seeing
