Almost all LLM training datasets include copyrighted content so almost all open source LLM distribution is piracy and almost all API based LLMs, including ChatGPT, are also piracy and copyright laundering.
Also, most image-text dataset pairs contain far worse than that. You might want to check out LAION-5B and what stanford researchers have found in there. Technically, anyone who even touched that could in theory be in some serious, serious trouble. I find it quite remarkable that nothing has happened yet.
> almost all open source LLM distribution is piracy and almost all API based LLMs, including ChatGPT, are also piracy and copyright laundering
That's an amplification of copyright, original expression is protected, but not the ideas themselves, those are free. And don't forget when we actually get to use these models we feed them questions, data, we give corrections - so they are not simply replicating the training set, they learn and do new things with new inputs.
In fact if you think deeply about it, it is silly to accuse AI of copyright violation. Copying the actual book or article is much much faster and cheaper, and exact. Why would I pay a LLM provider to generate it for me from the title and starting phrase? If I already have part of the article, do I still need to generate it with AI? it's silly. LLM regurgitation are basically attacks with special key, entrapments. They don't happen in normal use.
Models are not information archives. The size of the final model is orders of magnitude smaller than the size of the training data.
Somehow people are just not able to get this through their heads. Stable diffusion is like 12GB or something and you have people convinced it's a tool that is cutting and pasting copyrighted works from an enormous image archive.
The courts (in the US) have not found LLM model weights to be piracy, nor the outputs, but it’s really surprising that LAION was used for so long consider the content you allude to.
There exists databases of “the hash of problematic photos” (CSAM), so it seems trivial to search your billions of photos against them before training an AI. You can’t catch everything, but this seems like an obvious miss considering the explicitly tried to scrape pornography.
These hashes is exactly how researchers later discovered this content, so it’s clearly not hard.
The Stanford researchers also found a substantial number of CSAM images in the LAION-5B dataset which were not recognized by PhotoDNA, probably because the images in question were not in wide distribution prior to their inclusion in LAION.
You are uploading 5 billion examples of <something>. You cannot filter it manually, of course, because there are five billion of it. Given that it is the year 2024, how hard is it to be positive that a well-resourced team at Stanford in 2029 will not have better methods of identifying and filtering your data, or a better reference dataset to filter it against, than you do presently?
You don’t have to do it manually. There is a database of file hashes.
And this isn’t just “one engineer”. Companies like StabilityAI, Google, etc have used LAION datasets. If you built a dataset you should expend some resources on automated filtering. Don’t include explicit imagery as an intentional choice if you can’t do basic filtering.
It'll be some epic lawsuit like google-v-samsung that will get drawn out for a decade, awarded, and reduced, appealed, etc. where the only winners will be both party's lawyers.
- OpenAI and others will just settle with MPAA, RIAA and the likes for a revenue stream (a single digit billion a year, likely) + some kind of control over what people can and cannot do with the AI + the access to the technology to produce their own content.
- artists will see peanuts from the deal, and the big names are going to be able to stop doing any kind of business with artists which are just expenses in their eyes. They will have been replaced by machines that where trained using their art with no compensation whatsoever.
IP is already predatory capitalism, AI will definitely be weaponized against the workers by the owners of the means of “production”.
Also, most image-text dataset pairs contain far worse than that. You might want to check out LAION-5B and what stanford researchers have found in there. Technically, anyone who even touched that could in theory be in some serious, serious trouble. I find it quite remarkable that nothing has happened yet.