I think these sort of efforts are mostly self-soothing at this point. It is almost certainly the case that the labs are at a minimum running inference over the information they're pulling and ensuring that it's useful/suitable for pre-training. The models are at least good enough to know whether they're looking at utter nonsense.
Actually it was shown a couple of times already, some of it also by Anthropic's own research, that the LLMs are extremely easy to poison with small datasets.
Ya I feel like these AI companies have the ability to be somewhat selective about their training sets. They don't have to add everything. I guess the idea is the filters wouldn't catch it, but if the junk is indistinguishable from the real stuff, then won't the platforms just be ruined by a bunch of junk?
I doubt things like this work against any serious Ai lab. They know data curation is paramount. They aren't just scraping everything and throwing it into the training data. You don't need to train on all of the internet, that actually hurts.
Poison Fountain: https://news.ycombinator.com/item?id=46577464
Poison Fountain on Reddit: https://www.reddit.com/r/PoisonFountain/
Miasma Poison Tar Pit: https://news.ycombinator.com/item?id=47561819
I think these sort of efforts are mostly self-soothing at this point. It is almost certainly the case that the labs are at a minimum running inference over the information they're pulling and ensuring that it's useful/suitable for pre-training. The models are at least good enough to know whether they're looking at utter nonsense.
Actually it was shown a couple of times already, some of it also by Anthropic's own research, that the LLMs are extremely easy to poison with small datasets.
Ya I feel like these AI companies have the ability to be somewhat selective about their training sets. They don't have to add everything. I guess the idea is the filters wouldn't catch it, but if the junk is indistinguishable from the real stuff, then won't the platforms just be ruined by a bunch of junk?
Finally an AI project with a sense of purpose!
Good name too ...
I doubt things like this work against any serious Ai lab. They know data curation is paramount. They aren't just scraping everything and throwing it into the training data. You don't need to train on all of the internet, that actually hurts.
I fully support this effort.