They really do believe AI might kill everyone. They also are highly motivated by hoarding capital and power. I don't know if researchers are aware that they're acting indistinguishably from someone who is just really self-interested and greedy but they are. Also the idea that there is this thing that's going to be apocalypse causing and completely inscrutable if built, but somebody will build it anyway so it might as well be us in the hopes that we get it right is schizophrenic thinking. It is a delusion of grandeur.
It might be possible that world changing singularity-esque AI superintelligence is soon upon us. There are a lot of fundamental physical and financial limitations that so-called rationalists ignore. But AI doesn't have to become a singularity causing entity that fundamentally alters the fabric of understanding of reality to be really harmful. It can be, and is today, harmful in much more mundane ways. One, it is a risky concentration of capital that if it fails could cause a recession and if it succeeds could just be a small group taking yet another larger piece of the economic pie. Two, it amplifies asymmetries that malicious actors have been pummeling us with recently. e.g. it makes hacking even easier. It makes flooding online social spaces with disinformation easier, faster, and for many more convincingly.
I think these reasons are why people do not take these statements seriously or if they do, do not treat this guy who quit with any respect. How self-interested do you have to be to say "oh I think my work is going to cause the end of days"? Even if you're not causing the apocalypse you're making people's lives worse on the whole but that's not what he noted as the reason he quit. And then finally, I do think the technology is impressive but its not a great sign when a couple years in the harms of this technology are here and obvious (and potentially could get much worse) while the positives are subjective, potential, and still not convincingly verifiable.
They really do believe AI might kill everyone. They also are highly motivated by hoarding capital and power. I don't know if researchers are aware that they're acting indistinguishably from someone who is just really self-interested and greedy but they are. Also the idea that there is this thing that's going to be apocalypse causing and completely inscrutable if built, but somebody will build it anyway so it might as well be us in the hopes that we get it right is schizophrenic thinking. It is a delusion of grandeur.
It might be possible that world changing singularity-esque AI superintelligence is soon upon us. There are a lot of fundamental physical and financial limitations that so-called rationalists ignore. But AI doesn't have to become a singularity causing entity that fundamentally alters the fabric of understanding of reality to be really harmful. It can be, and is today, harmful in much more mundane ways. One, it is a risky concentration of capital that if it fails could cause a recession and if it succeeds could just be a small group taking yet another larger piece of the economic pie. Two, it amplifies asymmetries that malicious actors have been pummeling us with recently. e.g. it makes hacking even easier. It makes flooding online social spaces with disinformation easier, faster, and for many more convincingly.
I think these reasons are why people do not take these statements seriously or if they do, do not treat this guy who quit with any respect. How self-interested do you have to be to say "oh I think my work is going to cause the end of days"? Even if you're not causing the apocalypse you're making people's lives worse on the whole but that's not what he noted as the reason he quit. And then finally, I do think the technology is impressive but its not a great sign when a couple years in the harms of this technology are here and obvious (and potentially could get much worse) while the positives are subjective, potential, and still not convincingly verifiable.