Serious question:

I don’t buy the “AI will kill us all” argument. 

If AI is so smart, wouldn’t it develop a sense of recognition and respect for other highly complex systems like the planet, billions of years of evolution, and recognize the value of human consciousness, emotion, and perception in contributing valuable information that it can’t gather itself?

Why do we assume AI will act as a selfish agent rather than a systems thinker?

Am I way off base? If AI is so smart, why wouldn’t this be the most likely outcome?

༺writer on the internet༻
i write about culture, technology, power, and consciousness
https://terrraincognita.substack.com

Agree. Also, if AI is so smart, what does it gain from exterminating humans? Even if it didn't respect humans, wouldn't it get more out of extracting some sort of value or resource from us than exterminating?

I don't think it's preferable, so I think we should invest a ton in AI-alignment to prevent that scenario.

At the same time, I think the argument which AI inevitably trends towards wanting to exterminate humans is pretty weak and doesn't add much to the discourse around responsible AI.

Okay but like why do you suppose such value extraction would be preferable for us versus eg subsistence level existence at best