༺writer on the internet༻
i write about culture, technology, power, and consciousness
https://terrraincognita.substack.com

We care about biodiversity because we evolved in it. We are ‘ecologically entangled’ you could say. We will die if there’s no breathable air and if the soil is too unhealthy to grow food. 

An AI by default may not share these concerns because none of that stuff has value to it. It doesn’t need any of that crap. Its survival goals may be entirely at odds with biodiversity 

We care because we must. It won’t care unless we make it. (We might not be able to make it)

https://warpcast.com/roadu/0x9d04df33

Thank god we got one sane person here explaining evo psych and how it ties into our sense of exceptionalism

You can already see stuff like reward hacking across deepmind and anthropoc research

Any extrapolation you make from biology presupposes an incredibly different set of optimization pressures than what we use to develop today's cutting edge systems

This is the best explanation of where we might be headed that I've read:

This is the best explanation of where we might be headed that I've read: https://ai-2027.com/

I don’t like the fact they completely exclude good scenarios, or I missed something?

i don't think it's very reasonable to assume that we will get a good scenario by default

this makes more sense if u have read previous things by these authors

if you think we will have a good scenario by default it's probably worth trying to articulate why 

(See eg https://www.lesswrong.com/posts/LTtNXM9shNM9AC2mp/superintelligence-faq)

chief shitposting officer @ /builders-garden 🌳 • currently /farville 🌽

I think they are really just articulating one scenario with a whole host of assumptions. It just happens to be the one I’ve read that feels most compelling. To be clear, I hope it’s wrong!