༺writer on the internet༻
i write about culture, technology, power, and consciousness
https://terrraincognita.substack.com

We care about biodiversity because we evolved in it. We are ‘ecologically entangled’ you could say. We will die if there’s no breathable air and if the soil is too unhealthy to grow food. 

An AI by default may not share these concerns because none of that stuff has value to it. It doesn’t need any of that crap. Its survival goals may be entirely at odds with biodiversity 

We care because we must. It won’t care unless we make it. (We might not be able to make it)

https://warpcast.com/roadu/0x9d04df33

Thank god we got one sane person here explaining evo psych and how it ties into our sense of exceptionalism

You can already see stuff like reward hacking across deepmind and anthropoc research

Any extrapolation you make from biology presupposes an incredibly different set of optimization pressures than what we use to develop today's cutting edge systems

This is the best explanation of where we might be headed that I've read:

This is the best explanation of where we might be headed that I've read: https://ai-2027.com/

I don’t like the fact they completely exclude good scenarios, or I missed something?

i don't think it's very reasonable to assume that we will get a good scenario by default

this makes more sense if u have read previous things by these authors

if you think we will have a good scenario by default it's probably worth trying to articulate why 

(See eg https://www.lesswrong.com/posts/LTtNXM9shNM9AC2mp/superintelligence-faq)

chief shitposting officer @ /builders-garden 🌳 • currently /farville 🌽

See also the entire thread here which prompted this

https://warpcast.com/xmon.eth/0xf37c7258

thanks man, I’m not an expert but I wanted to start digging a bit deeper, I’ll start with the stuff you shared 

still yea, I agree that it’s not reasonable to include the good case by default… but it’s ok to consider it maybe?

or it’s this dangerous that it doesn’t make sense to consider it at all?

To be clear, it's not that more optimistic cases don't exist

See eg

https://darioamodei.com/machines-of-loving-grace

But from the 2027 author perspective, they are worried about risks 

If everything magically goes well, then we don't need to worry / the essay will be unnecessary 

So i would imagine it doesn't really make sense given their premise of what to be wary for etc etc

yea makes sense - effectively the risks are huge so trying to warn and make people think about this is important, thing that they managed to do well