In you're in favor of "AI safety" (broad definition), what's your most compelling cast-length argument?

i’m in the all-gas-no-brakes camp but will do my best to steelman:

1. we’re on the cusp of agi 
2. agi is like nuclear: extinction-level threat, global superpower-level leverage, and potential civilization expanding resource
3. we dont really know how it works
4. thus, we should proceed with *extreme* caution

on point 2, there were no brakes for decades and everything worked out ok?

So the USG control will stop people in other countries from making progress?

only if by “no brakes” you mean complete u.s. government control

by that definition i’m pretty sure the the ai safety folks would agree with you

For decades we were not anywhere close to important thresholds for how much actual compute is available relative to how much is in our brains. Now we are.

except for a few incidents where we feared the extinction of human and biological existence, yeah

Enriching enough uranium for a bomb is a lot harder then gathering GPUs 
Also, people understood that; big bomb = bad 
People don’t seem to care/understand the risks of non-aligned AGI