In you're in favor of "AI safety" (broad definition), what's your most compelling cast-length argument?

The technology is obviously epochal, but its path still largely unknown. 

It’s appropriate to have at least *some* humility toward “known unknowns” of a historical magnitude.

building /bracket | talking /greatbooks | ex-BCG, ex-PHD | poli + phil + econ

A high level of care was present in the development of nuclear bombs. Eg, intense secrecy and multiple layers of safegaurding before possible deployments

This isn't "apples to apples", ofc, bc nukes were "known" danger, whereas AI is "possible" danger

But some amount of this humility in makers feels right here, too

But nukes were not "radically accelerated". We tightly controlled who could have them, implemented safeguards, etc, and this – along with MAAD – has led to a very beneficial equilibrium.

Isn't it possible to believe ^some safety can be good while also believing safety can go too far? (eg, banning nuclear energy)

But nukes moved forward and nothing bad happened between nuclear powers? Instead we got a moral panic about nuclear energy and stagnated?