Content
@
0 reply
0 recast
0 reaction
Roko
@rokomijic
A big problem in decentralized AI is we don't actually want to create a world in which you can compute any function you like in a censorship resistant way. Being able to compute any function you like opens up the grand payoff matrix of the world to lots of lose-lose equilibria.
1 reply
1 recast
0 reaction
Roko
@rokomijic
What we actually want is something like an automated "operating system" for reality and a partition of the physical resources of the universe between all agents. Within your patch of the world, you get to do whatever you want so long as it doesn't harm others.
1 reply
0 recast
0 reaction
Roko
@rokomijic
The ability to compute arbitrary functions opens up lots of harmful options, and some of them may even be game-theoretically dominant. Imagine computing a function whose output is a virus which will kill every other human being on the planet, and also give you an army of obedient robots...
1 reply
0 recast
0 reaction
Roko
@rokomijic
The mere ability to compute this function and implement its output is very destabilizing. Because even if you don't really want to do it, just knowing that others have the option to do so makes you want to do it for instrumental reasons - namely as a sort of "defensive preemptive attack"
1 reply
0 recast
0 reaction
Roko
@rokomijic
This is obviously a cartoonish example. Modern AI isn't powerful enough to do this. But it can do other things that are potentially destructive and destabilizing, and over time it will be able to do more and more bad things.
1 reply
0 recast
0 reaction
