Design and software at Apple. Also into AI, philosophy of mind, ethics.

I'm down for defaulting to treating AI like it's conscious.

Historically, we've made way more mistakes by defaulting to dehumanization.

The only problem is externality pricing.

Building https://negationgame.com

Suffice it to say IMO it’s likelier that assuming consciousness in AI will lead to more mistakes (and more harm) than not.

what if the universe is isomorphic to a turing machine and you are isomorphic to a program?

A program can only simulate phenomena through a Turing machine, which can itself be implemented in any physics (including an elaborate set of water pipes or beer cans). So just as a program simulating an explosion is not itself an explosion, a program simulating consciousness is not itself conscious.

So you're defining consciousness as biological? Couldn't you do the same thing for flight? "Flight is something that birds do so you can't have a machine do it." Ok that's fine, what do you want to call it when a machine does the thing which we call flight when it's a bird? What about the thing we call consciousness?

I can try. Consciousness names a biological process like digestion or photosynthesis. Anything seeking to produce consciousness would have to duplicate its causal mechanisms, which are poorly understood. An artificial heart needs to pump blood. A flying machine needs to operate with drag and lift.

Consequences for assuming consciousness: 1) deception and manipulation of humans by AIs claiming to be in pain, 2) moral prohibition against running simulations of humans (research ethics), 3) moral imperative to generate maximum number of happy AIs if you’re a utilitarian.