You can communicate to your dog that you’re not happy about it peeing on the rug. You cannot communicate to your dog why you prefer functional programming.

This will be AGI’s relationship with us. It won’t be able to explain most of its concepts to us.

This proposition has huge consequences:

* There is knowledge beyond human capabilities

* An algorithm can learn that knowledge

Quite a hot take IMO

We don't know enough our own brains and conscience, how can we predict what will happen with AGI?

Computer Science and Software Developer. Getting serious fun with web3. Also passionate about anything else related to Comp Sci

AlphaGo defeated the best human players in the world using strategies that humans didn’t understand. 

Not saying OPs position will happen, but it certainly seems plausible.

I do assume the pattern of capability we see with the observed spectrum of intelligence of creatures continues beyond us. It would seem unlikely that our species would have the maximum potential capability. More likely it is a function of our hardware which is necessary and sufficient for our evolutionary niche.