Cassie Heart on Warpcast

polynya pfp

This reply is representative of many others. AGI will not be a single entity like a human or a simple bot on a single server, but rather a set of networked intelligence(s) across millions of machines around the world. We can already see this property emerging in today's LLMs - GPT-4 was trained across 25,000 GPUs, inferencing over 128 GPUs. Side-note: blockchains are not immutable or permanent, as can be seen with Ethereum blobs that are deleted every 18 days; and the rest of Ethereum will too with EIP-4444 and other initiatives in The Purge. Indeed, blockchains will rely on non-blockchain P2P solutions like Bittorrent or Portal Network. As I've written about at length, the only unique feature of blockchains is achieving strict global consensus in realish-time. With AGI, this is no longer relevant as they can achieve strict global consensus trivially, and go far beyond that to attain more complex and subjective forms of consensus. Again, caveat for all of this - if an AGI actually happens.

8 replies

11 recasts

187 reactions

jp 🎩 pfp

cc @cassie what thoughts does this bring in relationship to Q?

1 reply

0 recast

1 reaction

Cassie Heart pfp

I had a few initial reactions on reading this, but I wanted to let it sit before I replied. Spicy reply inbound. First, I generally agree with a lot of polynya's writings, but this isn't one of those. I take pause with calling even what Ethereum is today a block chain at all, as definitionally speaking, a block chain is a continuously growing chain of blocks relying on a probabilistic security measure that strengthens consensus via a statistical or economic measure that converges on one side of a fork, and it does none of those things in fact: - blobs are purged, meaning the history of the chain (even if only active state) is not fully preserved by the protocol - finality is predicated on a social contract per weak subjectivity – not only snap sync, but in the epochs of the beacon chain. This is argued as a net good because it limits reorg depth and overall slashing risk, but one would argue both are conditions that make PoS bad, not weak subjectivity good.

1 reply

0 recast

2 reactions

Cassie Heart pfp

- structurally the block chain of Ethereum (i.e., the original 1.X chain) is further confounded in structure by virtue of the separation through the beacon chain – these are technically two separate chains, even if the core execution layer's block chain is intrinsically bound to the outer beacon chain's linkage. Second, and more to the main point: AGI does not by any principle surface from a block chain or strict global consensus – this I agree, AGI does not have any clear definition as it is, and ultimately, if it truly matches the frailty of the human brain it will likely be just as incapable of marching in a straight line logically as we are physically (seriously, if you've never seen an example, we can't do it without a guide, we eventually start walking in literal circles). It being a source for any form of consensus, however, is likely false.

1 reply

0 recast

1 reaction

Cassie Heart pfp

But where I depart significantly is that I believe the intersection of AI and crypto is something valuable – with no true incentive/disincentive structure for a self-aware AI, giving it one by having its resources economically aligned to a network's economics is the most practical application for giving it a drive for survival that we could possibly engineer. An AGI (or more concerningly, an ASI) under the direct reins of a corporate or nationalized owner is more likely to misbehave as its training and direction is aligned strictly with the influences and concerns of the managing entity. An AGI that is intrinsically bound to a decentralized network for survival is both a free agent, but must also be aligned with the operators of the global network.

3 replies

0 recast

0 reaction

Cassie Heart pfp

Third, and speaking directly to the argument: the idea that AGI would not be a single entity because it is not on a single machine seems somewhat muddled – GPT-4 is decidedly a singular entity despite how many machines it trained across initially and runs inference on presently because the entirety of the bounds in which it operates and is trained is collectively a singular system. Our brains are arguably not a singular entity under the same framework, as our neurons are independent and can grow and die independently of another, and regionally there are clearly independent operations in the brain. AGI will very likely be an emergent property of many different AI systems, but I believe the best suited place for that is a decentralized, crypto-economically incentivized network.

2 replies

0 recast

1 reaction

jp 🎩 pfp

“An AGI that is intrinsically bound to a decentralized network for survival is both a free agent, but must also be aligned with the operators of the global network.” -> who are the operators of the global network in this case? the people that run the nodes? the people that use it?

1 reply

0 recast

1 reaction

jp 🎩 pfp

“giving it one by having its resources economically aligned to a network's economics is the most practical application for giving it a drive for survival that we could possibly engineer.” -> do you think we will engineer its survival or it will engineer it itself?

1 reply

0 recast

1 reaction