Vitalik Buterin
@vitalik.eth
My impression of chatgpt so far is that it's good at replicating existing human behavior, but bad at doing anything out-of-sample. It anchors quickly onto patterns that have been done by many people before, but there's definitely something that we call "original thinking" that it's not doing at all.
16 replies
0 recast
3 reactions
Vitalik Buterin
@vitalik.eth
As a random example, consider these AI-generated debates (credit to pmarca for the format). Notice that the Chinese perspective is not even consistent across the two debates! It's pattern matching to the usual yes/no debate, and... https://i.imgur.com/OebQ19c.jpg
3 replies
0 recast
1 reaction
Venkatesh Rao ☀️
@vgr
This is to be expected given how transformers work I think. I was not expecting creativity except by accident.
1 reply
0 recast
0 reaction
Vitalik Buterin
@vitalik.eth
There definitely is the "scale is all you need" school of thought that says that if you just scale up the param count by another 1000x or 1000000x then insight and creativity will appear. Seems false so far, but... I guess we'll see!
1 reply
0 recast
0 reaction
Venkatesh Rao ☀️
@vgr
I’ve been arguing embodiment is necessary for this reason. It’s how human brains are constantly immersed in sensations for 90% of which there are no words. LLMs don’t even have true image data really. The latent diffusion models use tagged image training data so kinda stay within verbal worlds.
3 replies
0 recast
0 reaction
Tom Beck
@tombeck.eth
Embodiment does seem important. What's lacking in GPT is what Knut Hamsun wanted from literature: "the whisper of blood and the pleading of bone marrow."
0 reply
0 recast
0 reaction
