a crazy idea: the fact that most (if not close to all) models, papers, are all going to be obsolete in a few years

even though they feel so new today

creating and destroying; built flying cars & autonomous vehicles, now building /faust, more here: /july, web: july.rocks

well or they won't. Progress isn't guaranteed without new conceptual breakthroughs.

I don't really see any improvements coming until models get embodied (i.e. exist as robots so can learn from reality)

Tech entrepreneur. Founder @electricsql. Founder @gatsbyjs (acquired by Netlify). @kylemathews on Twitter

So you think we’re near the end of this cycle?

quite possibly. All the new models released in the last year have been somewhere between gpt 3 & 4. Lots to do w/ evals, fine-tuning, training on propritary data, efficient inference, etc. but "intelligence" of models seems like it's topped out — see https://warpcast.com/kam/0x1dd60ed3 & linked @vgr post

Blogger at ribbonfarm.com, running summerofprotocols.com, host of /gloom ☀️

Building something new. Host: /venturecapital /adventures /theitinerant

the thing is though that GPT4 is very very good — if LLMs are in case just an extreme type of data mining — then GPT4 is perhaps near the limits of what you can mine from human language.

Agree with you on this, we’ll probably need a step change akin to CNNs did for deep learning that transformers have done for LLMs