a crazy idea: the fact that most (if not close to all) models, papers, are all going to be obsolete in a few years

even though they feel so new today

creating and destroying; built flying cars & autonomous vehicles, now building /faust, more here: /july, web: july.rocks

well or they won't. Progress isn't guaranteed without new conceptual breakthroughs.

I don't really see any improvements coming until models get embodied (i.e. exist as robots so can learn from reality)

Tech entrepreneur. Founder @electricsql. Founder @gatsbyjs (acquired by Netlify). @kylemathews on Twitter

So you think we’re near the end of this cycle?

quite possibly. All the new models released in the last year have been somewhere between gpt 3 & 4. Lots to do w/ evals, fine-tuning, training on propritary data, efficient inference, etc. but "intelligence" of models seems like it's topped out — see https://warpcast.com/kam/0x1dd60ed3 & linked @vgr post

Blogger at ribbonfarm.com, running summerofprotocols.com, host of /gloom ☀️

Building something new. Host: /venturecapital /adventures /theitinerant

Agree with you on this, we’ll probably need a step change akin to CNNs did for deep learning that transformers have done for LLMs

the thing is though that GPT4 is very very good — if LLMs are in case just an extreme type of data mining — then GPT4 is perhaps near the limits of what you can mine from human language.

How long until we get a model that can interpret structures data and equations? (i.e., finance)

Plenty of specialized models out there that do this. Unsure exactly what natural language (LLMs) add besides a query layer? Crunch can already happen

"Creative Technology" 🤌🏽

@ google creative lab

Owner of /makerspace - come build with us!

I can't as it's not my field but friends at big shops all talk about the algos and models they run. It's why earnings are so unrelated to the actual numbers lol. OCR hits statements as they are published and it's pushed thru analysis pipeline that has different models tuned to different intents etc etc. I hate it 😅

Can you share an example?

To clarify: models that can interpret financial statements