Content
@
0 reply
0 recast
0 reaction
July
@july
a crazy idea: the fact that most (if not close to all) models, papers, are all going to be obsolete in a few years even though they feel so new today
7 replies
2 recasts
22 reactions
Kyle Mathews
@kam
well or they won't. Progress isn't guaranteed without new conceptual breakthroughs. I don't really see any improvements coming until models get embodied (i.e. exist as robots so can learn from reality)
1 reply
0 recast
1 reaction
Ben 🟪
@benersing
So you think we’re near the end of this cycle?
1 reply
0 recast
1 reaction
Kyle Mathews
@kam
quite possibly. All the new models released in the last year have been somewhere between gpt 3 & 4. Lots to do w/ evals, fine-tuning, training on propritary data, efficient inference, etc. but "intelligence" of models seems like it's topped out — see https://warpcast.com/kam/0x1dd60ed3 & linked @vgr post
1 reply
0 recast
0 reaction
July
@july
Agree with you on this, we’ll probably need a step change akin to CNNs did for deep learning that transformers have done for LLMs
2 replies
0 recast
1 reaction
Kyle Mathews
@kam
the thing is though that GPT4 is very very good — if LLMs are in case just an extreme type of data mining — then GPT4 is perhaps near the limits of what you can mine from human language.
1 reply
0 recast
0 reaction
Ben 🟪
@benersing
How long until we get a model that can interpret structures data and equations? (i.e., finance)
2 replies
0 recast
1 reaction
//trip
@heytrip.eth
Plenty of specialized models out there that do this. Unsure exactly what natural language (LLMs) add besides a query layer? Crunch can already happen
1 reply
0 recast
1 reaction
Kyle Mathews
@kam
I'd be surprised if there isn't already several teams working on it. It might be harder though. The advantage of language is there's enormous quantities of written language available. The models are still extremely inefficient at learning
0 reply
0 recast
1 reaction
