Amie
@amie
Companies are finding out the moat isn't in having petabytes of data. It's in having the right data. When it comes to training advanced AI, they're actually data-poor. As a result, there's been a big shift towards small data, small models and synthetic data.
1 reply
0 recast
5 reactions
Les Greys
@les
wild. I knew this was going to happen.
1 reply
0 recast
0 reaction
Amie
@amie
I am curious to hear more!
1 reply
0 recast
0 reaction
Les Greys
@les
idk where i posted it maybe I'll try to look up notes. The short version is that I knew that data quantity will not be the thing to create strong business models, it was going to be data quality. The best performing models were always data with great classification, which were normally better structured and smaller. Because the cost of training these models gets more expensive, and better quality data is necessary, at greater levels than today, there was no where to go except, general purpose foundation models paired with high specificity data, which is normally smaller.
1 reply
0 recast
1 reaction
Les Greys
@les
hope this makes sense. Just so you're aware, there is some fractal behavior in this. As in, as bigger models continue to get bigger, the size of the smaller data needs is relative to the continuous change.
1 reply
0 recast
1 reaction
Amie
@amie
Makes total sense. It’s interesting that what we are finding makes AI effective/ineffective mirrors most of life. Basic example: a restaurant with a big menu serving a little bit of everything is terrible to a foodie, good to an average person, great to those who don’t know any better.
0 reply
0 recast
1 reaction
