Companies are finding out the moat isn't in having petabytes of data. It's in having the right data. 

When it comes to training advanced AI, they're actually data-poor. As a result, there's been a big shift towards small data, small models and synthetic data.

I make technology work for businesses | Optimizing workflows & systems | prev head of content @makerpad (acq. by @zapier)

co-founder, terminally onchain family-man, building products on frontier solutions w frontier people.
/grey
lesgreys.com

idk where i posted it maybe I'll try to look up notes. 

The short version is that I knew that data quantity will not be the thing to create strong business models, it was going to be data quality. The best performing models were always data with great classification, which were normally better structured and smaller. 

Because the cost of training these models gets more expensive, and better quality data is necessary, at greater levels than today, there was no where to go except, general purpose foundation models paired with high specificity data, which is normally smaller.

hope this makes sense. Just so you're aware, there is some fractal behavior  in this. As in, as bigger models continue to get bigger, the size of the smaller data needs is relative to the continuous change.