A Path Towards Autonomous Machine Intelligence Version 0.9.2, 2022-06-27 (Yang LeCun) 

I think about this paper still every now and then

https://openreview.net/pdf?id=BZ5a1r-kVsf

creating and destroying; flying cars, roc.camera, july.rocks

Scientist and engineer interested in Life.

Work on (GOF)AI, Agriculture, Biology, Energy, Robotics, Space---all topics climate-aware.

Kuriyama, Yubari District, Hokkaido, Japan

I’m not an expert and haven’t been spending a ton of time in the space; so take with grain of salt

Hmm doesn’t seem like much progress, since model performance scales with data, synthetic data will get you farther. Yet seems like new architecture unlock (I.e. CNN -> Transformer) level is required for new plateau

More than a year later, do you reckon progress along that path?

Just realised almost never hearing about self-supervised learning techniques for quite a while—but honestly not listening either…

I think he has a point about beating representation collapse, it seems that that current genAI is good at mimicry b/c it doesn't need actual comprehension, but hallucination/nonpredictability needs to be solved before general physical intelligence.