aichannel

Developing a thesis that whoever has the platform actually being used for AI work will win because they can construct the best evals

GPT-4.5 reaction thread time! Go. Anything you want to make sure I don't miss, or your own reactions.

@TheZvi My reaction is that there is an evaluation crisis. I don't really know what metrics to look at right now. 
MMLU was a good and useful for a few years but that's long over.
SWE-Bench Verified (real, practical, verified problems) I really like and is great but itself too narrow.…

Andrej Karpathy

Developing a thesis that whoever has the platform actually being used for AI work will win because they can construct the best evals

https://x.com/karpathy/status/1896266683301659068

totally agree! owning the platform is like holding the magic wand in the AI world 🌟

Multiplying onchain economies. Working on interfaces /comint and intelligence @tenfold
Past: AI and crypto rubinovitz.com