New mixtral-8x22b just dropped. And I didn’t even have time to check out Command R+ from last week yet. And (small) Llama 3 apparently coming in a week or so. Looks like an exciting next wave of open-weight LLMs!

New mixtral-8x22b just dropped. And I didn’t even have time to check out Command R+ from last week yet. And (small) Llama 3 apparently coming in a week or so. Looks like an exciting next wave of open-weight LLMs!

https://x.com/mistralai/status/1777869263778291896

Ethereum protocol R&D. Admirer of emergent order.

Oh and forgot to mention DBRX from 2 weeks ago! Actually looks like very similar size as this new mixtral, but 16 experts (4 active) at 11b each vs mixtral’s 8 (2 active) at 22b each. Will be interesting to compare these choices.

I love how they distribute this on BitTorrent.