8 tps, painful to watch. Fair play for trying.

M4 Mac Mini AI Cluster

Uses @exolabs with Thunderbolt 5 interconnect (80Gbps) to run LLMs distributed across 4 M4 Pro Mac Minis.

The cluster is small (iPhone for reference). It’s running Nemotron 70B at 8 tok/sec and scales to Llama 405B (benchmarks soon). https://t.co/9fx39IP4ZZ

Alex Cheema - e/acc

8 tps, painful to watch. Fair play for trying.

https://x.com/alexocheema/status/1855238474917441972

Bias Research Ltd. • Druglord •
De-di-designer, autodidact. 🗿 UXAI/ML@ • 
 🫧 championing independent thot o7 • dont let your /antimeme ‘s stay dreams

Thunderbolt 5 is rated at 120Gbps, why they claiming 80 here..

Not sure what's more painful, the gameplay or the healthcare system in that game!