We're excited to announce our investment in Pluralis Research, advancing "Actually Open AI" through decentralized, collaborative model training. @usv and @coinfund-io co-led the $7.6M seed round, with @variant, Topology, and others participating

more thoughts: https://blog.usv.com/pluralis-towards-actually-open-ai

is this tensor splitting? (cut a neural net’s big data arrays (weights, activations) into pieces, spread across nodes to lower comms needs)

if so, it’s super neat - basically split the payload from like 100mb a step to like 1-5 mb. maybe even some form of prediction where each node guesses its neighbors’ tensors with a small model trained on past data/sync only when errors spike saving bandwidth etc. never thought about using prediction with sharding…

no clue if I’m right but guessing from reading this that it’s some type of tensor splitting where a model is shared across nodes. no one holds the full weight set—each gets a tensor chunk, like a weight matrix slice, cutting comms to 1-5MB/step over slow nets.

everything that can work, will work | unpacking papers, musings on mathematics, and meditations on craft from an anon founder (@yhprums_law on X)

had far more thoughts in a thread here: https://warpcast.com/yhprumslaw/0xcc4bfe1f