fc-updates

Scaling Gossip with Bundles

Hubs are running into scaling issues with libp2p. We're proposing a change to "bundle" messages to fix some of these issues. 

This may add a ~1s delay to casts moving between clients.

Scaling Gossip with Bundles

Hubs are running into scaling issues with libp2p. We're proposing a change to "bundle" messages to fix some of these issues. 

This may add a ~1s delay to casts moving between clients.

https://warpcast.notion.site/Scaling-Gossip-in-Hubble-e66c766fa6b04afcb407f4800134cd72?pvs=25

Technowatermelon. Elder Millenial. Building Farcaster. 

nf.td/varun

How did you determine the 1s delay? Are these dynamic batches or fixed in size?

Another thing might be to be clever with topics and batching.

Lowest hanging fruit might be to differentiate historic sync vs realtime gossip if you’re not doing that already.

So historic syncs you can have much larger batches vs gossip data might want to be smaller for latency

I get that batching will dramatically reduce the sizes, and I guess the dedup is fine.. do you have any data on how much each hub is dedupping? 

Perhaps you can get much more efficient with metadata only peering to more efficient pub IWANT in these batches

The batches will be dynamic in size. The 1s is because that’s the time resolution we use in the sync trie.