Content
@
0 reply
0 recast
0 reaction
Sanjay
@sanjay
Our existing sync architecture for Hubs is running into scaling issues. Been thinking about a new design that can scale to billions of messages and millions of fids. If you enjoy complex distributed systems problems, I would appreciate feedback https://warpcast.notion.site/Sync-V2-a9c0fd81d7b245a0b3fbd51e6007909f
16 replies
15 recasts
92 reactions
Chase
@harden-hardys
"Considered a bittorrent like approach to syncing fid specific messages, but it’s hard to determine which hubs have the right messages for the fid." Can you say more about this?
1 reply
0 recast
0 reaction
Sanjay
@sanjay
BitTorrent works well when the content doesn't change. Everyone has the same file, and you can chunk it up and fetch it from multiple peers based on which chunks each peer has. In our case, the "file" which you can think of as the set of messages for an fid is always changing. It's still possible, but tricky.
0 reply
0 recast
1 reaction
