Content
@
0 reply
0 recast
0 reaction
Sanjay
@sanjay
Our existing sync architecture for Hubs is running into scaling issues. Been thinking about a new design that can scale to billions of messages and millions of fids. If you enjoy complex distributed systems problems, I would appreciate feedback https://warpcast.notion.site/Sync-V2-a9c0fd81d7b245a0b3fbd51e6007909f
16 replies
15 recasts
92 reactions
jj 🛟
@jj
For snapshotting it’ll be really important to add integrity checks at scale this becomes a bigger and bigger problem where you might need to think about error handling or partially corrupt snapshots
1 reply
0 recast
0 reaction
Sanjay
@sanjay
Yes, good point, the snapshot right now comes from a trusted place (one of our own prod hubs). But still we should validate all the messages in the snapshot before we load it for the first time. Important, but low priority for now.
0 reply
0 recast
0 reaction
