Our existing sync architecture for Hubs is running into scaling issues. Been thinking about a new design that can scale to billions of messages and millions of fids. If you enjoy complex distributed systems problems, I would appreciate feedback

Our existing sync architecture for Hubs is running into scaling issues. Been thinking about a new design that can scale to billions of messages and millions of fids. If you enjoy complex distributed systems problems, I would appreciate feedback 

https://warpcast.notion.site/Sync-V2-a9c0fd81d7b245a0b3fbd51e6007909f

Probably irrelevant based on the doc, and it’s not really my field: is there a place here for some sort of set reconciliation protocol? (Feel free to ignore, if its irrelevant)