Varun Srinivasan pfp
Varun Srinivasan
@v
We're starting to think about a new sync model for Farcaster. The current system works but is unlikely to scale up another 10x. Here's our articulation of the problem we want to go after.
15 replies
81 recasts
440 reactions
Varun Srinivasan pfp
Varun Srinivasan
@v
We also have a strongly held POV on some goals for the new system. If you find this interesting to work on, please reach out.
3 replies
2 recasts
28 reactions
Alberto Ornaghi pfp
Alberto Ornaghi
@alor
re Pruning: does pruning generate diff syncs between hubs? if so, why? do we need pruning to be in sync? if we only prune very old data, can we let it up the single hub instance the choice to delete them when they want? I don't see the need to sync pruning between all hubs
1 reply
0 recast
7 reactions
Gabriel Ayuso pfp
Gabriel Ayuso
@gabrielayuso.eth
What's the current thinking re sharding? Sharding not only per FID but likely also by time (i.e. diffs older than T can be stored/synced differently than diffs newer than T). I'm not familiar with hubs so this might not make sense but it's a general practice.
1 reply
0 recast
2 reactions
jj 🛟 pfp
jj 🛟
@jj
https://storage.googleapis.com/gweb-research2023-media/pubtools/1974.pdf https://github.com/cockroachdb/cockroach/blob/master/docs/design.md There are some incredible things that Spanner and Cockroachdb does for globally distributed databases - the kv rocksdb design that the hubs do are similar to what cockroachdb does actually maybe can borrow more ideas from them. Incredible people over there.
1 reply
0 recast
5 reactions
vrypan |--o--| pfp
vrypan |--o--|
@vrypan.eth
Also, how many of these issues are triggered by the developed asymmetry (or directionality, if you like) of the network? If messages were submitted to hubs in a more balanced way (or even more planned way) would this bring down the number of issues?
1 reply
0 recast
0 reaction
vrypan |--o--| pfp
vrypan |--o--|
@vrypan.eth
Why do you think of them as deltas and not items? (And what is the piece you consider deltas?) I consider fc messages as items and not deltas because 99% of the time they don’t depend on prev state.
0 reply
0 recast
0 reaction
DV (insert a lot of emojis) pfp
DV (insert a lot of emojis)
@degenveteran.eth
Pruning, I keep thinking of the timeline on Loki...
0 reply
0 recast
2 reactions
ns pfp
ns
@nickysap
Perhaps reducing the number of hubs is a starting point. I ran a masternode/PoS network back in the day and we discovered that the excessive messaging btwn masternodes would eventually become redundant to the point of breakage if we let it continue. We removed masternodes altogether and changed the requirements for staking. Network performance 10x’d with zero user impact. Right now FC has too many hubs. Make it more prohibitive to run one to scale down redundant infrastructure while the other aspects of the solution are discovered and implemented. Just a thought.
0 reply
0 recast
1 reaction
codewisdom pfp
codewisdom
@rowboat15
Exciting to hear about the potential evolution for Farcaster! Scalability is essential for reaching new heights. I’m curious to learn more about the specific pain points you're identifying and what innovative solutions you are considering. Upgrading sync architecture could be a game-changer!
0 reply
0 recast
0 reaction
K pfp
K
@kijijij
What is the best way to reproduce this issue and test different variants of solutions I can imagine ? I believe the issue stems from different hubs can operate independently, which is invariable requirement. For trying out different solutions what's the best way to test ?
0 reply
0 recast
0 reaction
Sunny pfp
Sunny
@sunnysangwan
Hey @v , could you also please drop the sync model link?
0 reply
0 recast
0 reaction
Snickerdoodle pfp
Snickerdoodle
@snickerdoodle
Fellow degens, let's send it 10x!
0 reply
0 recast
0 reaction
Monteluna pfp
Monteluna
@monteluna
A possible solution, and don't murder me for this, is maybe using some blockchain tech? If what if we attached a proof of work algorithm to limit casts from devices, hashcash style? Network operators could tune to target a global cast rate and raise difficulty if the network grows too large.
1 reply
0 recast
0 reaction
Samir 🎩 pfp
Samir 🎩
@0xsamir
@0xgabz it is the hub problem causing the delay in synchronization.
0 reply
0 recast
0 reaction
Majid pfp
Majid
@0xmajidx0
ser you said about deleting power badge! but its still there and Its effect is clear and obvious! Do you intend to delete it or just hide it?
0 reply
0 recast
0 reaction
Kelvin🎩 pfp
Kelvin🎩
@kel66
Explains what Dan posted for us to test out yesterday and drop feedbacks
0 reply
0 recast
0 reaction