Content
@
0 reply
20 recasts
20 reactions
Varun Srinivasan
@v
Hubble Replicator v2 We have a proposal for a new design to tackle the problems with the existing replicator. If you run a replicator today, we'd love to get your feedback on this. https://warpcast.notion.site/Replicator-v2-Public-6562ad1e6eef4c4ab2c27cf82aa759c1
9 replies
8 recasts
57 reactions
Kartik
@slokh
Similar to how ours works, except we're just using one event stream because we're throwing away a lot of stuff. Most apps won't need the full dataset and likely would want to save on DB size. Separately, @typedarray.eth would be cool eventually to include Farcaster in Ponder.
0 reply
0 recast
7 reactions
David Furlong
@df
Love to see it! I think helpful in spec would be how it handles backfilling - how long backfilling takes & processing backfill events in a consistent order (not by fid) that allows for catching up without having to rewrite logic
0 reply
0 recast
1 reaction
Zach
@zachterrell
a third replicator has hit the south tower mr president
0 reply
0 recast
0 reaction
artlu is on lunchbreak!
@artlu
request dev tools, scripts, or functions to: - report db stats for slice of FIDs, date range - surgically backfill slice of FIDs, date range - self-diagnose, recover from outages and repair data holes - tutorials / docs on how to reason about replicator state and canonical source of truth
1 reply
0 recast
1 reaction
🦒
@srijan.eth
@vhawk19
0 reply
0 recast
1 reaction
vhawk19.eth
@vhawk19
Issues am facing with current replicator setup - Backfill takes a long time (Public snapshots would help here) - Cannot prioritise certain messages over others - vv cpu intense, and cannot scale horizontally Is it possible to have multiple processors read from the event stream?
1 reply
0 recast
1 reaction
Hustor
@hustor
https://github.com/farcasterxyz/hub-monorepo/issues/1823
0 reply
0 recast
0 reaction
BradGao
@bradgao
The backfill process parses and upserts messages one by one, making it slow and difficult to customize.
0 reply
0 recast
0 reaction
Vishal Sachdev
@vishalsachdev
@shuhari.eth you might have some inputs given the replicator we are running
0 reply
0 recast
0 reaction
