Farcaster Devs

Update: shuttle does not appear to be a 1:1 replacement for replicator. Docs could be much clearer on the architecture here: it seems like shuttle owns one table in pgsql, "messages", and your app can handle messages in the same tx to populate additional tables e.g. the example app populates a casts table. Not sure if I like that - would rather split the loading & transforming processes.

Update: I have doubts about shuttle

I trimmed the shuttle example app down to bare bones "just write the messages" into pgsql.

I'm running on a Hetzner CX52 VPS (16 cores / 32 GB RAM).

The docs are vague, but imply you can run more than 1 worker. I started with 1 and observed high CPU utilization so left it there. There's no guidance on when to increase workers. There is no throughput measure I can see?

In 12 hours, I have 8.7M messages in postgres. Hubble dashboard shows 502M messages, and I assume that's what I'll end up with in PG, therefore at this rate the process will complete in 29 days.

I've started up 3 more workers.

Husband, dad, dev • @dune.eth wizard • Ultimate Ⓜ️oxie Dashboard https://tinyurl.com/moxiedash • Ultimate Degen Tips FAQ https://tinyurl.com/degenfaq