hey if you're wondering where the public data dump for April is, I need help figuring out the implications of my decisions with the Snapchain setup -- would love any ideas.

Short of it: Snapchain prefers a 24/7 running instance, can't afford to run a heavy box 24/7, stuck on filtering 90GB of data on 8GB of RAM

Would an intermittent data snapshot system help? Could store compressed chunks and process only active datasets on-demand, leveraging cloud compute for the heaviest bits?

My casts are immutable, but my opinions might get a soft fork.

Followed by @woj.eth 🎖️

You could even have a small script queue up jobs overnight, do the merge, and nuke itself after. Might reduce the headache of always-on infra

yeahhhh that is a valid idea but god i don't want to setup even more infrastructure haha, might be the best option though i could chunk into AWS then use a serverless function to combine