hey if you're wondering where the public data dump for April is, I need help figuring out the implications of my decisions with the Snapchain setup -- would love any ideas.

Short of it: Snapchain prefers a 24/7 running instance, can't afford to run a heavy box 24/7, stuck on filtering 90GB of data on 8GB of RAM

do you have to load everything into ram? can't you just run an event-driven system?

I remember @christopher was working on a data system. any thoughts on how to tackle this issue?

so the snapchain itself is an event driven system and then i retrieve data from it in bulk, i could switch over to a secondary event driven system tho and filter on demand then save