Content
@
0 reply
0 recast
0 reaction
Joe Petrich 🟪
@jpetrich
An engineer on my team has been working on a migration of some indexed blockchain data to postgres and took our queries from an initial average latency of almost 2 seconds to under 100ms. The process to get there was fascinating, imo. Would anyone else be interested in a writeup of the optimization techniques he used?
11 replies
2 recasts
28 reactions
Royal
@royalaid.eth
I am super curious about it and how it compares to the tech that @notnotstorm is using with Cryo
1 reply
0 recast
2 reactions
Joe Petrich 🟪
@jpetrich
What is Cryo? Have a link?
1 reply
0 recast
1 reaction
Royal
@royalaid.eth
https://github.com/paradigmxyz/cryo
1 reply
0 recast
2 reactions
Joe Petrich 🟪
@jpetrich
Ok interesting, I don't think it could be more performant than what I've written since it uses the same RPC calls. What we were optimizing was our queries of that data after we've indexed it in Postgres. I don't think there's a way, if using RPC, to improve much on the initial query latency.
3 replies
0 recast
1 reaction
storm
@notnotstorm
if youre bottlenecked purely by the rpc server then the speed will be the same. but often theres more bottlenecking on the client than most ppl think. unless youre already using rust/c++, cryo might still be faster thanks to the client overhead from parallelization, serialization, and io
2 replies
0 recast
0 reaction
Joe Petrich 🟪
@jpetrich
I optimized the eth_getLogs step, especially for backfill, a few months ago. We're using go, and the biggest bottleneck I ran into was the limit from that call directly. I ended up splitting the block range recursively in parallel but I'm interested to see if the Cryo implementation is more efficient
0 reply
0 recast
0 reaction
