Content
@
0 reply
0 recast
0 reaction
Joe Petrich 🟪
@jpetrich
An engineer on my team has been working on a migration of some indexed blockchain data to postgres and took our queries from an initial average latency of almost 2 seconds to under 100ms. The process to get there was fascinating, imo. Would anyone else be interested in a writeup of the optimization techniques he used?
11 replies
2 recasts
28 reactions
Royal
@royalaid.eth
I am super curious about it and how it compares to the tech that @notnotstorm is using with Cryo
1 reply
0 recast
2 reactions
Joe Petrich 🟪
@jpetrich
What is Cryo? Have a link?
1 reply
0 recast
1 reaction
Royal
@royalaid.eth
https://github.com/paradigmxyz/cryo
1 reply
0 recast
2 reactions
Joe Petrich 🟪
@jpetrich
Ok interesting, I don't think it could be more performant than what I've written since it uses the same RPC calls. What we were optimizing was our queries of that data after we've indexed it in Postgres. I don't think there's a way, if using RPC, to improve much on the initial query latency.
3 replies
0 recast
1 reaction
Joe Petrich 🟪
@jpetrich
That does look super convenient though and I'm interested in the bit of their README that mentions a future improvement to query a node directly. eth_getLogs is so incredibly maddening to use
1 reply
0 recast
1 reaction
Eddy Lazzarin ðŸŸ
@eddy
The reason why Shovel https://indexsupply.com/shovel/ is so fast is that it skips the RPC stage altogether.
2 replies
0 recast
1 reaction
storm
@notnotstorm
if youre bottlenecked purely by the rpc server then the speed will be the same. but often theres more bottlenecking on the client than most ppl think. unless youre already using rust/c++, cryo might still be faster thanks to the client overhead from parallelization, serialization, and io
2 replies
0 recast
0 reaction
