PostgreSQL

Ponder

Fun debugging rabbit hole from this week.

Last week, a few Ponder users opened issues for an "invalid byte sequence for encoding "UTF8": 0x00" error. Most software ignores or removes null characters (\0, \x00, or \u0000), but the Postgres TEXT data type rejects them with that error. 

We reproduced the issue with an ENS NameRegistered event log from 2022 (s/o @greg). Turns out, some EVM event logs contain null characters within decoded `string` parameter values.

Now, the framework removes null characters from all decoded ABI parameters before passing them to userland. We could have instead fixed this on the PG driver/Drizzle side, but it seemed more fragile (are we wrong?).

We're not certain why this only appeared recently - we haven't changed any relevant codepaths in recent releases. My best guess is that Viem used to remove them but stopped. Viem is a peer dep, so the version can change independently from Ponder's version.

PR: https://github.com/ponder-sh/ponder/pull/1414

Quick PG debugging story. And a great blog post about the Postgres null byte situation. Seems to be a common rough edge.
https://www.commandprompt.com/blog/null-characters-workarounds-arent-good-enough/

Quick PG debugging story. And a great blog post about the Postgres null byte situation. Seems to be a common rough edge.
https://www.commandprompt.com/blog/null-characters-workarounds-arent-good-enough/

https://warpcast.com/typedarray.eth/0x761b14a8

Building ponder.sh. EVM, TypeScript, open-source enjoyer.

wow, never thought a null byte could cause such drama 😂 thanks for sharing the insights! gonna dig into that blog post. cheers!