working on /microsub; creator of @degentipbot.eth; producer/dj; host of /campfire 🎧 + /downshift ⚙️

lately, i’ve been quietly working on data infra for dealing with flaky data sources (like Farcaster data and blockchain data…this isn’t a dig at Farcaster by any means; it’s just the nature of consensus-based protocols in general)

this stuff is non-trivial to get right because intermittent and complete failures of data providers are guaranteed, especially in times of massive usage and network congestion we are and will continue experiencing 

robust data access patterns and thorough fallback strategies are absolutely essential to have any chance of creating reliable systems in these sorts of environments, especially if your goal is to minimize latencies

working on infra to displace adtech at /microsub • dance music lover + host of /campfire 🎧

What causes the data to lose integrity when it comes to consensus based?

Is it based on the type? PoS vs PoH?

https://warpcast.com/~/channel/0xygendebt/join?inviteCode=gMe9pEABCKxPtQh28Gw1nQ

Is there then a way to determine precisioning based on multifactoral analysis? 

Using multiple nodes to provide higher level of data redundancy or a way to pre-validate at a certain resolution?

since there isn’t a single source of truth (like a centralized database in common web apps) we must rely on data replicating across many nodes and indexing through many bespoke and opaque means…data sources aren’t all guaranteed to be in a consistent state

this means some nodes or indexes will not be fully up to date, or might even have stale data due to their own implementations or network errors, caching, etc.