Content
@
0 reply
0 recast
2 reactions
Varun Srinivasan
@v
Anyone here worked on an eng team that processed more than 100M background jobs /day? Starting to run into issues with our setup. Looking for ideas on what people typically turn to at this scale.
28 replies
20 recasts
142 reactions
Jacek.degen.eth 🎩
@jacek
Consider shifting focus from batch jobs to streaming data. You could implement websockets or leverage AWS Kinesis. Identify the primary purposes and time intervals of the jobs. Short-interval tasks should migrate to streaming data for optimal performance.
2 replies
0 recast
10 reactions
0xChris
@0xchris
Streaming is a path to 🟢
1 reply
0 recast
0 reaction
Samuel ツ
@samuellhuber.eth
what are the best starters to learn? currently looking into redis https://redis.io/docs/latest/develop/data-types/streams/ any zero to hero guides with best practices and learnings?
0 reply
0 recast
0 reaction