Content
@
https://warpcast.com/~/channel/fc-devs
0 reply
0 recast
0 reaction
vrypan |--o--|
@vrypan.eth
Many *Request protobuf messages have a page_token (returned by the prev response). Can this token be constructed programmatically? My problem: Writing a backup script that saves an FID's casts, reactions, links. I want next time I run the script to start where I stoped. How could I do it efficiently? Can I use the last message saved to generate a page_token that will start from there? The alternative is to use reverse=true and stop at the last message previously saved (and hope it was not deleted), but I'd rather use reverse=false, if I can create a page_token. Maybe @sanjay has an idea?
1 reply
0 recast
0 reaction
Sanjay
@sanjay
We can't guarantee the pagination is stable because older messages could be deleted/pruned/revoked. Depending on how frequently you're doing this, you might want to consider using the events subscribe endpoint (you can filter by fid). With snapchain, you can also use block numbers.
1 reply
0 recast
0 reaction
vrypan |--o--|
@vrypan.eth
My understanding: - page_token is associated with the rocksdb key of the last message fetched. - if I have a message from a previous run of the program, and I could calculate its key, I could use it as page_token in my next query. - my guess is that in a typical kv-store query like this, it doesn't matter if page_token actually exists in the db, it is only used to iterate over keys that are larger than it. Anyway, it seems that fetching all user casts, reactions and links, even for fid 3 🤣, only takes a couple of seconds, so optimizing to fetch only new ones is not worth the effort.
1 reply
0 recast
0 reaction
