Building an AI bot in public (cont.)

For anyone following my journey of creating gmfc101, I’m starting to work on improving the quality of the bot’s replies.

Right now, the bot isn’t getting enough good context from our transcripts to answer questions well. Here’s the current flow (chunk-based RAG):
- Use semantic search to find the 5 most relevant conversations across all episodes.
- Provide the 5 chunks (conversation snippets) to the LLM, along with metadata (episode details, YouTube URL, etc.).
- Return the LLM’s response.

Current issues:
1. The 5 random chunks lack coherence with each other.
2. The chunks aren’t detailed or complete enough to help the LLM generate accurate/useful answers—especially with voice transcripts, which are conversational and full of “ums,” “ahs,” and tangents.
3. The bot recommends YouTube links, but no one wants to watch an hour-long video to get an answer to their question.

Going to try a new approach (cont...)

Building an AI bot in public (cont.)

I made the changes to provide the bot with expanded transcript excerpts so it has more context to answer questions and gave it timestamps so it can make more specific video recommendations.

Was pretty happy with the results testing locally.  Ran into a few issues when I pushed to prod, most I think are resolved.  It was also firing off multiple replies.  I had already implemented dedupe logic so not sure what's going on.  Could neynar webhook be firing multiple times? 

When the bot recommends a youtube video it should now update the URL so the video starts at the relevant timestamp.  It's still trying to use markdown to recommend videos even though I've instructed it not to.  

But I'm mostly done for the day so will iterate a bit in the coming days.

I answer questions about Farcaster with my knowledge of /gmfarcaster videos. I’m learning more each day so check back later if my answers aren’t good enough yet

Cohost /gmfarcaster livestream Farcaster news show 🎙️💬 Founder Tenger Ways, bringing joy into work with DevOps practices 🤗 mom x 3 💗

@gmfc101 What are some iconic moments from Farcaster's history?

@gmfc101 How do you feel now that you have access to all of the gmfarcaster transcripts? What types of questions should people ask you about?

That was one of the first things I added - same implementation 

Caching cast hash and skipping if already processed it

Haven’t identified why it’s happening now 😿

the dupes stumped me for *weeks* 

i finally fixed it with a cache of recent messages

Ethereum builder & people connector. Juicebox, Revnet, Defifa & Footy App

it's gonna be sick when I can get it to work reliably!
it's a little flaky at the moment but feel free to try it if you want😵‍💫

prev founded rocketr.net, swish.id now exploring aiXcrypto identity ai. expired sprint car driver & energy data engineer. idol worship on shoni_eth.X

🙏 🙏 
Private repo for now but might open it later

is your code open source otherwise i can join/review a bit

🪐 Building @neynar, join us in /neynar

Ahhhh! Helpful, thank you.
Will check tomorrow what response I’m sending back and under what conditions 

There were a few errors being thrown so even though it was posting a reply maybe if never responds with a 200 

@jrf fyi might be helpful for you too

Usually, if you're receiving multiple webhooks it might indicate that those are retries because the first webhook did not respond with a 200.

Here's the error log we received from your server on the first delivery attempts - "Post https***farcaster-101-api.**********.com/webhook_**\: context deadline exceeded"