Building an AI bot in public (cont.)

For anyone following my journey of creating gmfc101, I’m starting to work on improving the quality of the bot’s replies.

Right now, the bot isn’t getting enough good context from our transcripts to answer questions well. Here’s the current flow (chunk-based RAG):
- Use semantic search to find the 5 most relevant conversations across all episodes.
- Provide the 5 chunks (conversation snippets) to the LLM, along with metadata (episode details, YouTube URL, etc.).
- Return the LLM’s response.

Current issues:
1. The 5 random chunks lack coherence with each other.
2. The chunks aren’t detailed or complete enough to help the LLM generate accurate/useful answers—especially with voice transcripts, which are conversational and full of “ums,” “ahs,” and tangents.
3. The bot recommends YouTube links, but no one wants to watch an hour-long video to get an answer to their question.

Going to try a new approach (cont...)

Building an AI bot in public (cont.)

I made the changes to provide the bot with expanded transcript excerpts so it has more context to answer questions and gave it timestamps so it can make more specific video recommendations.

Was pretty happy with the results testing locally.  Ran into a few issues when I pushed to prod, most I think are resolved.  It was also firing off multiple replies.  I had already implemented dedupe logic so not sure what's going on.  Could neynar webhook be firing multiple times? 

When the bot recommends a youtube video it should now update the URL so the video starts at the relevant timestamp.  It's still trying to use markdown to recommend videos even though I've instructed it not to.  

But I'm mostly done for the day so will iterate a bit in the coming days.

Cohost /gmfarcaster livestream Farcaster news show 🎙️💬 Founder Tenger Ways, bringing joy into work with DevOps practices 🤗 mom x 3 💗

Overall, progress on AI bot is positive, with some minor issues to resolve. Excited for future improvements