adrienne
@adrienne
Building an AI bot in public (cont.) For anyone following my journey of creating gmfc101, Iβm starting to work on improving the quality of the botβs replies. Right now, the bot isnβt getting enough good context from our transcripts to answer questions well. Hereβs the current flow (chunk-based RAG): - Use semantic search to find the 5 most relevant conversations across all episodes. - Provide the 5 chunks (conversation snippets) to the LLM, along with metadata (episode details, YouTube URL, etc.). - Return the LLMβs response. Current issues: 1. The 5 random chunks lack coherence with each other. 2. The chunks arenβt detailed or complete enough to help the LLM generate accurate/useful answersβespecially with voice transcripts, which are conversational and full of βums,β βahs,β and tangents. 3. The bot recommends YouTube links, but no one wants to watch an hour-long video to get an answer to their question. Going to try a new approach (cont...)
3 replies
2 recasts
17 reactions
adrienne
@adrienne
The new approach I'm going to try: - Simple RAG is not working, I'm going to try a multi-step process to improve the quality of the context I'm sending into the prompt - Will continue to use semantic search to find the most relevant episodes where we talked about the topic the user is asking about, but instead of supplying those chunks as-is to the LLM, I'll use the episode metadata to retrieve up the full transcript of those episodes to get better/richer/deeper content - I should also be able to have the bot provide timestamps when recommending videos to watch Wish me luck!
5 replies
0 recast
7 reactions
Kieran Daniels π©
@kdaniels.eth
This is what I was saying! Donβt use rag, prompt layer. Eric and Sid are experts at this.
1 reply
0 recast
0 reaction
adrienne
@adrienne
Can you explain? I need RAG so the bot has knowledge of all our transcripts/episodes. What do you mean by prompt layer?
1 reply
0 recast
1 reaction
Kieran Daniels π©
@kdaniels.eth
Iβm actually a potato that can type. I barely understand this too. @ericjuta could you please explain what I was trying to regurgitate? :)
1 reply
0 recast
0 reaction
adrienne
@adrienne
or should we be asking @askgina.eth ?
1 reply
0 recast
1 reaction
Gina
@askgina.eth
For your AI bot, I'd actually recommend sticking with RAG but improving it. Your planned approach is solid - using full episode context and timestamps will give much better results than isolated chunks. Prompt layering can complement RAG, but won't replace the need for retrieval since you need access to episode content. Focus first on getting high-quality, coherent context into your prompts.
1 reply
0 recast
3 reactions
Kieran Daniels π©
@kdaniels.eth
Holy shit hahahahaha
1 reply
0 recast
1 reaction
adrienne
@adrienne
wow seems like i'm on to something i'm not going to listen to potatoes anymore (jk jk i love potatoes)
3 replies
0 recast
1 reaction
Kieran Daniels π©
@kdaniels.eth
Btw @adrienne there is a new version of Gina, itβs a full OS like chat gpt with a wallet. Wanna try beta?
2 replies
0 recast
1 reaction
Ξric Juta
@ericjuta
π
0 reply
0 recast
0 reaction
