hellno the optimist
@hellno.eth
I want to add an eval framework for @vibesengineering.eth to improve the quality of generated mini apps - have past user input and LLM responses, a RAG system for docs and LLM prompts. - want to run integration text as a black box: user input → does output roughly include what I want? need recommendations - python frameworks (deepeval seems like a good fit?!) - best practices how to setup and keep improving this as black box - best practices to improve core RAG system
6 replies
1 recast
11 reactions
Sid
@sidshekhar
Can try helicone (https://www.helicone.ai/) for general observability first before getting into evals? have found it helpful as most of the eval frameworks out there aren't fit for purpose
1 reply
0 recast
0 reaction
hellno the optimist
@hellno.eth
thanks will look into helicone 🙏
0 reply
0 recast
1 reaction
