Content
@
0 reply
0 recast
0 reaction
Greg
@greg
I haven't been following the AI space very closely, especially outside of OpenAI stuff. How far behind are open source alternatives? I've heard about stuff like llama2 supposedly being decent for chat but I'm assuming the agents/function calling and whatnot is a different story?
8 replies
0 recast
11 reactions
Katsuya
@kn
It depends on how you measure it/use cases, e.g. some OS models claim they are better than GPT-4 in some benchmark. But in general, my intuition is that OpenAI is ~1 year ahead than OS models. My building approach is: Just go with OpenAI if possible then optimize with other options if absolutely need to.
1 reply
0 recast
2 reactions
Kyle Mathews
@kam
I switched from gpt3.5 to openhermes-2-mistral-7b for a side project as the gpt3.5 API was taking 35-85 seconds to finish where the oss LLM on https://www.perplexity.ai took 5-8 seconds with comparable quality. Gpt4 is mildly better quality wise but the huge difference in speed (and price) meant a lot more
3 replies
0 recast
4 reactions
Katsuya
@kn
gpt3.5 taking 10x longer is surprising to me. Is it consistently that way? Going with smaller models for speed/price makes sense if that works for your use cases. In general though I think devs tend to spend too much time optimizing for these choices prematurely.
2 replies
0 recast
1 reaction
Katsuya
@kn
(obviously, makes sense to go deep in these details if for side projects/fun :)
0 reply
0 recast
0 reaction
Kyle Mathews
@kam
Hard to say. I saw it for several days and people on forums said it'd been happening for weeks at that point. So perhaps something internally has gotten overloaded 🤷♂️ I'm sure it's fixable. But switching took all of 10 mins and was immediately way better so stayed
0 reply
0 recast
1 reaction
