Content
@
0 reply
0 recast
0 reaction
Greg
@greg
I haven't been following the AI space very closely, especially outside of OpenAI stuff. How far behind are open source alternatives? I've heard about stuff like llama2 supposedly being decent for chat but I'm assuming the agents/function calling and whatnot is a different story?
8 replies
0 recast
11 reactions
Katsuya
@kn
It depends on how you measure it/use cases, e.g. some OS models claim they are better than GPT-4 in some benchmark. But in general, my intuition is that OpenAI is ~1 year ahead than OS models. My building approach is: Just go with OpenAI if possible then optimize with other options if absolutely need to.
1 reply
0 recast
2 reactions
Kyle Mathews
@kam
I switched from gpt3.5 to openhermes-2-mistral-7b for a side project as the gpt3.5 API was taking 35-85 seconds to finish where the oss LLM on https://www.perplexity.ai took 5-8 seconds with comparable quality. Gpt4 is mildly better quality wise but the huge difference in speed (and price) meant a lot more
3 replies
0 recast
4 reactions
Kyle Mathews
@kam
Which is to say, there's a lot of tradeoffs at play
0 reply
0 recast
1 reaction
vincent
@pixel
Perplexity's inference is insanely fast. Sad that I have to sub $20 for their API, wish it was pay-per-use!
1 reply
0 recast
1 reaction
Katsuya
@kn
gpt3.5 taking 10x longer is surprising to me. Is it consistently that way? Going with smaller models for speed/price makes sense if that works for your use cases. In general though I think devs tend to spend too much time optimizing for these choices prematurely.
2 replies
0 recast
1 reaction
