Content
@
0 reply
0 recast
0 reaction
Greg
@greg
I haven't been following the AI space very closely, especially outside of OpenAI stuff. How far behind are open source alternatives? I've heard about stuff like llama2 supposedly being decent for chat but I'm assuming the agents/function calling and whatnot is a different story?
7 replies
1 recast
13 reactions
Warpmaster General
@my
It's largely a question of UX more than capability. ChatGPT(+API) is turnkey, cheap, convenient. If you want to roll your own via LLaMA, Langchain, et al, you're looking down the barrel of a week or two just to get it running, let alone fine tuning it to a level where its output is actually usable (not even useful).
1 reply
0 recast
3 reactions
Greg
@greg
Makes sense. Just out of curiosity, assume I was going to spend time getting everything set up - would it even be possible to get close to OpenAI's current performance? Like are the base LLM's good enough?
3 replies
0 recast
1 reaction
vincent
@pixel
Perplexity provides open-source LLM API: https://docs.perplexity.ai/docs Also check out replicate, they serve some LLM API: replicate.com I follow open-source LLM somewhat, best way to follow is on X (I can share my private list if interested), and crawl through HuggingFace.
2 replies
0 recast
2 reactions
Frank
@deboboy
Mistral-7B is fast, lightweight, and generally accurate [your mileage will vary; depends on your prompts and retrieval evaluation]. Other open source LLMs on Hugging Face or other hosting platforms perform similarly. But unfortunately, they all lack OpenAI-like functions support.
0 reply
0 recast
1 reaction
Warpmaster General
@my
The annoying answer is "it depends." OpenAI is great bc you can basically throw anything at it and it can make sense of it. The open source ones, you have to choose if you want an LLM that's good at coding, or one that's good at poetry (none are good at that). Severely limiting outside of v specific use cases imo.
0 reply
0 recast
0 reaction
