Curious how do folks think about picking the right model for their use case? Ie, when use GPT-4o, Llama 3, Gemini 1.5, DeepSeek or other? How does pricing fit into consideration?

🇧🇷🇺🇸-  Book: Making Things Think: https://holloway.com/mtt. Investor in Wander, Carry, Footprint, Merkle Manufactory (Farcaster), Dynamic, Paragraph.

Building /bright-moments - minting onchain art IRL | /purple #15 | FID 129 #magikarp

Founder of /harecrypta community
https://twitter.com/Rencrypta

Founder ChainVine (Backed by Slow, Multicoin, Hustle Fund, and more) | former COO Setter (20M from Sequoia & NFX, sold Thumbtack) | FID #1277.

Pricing and performance are the 2 metrics that we used to deploy a model into production. In some cases, pricing is the most important factor. But also there is the license to take into consideration. The models that you listed are just the top ones. But, for example, I use a different model that is better for our need

tagging just a couple folks would love to hear from based on activity in this channel : @giu, @renatov, @phil

Funtion and performance, pricing is relative since if it will take you 10 prompts to get to the desired results with one model vs 1 in the other. You might up spending the same.

Pricing is the main reason not to use gpt4 or anthropic.
But depends on your use case if pricing will become a factor.

A few months ago gpt4 was the best at everything. Now there are other models on par or better.

Depends on how important marginally better results are.

I default to "it doesnt matter unless it does, and then it really does"

I start with requirements on latency. If I can tolerate high latency I start with large models and work down. If I can't tolerate high latency I start with the small models and work up until I get something that is giving me 85%+ success rate on typical questions.