New Claude Haiku 3.5 is good at coding, but this price increase vs Haiku 3 is sad. 😞

New Claude 3.5 Haiku comes in 4th on the aider code editing leaderboard with 75%. This is just behind the old 3.5 Sonnet 06/20.

84% Sonnet 10/22
80% o1-preview
77% Sonnet 06/20
75% Claude 3.5 Haiku
47% Claude 3 Haiku

https://t.co/mBVaUPGHPl https://t.co/1jAuSEpMW3

Paul Gauthier

summary_large_image

Claude 3.5 Haiku is now available on our API, Amazon Bedrock, and Google Cloud's Vertex AI.

Haiku is fast and particularly strong at coding. It outperforms state-of-the-art models—including GPT-4o—on SWE-bench Verified, which measures how models solve real software issues. https://t.co/SR0i9W7W0T

During final testing, Haiku surpassed Claude 3 Opus, our previous flagship model, on many benchmarks—at a fraction of the cost.

As a result, we've increased pricing for Claude 3.5 Haiku to reflect its increase in intelligence: https://t.co/UWTd1RDRVE.

Anthropic

New Claude Haiku 3.5 is good at coding, but this price increase vs Haiku 3 is sad. 😞 

https://x.com/paulgauthier/status/1853532441077850319?s=46&t=jQHN-g4HjwTKZe7EhUgxHg

https://x.com/anthropicai/status/1853498270724542658?s=46&t=jQHN-g4HjwTKZe7EhUgxHg

I’m a builder looking for the next challenge. 

Bro I don’t even think it’s actually much better than Opus