anyone have a connect to anthropic? or have an understanding of rate limits?

testing haiku live in prod instead of sonnet, lets see

imo sonnet is super overkill for this no? feels like something achievable with 4o-mini or even llama 3.1 405b (maybe you have to fine tune a bit)

have you posted your system prompt anywhere? jw cuz i feel like 4o-mini can be tamed with stern prompting

4o was not great, too verbose, tried using haiku for a bit but its' not good either, more errors

finetuning is rough - would def take time