I'm no expert and I'm biased because of my current work but fine-tuning (especially LoRA) is one of the most powerful and reliable ways to get LLMs to fit your custom needs.
Better than prompt engineering and even embeddings.

That's all I can say.

The open web is dead, long live the open web. ⌐◨-◨

Building /warpcast and /farcaster

— gabrielayuso.com

"Creative Technology" 🤌🏽

@ google creative lab

Owner of /makerspace - come build with us!

My pipeline has been:
- test wild prompts on largest (340B+) model I can find
- find best outputs and test on 24Bs
- generate 100+ examples and see if I can get it tuned on something much smaller
And yes, LoRA has proven to be incredible for a lot of my use cases

Have you considered arguments presented here :

Taking Design Contracts | Talking about Tools for Thought | Founder metamind.network, Medium/Product Designer, Writing at www.abhishek1point0.com, 📍Bangalore

Agree with this:

"Given the upfront time and computational resources required for fine-tuning, we’d recommend starting with this base first (along with the supplemental techniques mentioned) and seeing how far you get with it!"

It's good to start simple.

Have you considered arguments presented here :

https://warpcast.com/abhishek1point0/0x2c431a