bettik

Going from generic responses to high quality answers with an LLM / AI agent can be very challenging and time consuming due to the extremely high leverage an input has on the output.

Clean data + accurate context + key prompt terminology increases answer quality by orders of magnitude. Those who spend the time and capital on maximizing response quality like this will have major advantages over competitors.

Changing a single field name in our LLM response schema improved accuracy from 4.5% to 95% on GSM8k.

The fix was simple: going from final_choice to final_answer. Turns out our model was returning a multiple-choice index instead of the actual answer.

If you're working with…

jason liu

Going from generic responses to high quality answers with an LLM / AI agent can be very challenging and time consuming due to the extremely high leverage an input has on the output.

Clean data + accurate context + key prompt terminology increases answer quality by orders of magnitude. Those who spend the time and capital on maximizing response quality like this will have major advantages over competitors.
https://x.com/jxnlco/status/1856396997785182298