moneytrade
@moneytrade
2 replies
13 recasts
78 reactions
1 reply
13 recasts
61 reactions
6 replies
37 recasts
267 reactions
8 replies
19 recasts
108 reactions
on agent dev: sometimes a feature or bug fix is just adding another clause to the prompt, or fixing grammar.
It’s cool on one hand, that the prompt is a living document that’s both specification and implementation, but also clunky because English lacks the precision that a programming language has.
Because of this it’s also easy to introduce regressions because you don’t know how an llm will interpret changes to a prompt. Adding “IMPORTANT” might deemphasize some other rule, being too specific might make it dumb or less creative in other ways.
In code it’s deterministic, with llms it’s probabilistic.
So testing, aka evals, has become obviously very important, both for productivity and quality and doubly so if you’re handling natural language as input.
The actual agent code itself is quite trivial, prompts and functions, but having it work consistently and optimally for your input set is the bulk of the work, I think. 11 replies
12 recasts
65 reactions
6 replies
18 recasts
50 reactions
69 replies
27 recasts
170 reactions
2 replies
1 recast
14 reactions
7 replies
18 recasts
102 reactions
6 replies
10 recasts
59 reactions
4 replies
9 recasts
25 reactions
1 reply
19 recasts
63 reactions
0 reply
12 recasts
21 reactions
1 reply
3 recasts
30 reactions
67 replies
81 recasts
419 reactions
15 replies
5 recasts
117 reactions
1 reply
2 recasts
14 reactions
7 replies
18 recasts
97 reactions
6 replies
35 recasts
131 reactions
6 replies
30 recasts
79 reactions
