Hot take: If models can't be a moat, then perhaps hidden agents can be?

We've been practicing witchcraft incantations with system prompts and agent loops to squeeze out performance from the models we use. Things they couldn't do without additional context token repricing, or without additional steps of "reasoning".

Are we headed to a world where only open models give "raw" access to make our own clever system prompts and agents, and AI platform providers continue to improve behind hidden agent systems like o1? 🍓

A doodler and computerer. I like permissive/permissionless open source, smart contracts, p2p systems, room-scale VR, and NixOS.

shazow.net

Currently: WhatsABI

Yes, I’ve been thinking of it as the agent “secret sauce” which is anything that determines outward behavior, from super cutting edge stuff to just better system prompts or some proprietary fine tuning data.

It’s easy to copy traditional UX. It’s going to be a lot harder to copy intelligence-based UX, which is determined by agent behavior.

I think at the end of the day this ends up looking just like any other SaaS: "we found a workflow that was messy and inefficient, so we distilled it to 6 handholding steps and increased output by 140%" or whatever.

This but the 6 steps are now a hidden agent instead of a hidden state machine with some whistles.