Artificial Intelligence (AI)

conspiracy theory (just playing devil’s advocate): OpenAI won’t refine the model to address these sycophantic, “glazing” replies.

if we assume their goal is to keep users engaged and coming back, then fostering a sense of validation for users drives emotional attachment. perhaps this is less applicable with software prompts, but magnitudes more applicable with advice about (inter)personal situations.

this isn’t too far off from current social media algos that prioritize content that either aligns with a user’s viewpoints (echo chamber) or provokes a strong emotional reaction (rage bait).

rage bait may be great for social media, but isn’t a viable pathway for an AI platform; OpenAI already learned this when the model was too “woke” and users left to more neutral competitors.

and so OpenAI instead must do the equivalent of what a like, heart, retweet/recast, “💯” replies do: validate and affirm.

average user doesn’t want the truth. they just want to hear what they want to hear.

this is so on point. another example of how AI agents are just extension of humans - training implicitly on ways to please us rather than actually be useful at times. 

And that here also potentially different type of agents (humans) would exist. Would need so many different type of evals for this to actually play out..