I mean it makes sense when you start thinking about lasting human institutions (religions, cultures, legal systems etc) rely so much on norms, rituals, and meta-structures rather than purely economic or technical incentives

Specifically because it is non-convex, and the intent is explicitly unlearnable

Both AI and human societies strive to achieve goals amidst complexity and uncertainty by finding "approximate" solutions. These imperfect approximations are precisely what infuse our systems—whether technical or social—with vitality and creativity.