The main reason true (but not necessarily “good”) engineers flail so badly and turn into crackpots when they try to do philosophy or physics is they have an agency drive hardwired into firmware so strongly they cannot help but see the world exclusively in terms of ways it is “wrong” and can be “improved”

Blogger at ribbonfarm.com, running summerofprotocols.com, host of /gloom ☀️

Though I’m an engineer by training I’m mediocre at it aptitude-wise, but more importantly I’m not particularly “true” to the spirit of it. Seeing the world in improvability terms is *not* natural for me. I think I’m equally mediocre at physics and philosophy but more true to those orientations.

The orientation of philosophy is essentialism. What is the true nature of reality beneath the layers of words, concepts and constructs we use to attend to it? Again, not an improvability orientation. It’s less broadly curious than physics but cuts deeper once physics exhausts curiosity.

Example: 
take the idea of chirality (“handedness” or mirror-asymmetry). Engineers don’t give a shit about it. Use symmetry when it is useful for balance or aesthetics. Use handedness where necessary for orientability of parts or unavoidable like in screw threads.

Physics goes *much* deeper. Symmetry is fundamental to organizing the curiosities of physics. Mirror-asymmetry is catnip. Particle spin, CPT-conservation, polarity of molecules….

Philosophy goes deepest. I knew a philosopher once who was trying hard to analyze the very essence of chirality.

Mathematics is orthogonal to these three. There are engineering, physics, and philosophy styles of doing math. I’m pretty mediocre at math too, but it’s a tell that I’m most attracted to “philosophy math.” The discovery/invention of the hat aperiodic monotile fascinates me.

Point of this thread is that AI is an engineering development that severely stresses the philosophical and physics aptitudes of true engineers. They throw words like “agency” around (or worse “agentic”. — a crackpot word) with zero curiosity about the substance or poking at its essence.

They typically reduce agency to cost functions and optimization or equally nebulous words like “goal-oriented” or “intention” without digging further. It’s like deciding chirality is only about screw threads.

Perhaps most revealing: they are obsessed to the point of paranoia with (self-) improvability of AIs

What used to be called “strong AI” is really “engineering AI” — indifferent, incurious, and oblivious to aspects of reality that cannot be engaged in terms of “improvability” frames. There are aspects of AI that are invisible In improvability frames just like distant stars and galaxies.

For eg while many engineering-AI types have heard of Chalmers’ “hard problem of consciousness” (a precise philosophy of mind question), they usually aren’t interested in it, and if they are, genuinely don’t seem to get it. It’s not visible due to their firmware.

This is likely the biggest, most fascinating development of our lifetimes and it would be a real pity if you could only engage it in one fixed way. If aliens landed, would you limit yourself to the question “are they going to anal probe us?”

That’s what many engineering-AI takes sound like to me.

I will admit I have often wondered whether strong-AI types welded to improvability perspectives are in fact p-zombies. There is nothing it is like to be them, which is partly why they can project so strongly on AIs and fear them. They see AIs not as superintelligence but as superengineers reducing them to non-being.

Modern AIs get *vastly* more interesting once you start seeing them in physics and philosophy frames

What is the physics-nature and philosophical essence of agency? If you answer “utility maxing” you haven’t actually thought about it. If you answer “paperclip maxing” you don’t even care about the question.

Aside: Another group that has an engineering orientation to AI is so-called AI-ethicists. Despite the label they don’t care about any ethics question in a philosophy sense (“can an AI feel pain” is the obvious big one they entirely ignore). Only engineering practical, unreconstructed human interests into AIs.

I’m not saying the question of (for eg) racial bias in training data isn’t important. It just isn’t very interesting from a physics or philosophy frame. It’s an engineering concern on par with using GPUs more efficiently. Adequately addressable with a competent improvability frame.

This thread should not be taken as criticism from a traitor-engineer. I am as interested in engineering AI as ever. I made an ELIZA-style therapy chatbot for my high school computer science final project in 1992. My PhD in 2003 (pre-deep-learning) was about integrating GOFAI classical planning into control theory.

But as a mediocre engineer not locked into an improvability frame and capable of being curious about things I can’t control (I almost went to grad school for astrophysics) and interested in philosophical essences, I am disappointed by the self-limiting narrow perspective so many engineers are bringing to AI.

The orientation of physics is sheer curiosity. You want to know how things work regardless of whether you can improve them by design. Since 99.9999% of the universe is not improvable engineers are typically bored by it. It’s rare for an engineer to get into astronomy for eg though many like space tech.