thomas

Open any popular/trending Twitter post related to geopolitics or international affairs right now, and you’ll see dozens of replies along the lines of “@ grok, is this true?” or “@ grok, what are the implications for someone like me?”.

1/

Aviator, diver, Etherean, nexialist, PhD in aviation ∩ climate science | Host of /diving /ethfinance /eth-staker /geopolitics /science /singapore /thomas

It’s striking how fast LLMs have become arbiters of truth for so many consumers of social media. And while it’s refreshing to see people use such tools to critically parse, understand, and verify social media claims, it’s also placing enormous trust in those models. Trust which can, and will, be manipulated.

2/

I’d qualify this process as being mostly domain-agnostic and unbiased, given (i) the significant overlap in the training datasets across models and (ii) the limited investment in model alignment beyond table stakes techniques (guardrails, RLHF, temperature adjustments to limit hallucinations, etc).

4/

Once technical progress starts plateauing, however, I expect the focus to shift to the balkanization and politicization of LLM rule sets as the next evolutionary phase in their competitive differentiation.

5/

We’ve already seen a mild case of this with search engines, of course — Google doesn’t provide the same results whether you browse from Washington, DC or Beijing, China. Largely to satisfy national preferences or regulations.

6/

This schism will happen in LLMs too, and I consider it plausible that we’re headed toward a “multipolar world” comprised of blue-pill models, red-pill models, a few niche middle-of-the-road aggregators that claim neutrality, and some specialized models (e.g., for coding or scholarly research).

7/

What is evident to me is that someone like Musk didn’t invest $44B in Twitter just to have Grok roam free and spout out only “the most likely tokens” from its unadulterated training set. The opportunity cost of not shaping the message is becoming too great as more and more people rely on the model to DYOR.

8/

The same evolution will reflexively apply to other leading models, with ChatGPT as the putative antagonist to Grok. A couple of years from now, you could probably situate users on the political compass based on which LLM they prefer, just like with their newspaper subscriptions in the last decade.

9/

And if we thought the echo chambers of the 2010s were nefarious enough to the health of informed democracies, wait until people source free, instant, and qualified validation of their fringe ideas from mainstream LLMs. Belief reinforcement will be cheaper and faster than ever.

10/

This shift will also mark an ironic turning point in AI’s young history — after years of being trained on our thoughts, it is us who will start being trained on AI’s thoughts, no matter how perniciously calibrated they may be.

11/

Until now, the focus of LLM development has been primarily to outperform the competition on a technical level (larger context windows, chain of reasoning, increased specialization, etc). 

3/

not-knowing ✹ 🍕-fan ✹ productive discomfort + idk ✹ Marseille / Singapore / Auvergne ✹ The Uncertainty Mindset (2020) ✹ now: vaughntan.org/now-ish

I suspect it’s going to be much, much more so

is already being manipulated tacitly through system design

It is being manipulated already. Those models have been trained with a clear bias. It is happening right now, so things aren't going to get any better soon.