Misinformation is on a massive rise, and social media platforms are scrambling to figure out what to do about it.

In The Atlantic today, my Harvard colleague Jesse Shapiro and I offer a surprising answer: 👇 👀

@a16zcrypto • Harvard/HBS • 🧩 • QED

We're not saying that misinformation isn't a problem, but rather it's one that platforms aren't well-positioned to solve – and we have a suggestion for what they should do instead.

We're not saying that misinformation isn't a problem, but rather it's one that platforms aren't well-positioned to solve – and we have a suggestion for what they should do instead. https://www.theatlantic.com/technology/archive/2024/03/social-media-misinformation-moderation/677855/

Misinformation is everywhere these days, and with the increase in global conflict and the upcoming US election – not to mention the increasing ease of producing deepfakes using AI – the problem is only going to get worse.

But while platforms can block, flag, or mute content that they judge to be false, doing that doesn't necessarily stop users from believing it. 

In fact, because many lies are believed by those inclined to distrust the “establishment,” blocking or flagging false claims can even make things worse.

What do we mean by that? Here's an example: 

Back in 2020, then-president Trump posted a now-infamous message about election fraud, telling readers to “be there” on January 6.

If you visit that post on Facebook today, you'll see an annotation, sourced from The Bipartisan Policy Center, that “the US has laws, procedures, and established institutions to ensure the integrity of our elections.”

But does anyone seriously believe that the people storming the Capitol on January 6 would be persuaded that Biden won just because the Bipartisan Policy Center told Facebook that everything was above-board? If anything, they might have thought it was further evidence of a cover-up.

Jesse's and my research, which we detail further in a recent NBER working paper, shows that this challenge is intrinsic:

Unless users trust the platform's motivations and its process, any action by the platform can look like evidence of something it is not – and this means attempts to moderate misinformation can backfire. https://www.nber.org/papers/w32156

Our analysis shows that if users trust the platform to both know and do what's right – and the platform truly does know what's true and what isn't – then it's possible to successfully eliminate misinformation.

The logic is simple: If users believe the platform is benevolent and all-knowing, then if something is blocked or flagged, it must be false; and conversely if information is let through, it must be true.

But of course, users often don't trust platforms – they haven't for years!

But of course, users often don't trust platforms – they haven't for years! https://www.edelman.com/sites/g/files/aatuss191/files/2024-02/2024%20Edelman%20Trust%20Barometer%20Global%20Report_FINAL.pdf

And when users don't trust a platform, even well-meaning attempts to make things better can make things worse. (Indeed, when platforms seem to be taking sides, that can add fuel to the very fire they are trying to put out.)

Does this mean that content moderation is always counterproductive? 

Far from it. 

Our analysis also shows that content moderation can be very effective when it blocks information that can be used to do something harmful.

Going back to Trump's post, imagine that the platform had simply made it much harder for him to communicate the date (January 6) and place (Washington, D.C.) for supporters to gather.

Blocking that information wouldn't have prevented users from believing that the election was stolen – but by making it harder to coordinate where and when to go, it still might have helped slow the momentum of the eventual insurrection, thus limiting the post's real-world harms.

Unlike with misinformation, blocking information that enables harm can work even if users don't trust the platform's motives. That's because it's the information itself that enables the harm, blocking it blocks the harm as well.

A similar logic extends to other kinds of harmful content, like doxxing or hate speech. There, it's the content itself – not the beliefs it encourages – that is the root of the harm, and platforms can indeed successfully moderate it.

There are clear challenges and downsides to tech companies deciding what content is and is not directly harmful. But platforms already routinely make judgments about this, and often language or context makes it possible to do algorithmically.

And, if platforms accept our reasoning, they can divert resources from the task of deciding what is true, toward the still hard, but more pragmatic task, of determining what enables harm – and screening that harmful content out.

The bottom line: Even though misinformation is a huge problem, it's not one that content moderation can solve. Platforms can help keep us safer by focusing on what content moderation can do, and giving up on what it can't.

You can read the full article here: https://www.theatlantic.com/technology/archive/2024/03/social-media-misinformation-moderation/677855/

And if you found this interesting, please re-share and/or let us know your thoughts. (If you disagree, we'd love to hear that too – please tell us why!)

And if you found this interesting, please re-share and/or let us know your thoughts. (If you disagree, we'd love to hear that too – please tell us why!) https://warpcast.com/skominers/0x9f52bc66

Platforms should give up.

𝑊ℎ𝑎𝑡!?, you're thinking – but before you throw tomatoes 🍅 or quote this post to dunk on it, read on. 🧵

Great thread and article, we’re definitely at a tipping point, speaking of tipping

400 $degen

Digital Money Economist at the IMF. Interested in Microeconomic Theory and Blockchains/Cryptocurrencies.

Opinions are my own.

Happy to see that you finished up that project with an awesome result.