AI说服力研究：信息密度是关键，但现实注意力是瓶颈

Oliver Kemp for Transformer

News of new AI capabilities is now a steady drumbeat accompanying our lives.

Another benchmark broken by the latest model, another AI-generated video that you cannot tell from real footage, or another paper announcing how AI systems are getting pretty good at hacking their way around the internet or passing as human in extended conversations. There is no particular need to invoke transformative AI or superintelligence to agree that the most recent AI models can now do things long assumed to be decades away.

As an academic studying the societal impacts of AI at Oxford, one area that I pay a lot of attention to — and where models have become really very good it seems — is persuasion: the act of changing people’s attitudes, beliefs and even their actions with nothing more than a conversation. Such persuasion is about changing people’s minds by giving them reasons they can inspect (and of course reject), and is quite different from manipulation, which forgoes that process by deceiving people — even though the two are often and easily conflated.

Cut through the noise.

That AI systems can be persuasive is… pretty big news. If someone had told me five years ago that chatbots could be better than seasoned campaigners at getting people to part with their money, as a recent preprint found, I would have been pretty skeptical. And yet, that is exactly what you can do with the latest models — at least in an experimental setting.

These results have, unsurprisingly, prompted people to come up with all sorts of ideas, as well as quite a bit of excitement and fear. Platforms, governments, or public bodies could move people away from harmful conspiracies or could nudge them toward choices that serve them better, for example, quitting smoking. But, of course, every light side comes with a dark one: persuasive systems could allow politicians to steer how people vote, talk someone into joining a cult, make it easier to scam and swindle people out of money, encourage decisions that put their health at risk, and much more. The most apocalyptic of these scenarios imagine a new dawn of uber-persuasive machines against which we pitiful mortals stand no chance.

But is it really game over? I don’t think so. Persuasion is possible, but I doubt that it will make as much of a difference as some claim. Persuasiveness is actually the “easy” part. What is difficult is getting hold of someone’s attention in the first place. Shifting a view once someone has one is harder still. There are, in other words, a range of bottlenecks that will limit the persuasive power of these systems for the foreseeable future — for good and ill.

Persuasion through AI is real

So, what do we actually know? Our best evidence for AI’s persuasiveness comes from experiments in which researchers pay people to sit down with a chatbot and measure if their views have shifted.

In probably the largest public study so far of its kind, conducted by researchers from the UK’s AI Security Institute (AISI) and elsewhere, more than 42,000 UK participants were asked to argue about various political topics with 19 different language models on several hundred issues. Afterward, the research team measured how far people’s attitudes had moved compared to a control group. The answer was around 10 points on average on a scale running from zero to 100. That’s roughly equivalent to walking into a bar mildly opposed to the idea of beer, having a chat with the barman, and then walking out feeling lukewarm about beer, but overall still preferring white wine. That conversation was more effective than single, static messages — the kind of argument or claim you regularly see in a social media post or TV ad — by about 41% to 52% depending on the AI model involved.

While having a bigger model helped, post-training — the fine-tuning AI firms do once a model has been built to shape its behavior — towards better persuasion mattered even more. So did a specific persuasion strategy: prompting the model to aim for information density. Models that rammed more facts into their answers were better at persuading people, while personalizing answers to people’s attitudes and demographics — a recurrent worry ever since Cambridge Analytica — did not do very much.

Meanwhile, in a study published last year in Nature, participants were randomly picked to talk to an AI that supported one of the leading candidates during election campaigns in the US, Canada and Poland. The study found that AI conversations had a stronger effect on changing people’s candidate preferences than traditional video ads — and they found a similar mechanism as the AISI study, with relevant facts and evidence being the thing that persuaded people.

The immediate implications of such a finding, beyond the fact that AI models can do something long thought to be the prerogative of humans, are twofold. For one, persuasion is now, at least in theory, dirt cheap. You no longer need as many messy and costly humans to do it. In a recent paper on AI in political campaigns, Zhongren Chen and colleagues estimated that once you have accounted for the price of actually reaching people, LLM-based persuasion costs between $48 and $75 per persuaded voter compared with $100 for traditional campaign methods. Instead, you have persuasion on tap, anytime, in any language and at whatever scale someone is willing or able to pay for — quite different from the persuasion happening where people simply interact with a model for things like work or pleasure.

The AISI study also had another significant finding: Those models tuned hardest for persuasion also made more false claims as part of the process — in the most persuasive setting, nearly a third of the model’s claims were inaccurate, something that is obviously no problem if you are a scammer or propagandist, but does matter quite a lot if you have good intentions. However, the authors did note that inaccuracy seemed to be a byproduct of greater information density, not a cause of persuasion. Telling a model to make things up did not make it more persuasive.

Why the lab isn’t the real world

The AISI study sparked a big brouhaha when it first came out, but experts — including some of the study’s authors — have since pointed out that the findings need to be treated with a pinch of salt when applied to the real world. The biggest bottleneck is deceptively simple: for such persuasion to work, you need to get people to pay attention.

Experiments solve this via forced exposure: people are being paid to engage in back-and-forth conversations, otherwise it would be difficult for researchers to reliably detect effects. In the AISI studies, people had to engage in at least two conversation turns to receive an incentive, although on average participants talked to the chatbot for about seven turns (or roughly nine minutes). Yet, this is a form of exposure that you just cannot assume in the real world. In the wild, people’s attention is voluntary and scarce.

To see why this is, just remember how little time is left (in your life) to actively pay attention, accounting for the factors that shape the humdrum of our day-to-day existence.