Researchers warn: AI can repackage lies as ‘truth’

As artificial intelligence (AI) becomes an increasingly important part of our everyday lives, jobs, industries, healthcare, and information, researchers warn that generative AI can produce persuasive content and repackage lies as ‘truth.’

Indeed, AI can combine generated content and the so-called ‘CopyPasta’ campaigns that take advantage of the ‘repetitive truth’ effect by repeating the same text until it seems more likely to be true to those who encounter it many times, according to a study in PNAS Nexus published on July 22.

AIPasta: AI meets CopyPasta

Specifically, the study’s authors, Saloni Dash and colleagues, have explored how these two strategies can be combined into what they dubbed ‘AIPasta.’ In these cases, AI can produce many slightly different versions of the same message, giving the impression that various people have shared it and that it is therefore accurate.

To demonstrate the effectiveness of such campaigns, researchers have used both CopyPasta and AIPasta methods to create messaging around particular conspiracy theories. According to their results, exposure to AIPasta – but not CopyPasta – increased the perception that there was broad consensus that a narrative was true.

Furthermore, AIPasta didn’t reduce sharing intent, unlike CopyPasta, which people were less likely to share. Moreover, the AIPasta generated for the study managed to bypass AI-text detectors, which suggests it would be harder to remove from social media platforms than CopyPasta. In other words, this may further increase its effectiveness in comparison to CopyPasta.

As the authors wrote, “AIPasta is easy to produce and demonstrates characteristics that offer strategic advantages in information operations (such as the anticipated difficulty for machines and humans to recognize the content as verbatim or duplicate from a single source).”

Meanwhile, one of the founding fathers of AI, Geoffrey Hinton – often referred to as the ‘Godfather of AI’ – has expressed deep concern that AI may soon outpace human intelligence in ways we can’t imagine or control, stating bluntly that we may be facing the end unless we do something about it.