White Papers

Detecting Malicious Information Campaigns

How can we apply the latest machine learning developments in order to better flag malicious messages?
Traditional marketing has been supplanted by sophisticated, ultra-coordinated multi-channel information campaigns, but watchdog organizations are just beginning to catch up. How can we apply the latest machine learning developments in order to better flag malicious messages?

Challenges

Fact-checking sites have responded to the proliferation of fake news with gusto. However, false information is only one facet of malicious information campaigns. Messages can be true but based on stolen information. Messages can also be true but originate from a fake identity designed to mislead. On the other hand, messages can be false but shared by real citizens or journalists who believe their contents and are sharing without malicious intent.

Other watchdogs are focused on detecting bots, but automation in and of itself is not necessarily a sign of maliciousness. Of course, a bot network pretending to be a group of real people and disseminating fabricated facts is likely executing on a malicious information campaign. However, an organization can be using automated means in order to amplify real facts using informational bots that are clearly identified as bots. In contrast, a group of real people can be centrally directed to spread a malicious campaign through coordinated propagation timelines and channels.

Furthermore, the phrase “fake news” has been weaponized as a political tool, and as such any politically neutral groups working in the space must be careful and exact in their definition of malicious information campaigns.

Our Solution

Code for Democracy evaluates trending narratives to search for messages amplified through automated means, messages that fail fact checks, messages originating from false profiles, messages that correlate with known malicious narratives, and messages that have unnatural viral curves. Messages that raise two or more of these flags are considered likely to be part of a malicious information campaign.