A study of ISIS supporter activity on Twitter suggests the tactic of repeat account suspensions can take a toll on the spread of extremist propaganda on the platform.
Tackling terrorist appropriation of mainstream social media platforms has become a growing problem for the likes of Twitter, Facebook and Google’s YouTube in recent years as extremist groups have adopted Western technology tools to spread their messages — cranking up the political pressure for tech companies to find solutions to counter the extremist propaganda that is being spread on their platforms.
The research paper, produced by a fellow with the Program on Extremism at George Washington University, looked specifically at Twitter and how repeat account suspensions of a list of English-speaking ISIS adherents played out and affected their output and follower counts over time.
The study showed the overall size of the ISIS network remaining roughly the same over the study period, as suspended individuals respawned their accounts, but it found that the reach of individual accounts was diminished as repeat suspensions took a toll.
The researchers based their study on only one list of English tweeting pro-ISIS accounts maintained by a specific user (self labeled as ‘Baqiya Shoutout’), looking at social media activity between June and October last year. So there are plenty of caveats on the scope of the data, as the researchers themselves note.
But the implication — that systematic account suspensions can reduce the volume of extremist propaganda — is heartening nonetheless, offering a counterweight to the argument that fighting extremist propaganda on mainstream platforms via account suspensions is futile because new accounts can just be recreated (i.e. it’s not so futile if you are diminishing your target’s vocal chords).
Among their findings the researchers note that —
- Over time, individual users who repeatedly created new accounts after being suspended suffered devastating reductions in their follower counts.
- Network and individual declines persisted even when suspension pressure eased, suggesting that suspensions diminish activity in ways that extend beyond the simple removal of accounts.
- The amount of pro-ISIS content available on Twitter was also limited by suspensions, since all of a user’s tweets are typically deleted when his or her account is suspended.
Discussing four individual users they examined in detail, the researchers write:
We examined four cases in detail — users designated A, B, C, and D—who returned repeatedly under the same display name and with very similar Twitter profile information. These four users had the highest number of user IDs associated with their display names, meaning they were the most often suspended and most likely to return… We found suspensions typically had a very significant detrimental effect on these repeat offenders, shrinking both the size of their networks and the pace of their activity. Each user had a different trajectory, with some recovering more robustly than others, but all showed consistent declines over the monitored period.
The fact that suspensions reduce key metrics in the period immediately following suspension is not surprising in itself, but we found that the depressive effects of suspension often continued even after an account returned and was not immediately re-suspended. Returning accounts rarely reached their previous heights, even when the pressure of suspension was removed. Users B and C, for example, returned from suspension and then remained online for several consecutive days without approaching their earlier levels of performance.
They also found that pro-ISIS Twitter users deployed a range of countermeasures in response to repeat suspensions, such as using tools to automatically recreate followers lists to quickly get up and running again, or using a system of account donation — whereby ISIS supporters create Twitter accounts in bulk and then donate accounts to other supporters on request.
The study also flags up the tactic of extremist tweeters switching to alternative platforms — including secure messaging app Telegram — although the researchers assert that, at present, other platforms are used “primarily as backups for Twitter”.
That said, they note Telegram emerging as a “favored alternative” to Twitter for ISIS supporters, saying this was due in part to the launch last September of Telegram’s Channels feature — which lets users broadcast messages to an unlimited number of followers.
They write:
Within four days of Telegram’s announcement of this feature, the Islamic State launched an Arabic language channel to distribute its official media releases.
In the weeks following the launch of its Arabic channel, the Islamic State announced official Telegram channels in English, French, German, Indonesian, Bosnian, Bengali, Turkish, Kurdish, Urdu, Swedish, and Russian. These official channels have been used as the initial point of distribution for propaganda releases. Once released on Telegram, links to the propaganda content are subsequently distributed on Twitter.
Telegram’s own counter move to ISIS supporters making use of its Channels features was to block a swathe of pro-ISIS accounts from using its public channels. It also said it would be taking steps to make it easier for users to report “objectionable public content”.
In response to those blocks, the researchers note that ISIS supporters created new, public Telegram channels in conjunction with chat groups and supergroups — which, unlike the public broadcast Channels, require users to join the chat groups to view the content. Chat groups are limited to 200 members, while supergroups can have up to 1,000 members.
But such groups, while large, do not offer the same unlimited propaganda potential of Twitter which offers extremists the promise of reaching and recruiting from outside their existing pool of supporters. And which, evidently, keeps them trying to come back to Twitter.
You can read the full research paper here.
Twitter declined to comment on the study but in recent times it has been stepping up its efforts to combat the spread of hate speech on its platform. And in a blog post earlier this month specifically on combating extremism it flagged up its “ongoing work in this area”, writing:
Since the middle of 2015 alone, we’ve suspended over 125,000 accounts for threatening or promoting terrorist acts, primarily related to ISIS.
Our efforts have not stopped there. We have increased the size of the teams that review reports, reducing our response time significantly. We also look into other accounts similar to those reported and leverage proprietary spam-fighting tools to surface other potentially violating accounts for review by our agents. We have already seen results, including an increase in account suspensions and this type of activity shifting off of Twitter.