Part Two — How the Internet Research Agency regenerated on Twitter after its accounts were suspended

On October 17, Twitter released an archive of over ten million tweets posted by accounts from 2013 through 2018. Of the total, over nine million tweets were attributable to 3,800 accounts affiliated with the Internet Research Agency, also known as Russia’s infamous St. Petersburg troll factory. Over one million tweets were attributable to 770 accounts, originating from Iran.

Each set is included in the same archive; however, because the actors and activity were separate, our analysis was conducted accordingly.

In an effort to promote shared understanding of the vulnerabilities exploited by various types on online influence operations, as well as social media’s role in democracy, @DFRlab had a brief advance opportunity to analyze the nearly complete archive.

What sets this archive apart is Twitter’s consolidation and release of all accounts the platform maintains high confidence are associated with the Russian Internet Research Agency and separate Iranian accounts.

In Part Two of our series, @DFRLab took a deep dive into the content from the IRA with a specific focus on its second wave of accounts, which were active after 2017.

---

The Russian troll operation which was exposed for targeting the United States in the fall of 2017 did not cease operating after its exposure. Instead, it set up a new batch of troll accounts that posed as Americans to post polarizing content from both ends of the political spectrum.

The “Internet Research Agency” (IRA) in St. Petersburg, Russia, ran thousands of fake Twitter accounts and posted some nine million tweets between 2012 and September 2017. Some accounts targeted the 2016 presidential election, while others were aimed at inflaming partisan anger. Twitter suspended those accounts in September 2017, after the operation was exposed by Russian journalists.

After the exposure and mass suspension, the “troll farm” continued to post inflammatory and hyper-partisan content from over 1,000 more recently created accounts. These remained active until mid-June 2018, when they were exposed and taken offline.

The second wave of Russian troll farm accounts was an evolutionary step from the first. They maintained a focus on minority groups and sensitive political issues, in a matter which reached across platforms. They spread some content that was notably anti-Russian, which highlights a key assessment of disinformation. If successful in driving a country further apart rather than closer together internally, that country would be less able to enact anti-Russian policy.

The chief lesson is that the Russian attempt to spread division continued after its initial accounts were suspended. This second wave of troll accounts masqueraded as hyper-partisan activists. By the time it was shut down, it had begun targeting not only the 2018 midterm elections, but the 2020 presidential elections.

Given America’s online polarization, it would be unwise to assume that the takedown of this second front marked the end of the Russian-based information operation against the United States. American online communities remain vulnerable to further foreign influence campaigns, whether from Russia, Iran, or elsewhere.

Still Posting, More Slowly

For the purposes of this article, we will refer to the troll farm campaign that ran from 2014 to October 2017 as the “first wave,” and the campaign from November 1, 2017, onwards as the “second wave.” Some of the accounts active in the second wave were created in early 2017 but continued to post well into 2018.

According to the Twitter database, troll farm accounts posted over 43,000 tweets and retweets during the second wave — a rough average of 180 posts per day. This is a substantial volume of traffic, but pales into insignificance beside the roughly 9 million tweets posted during the first wave.

In one month alone, September 2017, Russian troll accounts in the first wave posted over 92,000 times — more than double the entire second-wave oeuvre. The dramatic slowdown shows the importance of large-scale action against troll accounts. Twitter’s suspension of over 2,500 Russian troll accounts in late 2017 disrupted the troll operation very significantly by suspending hundreds of its assets at the same time.

Make Troll Farms Great Again

Some of the accounts which made up the second wave of troll farm accounts masqueraded as hyper-partisan supporters of U.S. President Donald Trump, brandishing his 2016 election motto, “Make America Great Again” or MAGA.

This was in line with the first wave, which used the #MAGA hashtag massively — over 300,000 times in posts and biographies between 2015 and 2017 — according to the Twitter database.

One of the more active second-front MAGA accounts was @JohnCopper16, screen name “Marlboro Man,” which was created on September 4, 2017, and had a modest 4,966 followers by the time it was suspended in June 2018.

“Marlboro Man” left traces online, such as this archived #MAGA post.

In keeping with its assumed character, it posted hostile commentary about Muslims.

While “Marlboro Man” had under 5,000 followers, some were important amplifiers. The tweet shared by @speed32119 was also retweeted by Anne Coulter, a prominent Trump supporter and campaigner, and shared to her Facebook page.

It was not the first time Coulter had been taken in by an account from the troll factory. According to the Huffington Post, she retweeted one of the leading accounts from the first front, @TEN_GOP, at least nine times.

According to the Twitter archive, these were some of @JohnCopper16’s other posts:

“Dems are losers!

@SenSchumer and @realDonaldTrump are the winners!

We are finally taking OUR COUNTRY BACK!

Suck it, libs!”

(Posted on January 22, 2018)

“A MUST WATCH: Listen to one of the most powerful people worldwide give ENORMOUS praise to Donald Trump on the unbelievable job he’s done w/ the United States and the World.🌎”

(Posted on January 26, 2018)

“RT if you think that all Muslim-loving feminists must be sent to Saudi Arabia!

Practice what you preach, girls! Try to teach those people your marxist ideas!

They’ll really love that!

#ripjohnmahoney”

(Posted on February 6, 2018)

This post, from February 16, 2018, is likely to have elicited wry comments in the troll farm:

“Russians indicted today: 13

Illegal immigrants crossing Mexican border indicted today: 0

Anyway, I hope that all those Internet Research Agency f*ckers will be sent to gitmo.”

Importantly, it began commenting on the 2018 midterm elections as early as March 22, 2018, in polarizing tones.

“Just a friendly reminder to get involved in the 2018 Midterms.

They are motivated

They hate you

They hate your morals

They hate your 1A and 2A rights

They hate the Police

They hate the Military

They hate YOUR President”

Another apparently hyper-Conservative account created on the same day as @JohnCopper16 was @AmConVoice, screen name “Conservative Voice,” bio, “Your daily source of Conservative news, analysis, and opinions.”

Its activity lived up to the bio, as these two posts, preserved by open source researcher Josh Russell, illustrate.

It also shared memes from alt-right figures on inflammatory issues such as the #TakeAKnee controversy, according to online archival resource Twicsy.

Like other troll accounts before it, this one posted heartwarming patriotic messages, in an apparent attempt to ingratiate itself with the online community.

“I am #ThankfulFor the opportunity to live in the Greatest country on Earth!”

(Posted on November 23, 2017)

It interspersed them with pro-Trump messaging.

“RT if you agree: ‘Immigration is a privilege, not a right.’ Donald ‘delicate racist’ Trump.”

(Posted on January 13, 2018)

It also posted anti-liberal and racist comments.

“We have brave sons who are defending our Motherland abroad while liberal rats bring it to ruin.”

(Posted on January 12, 2018. Note the reference to the Motherland, a linguistic feature far more characteristic in Russian than American English.)

“People from Haiti and African countries move to the US for one and only reason. Their countries are sh*tholes and they don’t want to live there.”

(Posted on January 12, 2018)

As early as February 2, 2018, it posted messages of election fraud, looking ahead to the 2018 midterm elections.

“The only way the Democrats can win 101 GOP seats is to cheat like they always do with illegals & dead voters.”

Beyond the amplification by Coulter mentioned above, these accounts do not appear to have had large resonance. Their followings were low, and they only scored handfuls of retweets, unlike first-wave accounts such as @TEN_GOP, which amassed tens of thousands of followers, and could score over 25,000 retweets of a single tweet. They are important for the continued attempt they show to sow division, rather than their impact.

Looking Like Liberals

At least as much of the second wave’s efforts focused on anti-Trump liberal groups, especially the self-styled “Resistance” movement.

These included three accounts that posed as African American women. The best-preserved online was @wokeluisa, screen name “Luisa Haynes,” which was followed by over 50,000 accounts, but was following 57,000, suggesting either bot amplification or follow-back arrangements. This account was created in March 2017.

According to internet archives, it posted a range of pro-liberal, anti-Trump, and anti-administration messaging, and it achieved more impact than the conservative impersonators.

The account attacked libertarian U.S. Senator Rand Paul repeatedly, even though Sen. Paul is a vocal supporter of increased cooperation between the United States and Russia.

In the troll farm’s tradition of focusing on the most divisive issues, the account posted aggressively about the Parkland school shooting and the student activists in favor of gun control, who rose to prominence afterwards.

“David Hogg, survivor of Florida school shooting:

‘This is the 18th one this year. That’s unacceptable. We’re children. You guys are the adults. You need to take some action and play a role.’

When will they love their kids more than their guns?? #GunReformNow”

(Posted on February 15, 2018)

“Emma Gonzalez has more class as a high-school senior than Trump as a 71 y.o. illegitimate President.

Just FYI: it’s 7 years until she’s eligible to run for Congress… I’d vote for her now!

#ParklandStudentsSpeak”

(Posted on February 21, 2018)

It also targeted conservative social groups with rhetoric every bit as polarizing and divisive as the fake far right accounts.

“There’s actually one good thing about the Trump presidency. It has finally exposed ‘evangelical Christians’ for what they are — misogynists, pedophile supporters and Nazi sympathizers.”

(Posted on January 29, 2018)

It posted on race issues, another theme the original troll farm targeted.

“Ruby Bridges, the little Black girl who desegregated the New Orleans Schools, is only 62. Let that sink in”

(Posted on August 2, 2017)

As with the far right accounts, this one’s efforts to stay in character may have raised eyebrows in St. Petersburg, given the role Wikileaks played in the Russian election interference campaign.

“Donald Trump Jr. secretly conspired with Wikileaks, which the US intel community considers a ‘hostile foreign intelligence service’

Looks like ‘we’re no longer looking for conspiracies, we’re counting how many’”

(Posted on November 13, 2017)

The second ostensibly African American female account was @KaniJJackson, screen name “Kanisha Jackson.” This account was created on September 5, 2017, just one day after the MAGA accounts. It used three anti-Trump hashtags in its biography: #Impeach45, #Resist and #GunReformNow. It had 27,000 followers, and followed 23,000 accounts, suggesting a follow-back deal.

This account attacked Trump directly.

It also called directly on the #resistance movement, with some degree of impact, judging by the number of likes and retweets.

This account posted on many issues, including race and gun control, but it focused especially on Trump.

“In Trump’s world:

– Blue lives matter!

(except if they’re investigating Trump)

– Honor our veterans!

(except for Mueller, McCain, the widow Johnson, etc.)

– America First!

(except Russia, China, Saudi Arabia, the Trump Family, etc. etc.)”

(Posted on December 23, 2017)

“At Davos, Malala calls out Trump for sexism: ‘I just get so disappointed to see that people are at these high positions, and they are openly, they talk against women, they do not accept women as equal, they harass women…’

Thank you @Malala for this powerful message!”

(Posted on January 25, 2018)

The criticism of Trump even outweighed the account’s own Russian origins, in what is likely to have been an attempt to hide its background.

“Daily reminder: Trump still hasn’t imposed sanctions on Russia that were passed 419–3 in the House and 98–2 in the Senate. Shouldn’t that be grounds for impeachment?”

(Posted on March 18, 2018)

The third apparently African American account was @LaChristie, screen name “Christie Walker,” self-described as a “Resistance Girl” from Flint, Michigan. This account was created on September 11, 2017, and had over 7,000 followers, but followed over 9,000 other accounts.

It, too, attacked Trump.

Its presentation of American society was as polemic as any far right impersonator.

“The plus side to Trump becoming more unhinged by the day is more people are finally realizing that maybe having a treasonous madman sexual predator white supremacist in the WH maybe isn’t such a good thing.”

(Posted on March 10, 2018)

“RT if you think that @realDonaldTrump is the worst president America ever had.

“He is not worthy to be called and a human being.

#WednesdayWisdom “

(Posted on February 7, 2018)

The account centered on race relations, in line with its purported identity.

“‘A man in blackface who was caught in surveillance video trying to rob a bank was arrested in his Lake Elsinore home.’ So you a criminal AND a racist? Bruh pick a struggle “

(Posted on November 10, 2018)

It also posted on Native American rights and grievances.

“Don’t be afraid to talk about the real history behind Thanksgiving. It is the story of murder, enslavement and rape.”

(Posted on November 21, 2017)

Like the far right accounts, these troll farm accounts called for action at the midterms.

“Midterms are in 265 days, use this time to:

– Promote your candidate on social media

– Volunteer for a campaign

– Donate to a campaign

– Register to vote

– Help others to register to vote

– Spread the word

We have only 265 days to guarantee survival of democracy. Get to work!”

(Posted by @KaniJJackson on February 14, 2018)

Salt in the Wounds

The troll farm’s activity and audience reach was lower in 2018 than previous years, but it still targeted painful issues. The word “Parkland,” for example, featured in 201 separate posts, almost all original tweets rather than retweets. The best performing one, by @KaniJJackson, was retweeted over 5,000 times and liked 11,000 times:

“Since Trump is too busy today to address the deadly school shooting in Florida, let’s remember this heartfelt speech from a real President who actually cared about the American people.

#Parkland #guncontrol

(Posted on February 15, 2018)

The trolls used the word “shithole,” used by President Trump to refer to African countries, over 100 times in January 2018. In characteristic troll farm style, some used it to attack him, others to defend him.

“called African countries and Haiti ‘shitholes,’ while suggesting the US needs more people from countries like Norway.

Call it what it is — racist!”

(Posted by @wokeluisa on January 11, 2018)

“Y’all have to admit that Trump is right: Haiti is a shithole country. Nothing racist about it.”

(Posted by @TheTrainGuy13 on January 12, 2018, just two and a half hours after the above tweet.)

American users launched the hashtag #ShitholePresident in response. This Russian troll account was quick to promote the hashtag, picking it up within 90 minutes of its launch. It was liked 10,169 times, and retweeted 4,823 times.

“Now that Trump had called Haiti and African Nations “Shithole Countries” as opposed to Norway, we should make #ShitholePresident trending!

(Posted by @wokeluisa on January 11, 2018)

According to a scan with online tool Sysomos, the number of retweets @wokeluisa scored, at over 4,800, would have made it the eighth most-retweeted post on the hashtag worldwide.

Strikingly, the troll accounts stayed in character when they commented on Robert Mueller’s probe into the Russian interference in the U.S. election, focusing on fanning the flames, rather than directly attacking the probe.

“After testifying before a grand jury for 5.5 hours, Sam Nunberg says the Russia probe is warranted: ‘No, I don’t think it’s a witch hunt, It’s warranted because there’s a lot there and that’s the sad truth.’”

(Posted by @KaniJJackson on March 11, 2018)

“‘We saw, in our limited investigation, strong evidence of collusion. However, the Republicans have chosen to bury that evidence.’ — Rep. Eric Swalwell is a member of the House Intelligence Committee and says the GOP is hiding collusion”

(Posted by @JemiSHaaaZzz on March 16, 2018)

“Cost of Mueller’s investigation: $3,200,000

Benghazi ‘investigation’: $7,000,000

Trump’s golfing trips: $83,000,000

New debt from Republican tax bill:$1,500,000,000,000

Dear Republicans, please spare us your selective outrage about the cost of the Trump-Russia investigation.”

(Posted by @KaniJJackson on May 25, 2018)

“Here are Republicans who opposed the independent investigation of #TrumpRussia

Share this and CALL them all “

(Posted by @ResistersUnion on April 14, 2017)

“ Mueller team is filled with more Clinton lackeys than the Clinton Foundation itself…”

(Posted by @JohnCopper16 on January 5, 2018)

“Mueller’s WITCH-HUNT Team Just Lost a Major Player!”

(Posted by @aswimM0rris on August 17, 2018.)

The troll accounts spread their efforts far and wide, targeting sensitive questions of race, migration, and religion.

“Get back to Europe if you’re against immigrants”

(Posted by @warriors_aztlan on April 9, 2017)

“All ILLEGAL immigrants should go back to the countries they came from #DefendDREAMers”

(Posted by @JohnCopper16 on September 5, 2017)

“Why won’t Donald Trump say the words ‘Radical Christian Terrorism?’ #portlandstabbing”

(Posted by @ResistersUnion on May 27, 2017)

While these posts were highly personal, many others were not. One remarkable feature of the second wave is how often its accounts posted links to American websites, using an automation service called IFTTT (“if this, then this”). They used this approach over 290,000 times, mostly in late 2017 and 2018.

This approach allowed the troll operators to post more rapidly, and to share language directly from American websites, avoiding the linguistic errors characteristic of non-native speakers. However, it also meant that the troll posts were less likely to engage their target audiences than more genuinely personal posts would have done.

Conclusion

These accounts, and many others like them, used a similar strategy to the troll farm accounts of the first wave. They posted on divisive topics, and took both sides in some of America’s most heated arguments. They attempted to insert themselves into the target communities, then described the other side in dehumanizing and polarizing terms.

Their apparently “liberal” accounts were much more effective, in terms of user engagement numbers, than the apparently “conservative” ones, and focused very heavily on the personality of Trump himself. The personal focus likely reflects the president’s extremely polarizing influence, and an attempt to ingratiate these accounts with genuine American liberals.

The apparent greater focus on liberal communities also offers an intriguing expansion of the first troll wave, which emphasized the Black Lives Matter and alt-right movements, at a time when these were the main founts of outrage against the authorities. It suggests that the second front’s target list evolved, seeking out the most apparently outraged and active communities.

The accounts were taken offline some six months before the midterms, but they had already begun to mobilize voters on both sides. The efforts did not constitute a large-scale campaign, but they do highlight the importance of elections as points of social vulnerability.

The creation of the second front, after the main takedown, underlines the persistence of the troll effort. The operation did not close down when Twitter suspended thousands of its assets in September 2017. It would be unwise to assume that it closed down after the second setback, in June.

---

Follow along for more in-depth analysis from our #DigitalSherlocks.