Facebook removes inauthentic assets using AI-generated faces in Argentina
Network used fake profiles to create fictitious personas supporting Argentina’s Kirchnerism political movement
Facebook removes inauthentic assets using AI-generated faces in Argentina
Network used fake profiles to create fictitious personas supporting Argentina’s Kirchnerism political movement
On December 22, 2020, Facebook took down an Argentinian network of 663 Facebook accounts and 388 Instagram profiles engaged in coordinated inauthentic behavior. The DFRLab discovered that some of these accounts used AI-generated images and stock photo imagery as their profile photo. While the research did not find instances regarding who could be behind such an operation or the purpose of the network, it found connections between the assets and the Argentine political movement Kirchnerism.
Some of the user accounts declared in their descriptions their support for the Argentine political movement Peronism and its successor, Kirchnerism, while others shared content in favor of Sergio Berni, the minister of security for the province of Buenos Aires, Argentina’s capital.
Although the accounts did not post much content to their timelines, the DFRLab was able to corroborate the connections between the assets, as many of them followed each other on Instagram and shared friendship connections on Facebook.
In its announcement of the takedown, Facebook stated:
The people behind this activity used fake accounts — some of which had already been detected and removed by our automated systems — to create elaborate fictitious personas using profile photos generated by artificial intelligence, post, like and comment on content to make it appear more popular than it was. This network posted in Spanish and focused primarily on inauthentically amplifying posts and news articles related to Sergio Berni, Buenos Aires’ Minister of Security. These accounts liked and re-shared the same posts at once from this politician’s official Page. They also posted about hobbies, animals, and cooking to appear more authentic.
We found this network after reviewing public information about a small portion of this activity shared by an open source researcher. Our assessment benefited from additional findings shared with us by FireEye, a cybersecurity company.
Network’s links to Kirchnerism
The accounts in this network posed as locals, supposedly based in Buenos Aires. The DFRLab reviewed public posts on 662 Facebook accounts and 226 Instagram accounts that were removed from the platform. According to the accounts’ descriptions on Facebook, some of the assets supported the Argentine political movement Peronism, supporting the legacy of Juan Domingo Perón, who served as president of Argentina between 1946 and 1955 and then between 1973 and 1974. Some accounts also included descriptions supporting Kirchnerism, another political movement based on the ideals of presidents Néstor Kirchner and Cristina Fernández de Kirchner, who governed Argentina from 2003 to 2015. Fernández de Kirchner is currently the vice president in Alberto Fernández government.
The research also found connections between the removed assets and Kirchnerism by analyzing the pages liked by the accounts. On Facebook, some of these pages were connected primarily to current Argentinian President Fernández, current Vice President Fernández de Kirchner, and two local politicians linked to Kirchnerism: Axel Kicillof, the governor of Buenos Aires, and Sergio Berni, the minister of security for Buenos Aires.
The accounts also liked assets connected to political parties, movements, and organizations including the aforementioned Peronism and Kirchnerism, as well as the left-wing coalition Frente De Todos (Everybody’s Front), which supported Alberto Fernández in the Argentine November 2019 presidential elections.
The network’s activity on Facebook showed signs of coordination. Many assets had their last post between May and August 2020. On some days, around 50 Facebook accounts from the network shared or updated their profile on the same day.
Some of them updated their profile picture within the same hour.
The accounts did not share a lot of content on Facebook. Some accounts only had their profile and banner pictures as the last activity. However, a subset of the accounts shared similar posts, mostly highlighting the assistance provided by the Buenos Aires government to one of the neighborhoods in the province of Buenos Aires during the 2020 pandemic. The activity of the network’s accounts increased significantly after the first post on May 26, 2020.
Other accounts followed a similar pattern but with a different post related to Berni.
On Instagram, the removed profiles showed connections with Kirchnerism as well, but to a lesser extent. A social network graph showed the abovementioned Argentine politicians among the most followed accounts by the analyzed assets.
By analyzing the Instagram connections, the DFRLab found that many of the removed assets also followed the account @madison.seguidores, which belonged to Buenos Aires-based digital marketing firm Madison. According to Madison’s website, the firm offers services to boost engagement and followers on Instagram for its clients. The DFRLab was unable to establish a connection between the Argentine politicians mentioned above and the firm.
Analyzing the fake profiles
A large proportion of the Facebook and Instagram accounts had AI-generated profile pictures. They used generative adversarial networks (GANs), an artificial intelligence technology behind websites such as This Person Does Not Exist. The use of this technology in this network aimed to create fictitious personas to engage on the platforms, mainly to amplify Berni-related posts or to like pages linked to Kirchnerism.
AI-generated faces using GANs are evolving toward more believable pictures; still, these profiles can be recognized by analyzing features on the image, such as background details and glitches in hair, lips, eyes, and ears.
Another feature of these AI-generated faces is that the eyes occur in the same place, regardless of the orientation of the computer-generated face. Detecting the eye-orientation in a collection of potential AI-generated images can be a useful technique to recognize these profiles on social media. The following image shows an example of this technique.
Hundreds of assets involved in the takedown used AI-generated faces. The following images showed more than 80 AI profile pictures used in the network. In all these profiles, the eyes match in both vertical and horizontal aligns.
Another noteworthy technique to analyze these profiles, exposed by open source analyst @conspirator0 while analyzing thousands of AI-generated images from thispersondoesnotexist.com, is that the eye alignment can also be detected when faces are superimposed on one another.
The following graphic in .gif format shows a superimposed image of 83 AI-generated profiles involved in the takedown. The comparison shows how the eyes aligned precisely.
AI-generated faces were not the only method used by the network to create fictitious personas. Other accounts took their profile picture from stock photo imagery and websites. The following image shows examples of these profiles.
Asset connections
The DFRLab mapped the connections between the assets on both Facebook and Instagram. Social network analysis techniques showed that the Facebook accounts shared friends on the platform, while the profiles on Instagram mainly followed each other.
On Instagram, the follower connections indicate that most of the removed profiles were following each other.
Conclusion
The accounts of this network did not show to be frequently active — at least not publicly. Nevertheless, the network’s connections to the Argentine political movement Kirchnerism suggested coordination of the user accounts, either through pages’ likes or similar posts supporting Kirchnerism-allied politician Sergio Berni.
Although profile photos of the accounts followed a particular pattern of taking images from elsewhere online, hundreds used AI-generated profile photos, which could have made it more difficult for social media users to detect the inauthenticity of the network.
As AI-generated faces evolve toward more realistic human faces, the persons behind social media-based information operations choose to create fictitious personas using such AI technology. For social media users, this represents a potential threat while engaging in political and democracy-related conversations.
Esteban Ponce de León is a Research Assistant, Latin America, with the Digital Forensic Research Lab.
Daniel Suárez Pérez is a Research Assistant, Latin America, with the Digital Forensic Research Lab.
Follow along for more in-depth analysis from our #DigitalSherlocks.