Politically themed Facebook pages and groups are in overdrive ahead of next month's U.S. midterm elections, with users sharing news and opinion in their discussions. However, how can readers know when such activity is authentic versus when posts are coming from coordinated inauthentic accounts? How can they be sure that the pages and groups they interact with don't include leaders who misrepresent both themselves and the facts? The truth is that, sometimes, it's really tough to tell.
One such curious example was a network of Facebook fan groups that falsely claimed to be officially affiliated with Gov. Ron DeSantis, R-Fla. Two of the groups displayed the names Ron DeSantis Campaign Headquarters and Ron DeSantis Official Support Page. One of three admin accounts for the network displayed an AI-generated profile picture. On top of that, none of the groups' admins appeared to have ever posted publicly viewable photographs of themselves.
Additionally, we were unsuccessful at reaching anyone associated with the accounts or groups. We sent the admins multiple messages via Messenger, comments, and guest posts, and no one returned them. After we identified ourselves as reporters, the admins banned us from participating in the groups.
Given those signs, we had reason to question whether the accounts were operated by genuine people in support of DeSantis, who is seen as a potential 2024 presidential candidate, or were part of an effort to promote narratives that favor his political agenda using fake aliases. We shared findings of this investigation with Meta, the social media platform's parent company, and the company said the groups' and admins' activity didn't violate Facebook's guidelines.
"We strive to strike the right balance between taking action against violating activities and ensuring we don’t silence people sharing their views," Meta said in a statement. "We will continue to enforce our policies and take action if we detect abuse of our platform."
The Network Includes a Random 'Downton Abbey' Fan Group
Most groups in the in-question network (i.e., a collection of Facebook pages or groups with the same admins) seemingly served as spaces to share, or create, content in support of DeSantis or right-wing talking points. However, one was dedicated to a popular television show for reasons that were suspiciously unclear.
Three pro-DeSantis groups showed a combined member count of more than 94,000 users. The largest group had 81,300 members and was named Ron DeSantis Campaign Headquarters. The other two were named Ron DeSantis Official Support Page and Ron DeSantis International Support Page. We found no indication that DeSantis or any of his team had anything to do with the groups.
The admins were also in control of a fan group for the television series "Downton Abbey." That group's name was Downton Abbey International Facebook Group, with a user count of 6,600. The admins' posts about the show were simple, often doing little more than showing pictures of the cast and crew.
Was the creation of the non-political group part of admins' efforts to appear authentic to American users? (The TV series saw soaring ratings during its initial run in the U.S.) The Downton Abbey group was one of several flags that led us to believe the network of Facebook groups, including the DeSantis pages, may be part of coordinated effort in which people, or entities, promote messages under fake identities.
We also located a fifth group, President Trump's Official Support Page, which had 1,300 members. It's unclear if additional groups were part of the network.
Admins Haven't Posted Public Photos of Themselves
The groups' admins showed the names "Ann Ambry," "Kristen Lee," and "Margaret Pawlak." It's unclear if these names were real. Facebook's community standards require that users "connect on Facebook using the name they go by in everyday life." But the company's policies do not explicitly say that a birth name or legal name is required.
From what we could tell, none of these three admins posted publicly available photographs of themselves. Instead, their profile pictures showed various images of American flags or a photograph of Highclere Castle, which is shown on "Downton Abbey."
The oldest profile photographs on all three accounts were from either June or July 2020, which perhaps indicated that the accounts' activity increased following the May 2020 video-recorded murder of George Floyd. The aftermath of the murder was arguably a unique and turbulent moment on the subject of race in the U.S. (The topic of race had been wielded as a tool by foreign actors to create conflict ahead of previous U.S. elections.)
A Potential Red Flag: AI-Generated Profile Pic
The main admin in the Facebook network was "Ann Ambry," with a bio: "Hi. I'm Ann. Nice to meet you." According to member data, this user created the groups.
On June 21, 2021, the account posted a new profile picture that, based on visual clues, appeared to have been generated by artificial intelligence. (We've had some experience with this subject in the past.)
We sent the profile picture to Giorgio Patrini, the founder and CEO of Sensity, a company that specializes in the creation of fraud-detection tools. (Sensity recently collaborated with Stanford University researchers to expose fake photographs on thousands of LinkedIn profiles that were used for sales-lead generation.) In Twitter messages with Snopes, Patrini confirmed that the "Ann Ambry" picture was fake.
A scan of the photograph with Sensity's software revealed a 99.9% chance that it did not show a real person.
"These [photographs] are usually generated by one of these two sites," Patrini said, linking us to thispersondoesnotexist.com and generated.photos. "The second one [site] is more likely, given that it lets a user remove and replace the background."
He said Sensity recently estimated that "about 0.2 percent to 0.7 percent of user profiles on popular social networks use computer-generated photos." He also told us that fake faces are mainly used for scam and fraud attempts not related to politics.
"The research and technology in this space is advancing very rapidly," Patrini said. "2022 is the year of breakthrough in text-to-image generation. Models such as Stable Diffusion and Dall-E will give superpowers to bad actors and fraudsters."
At some point the fake profile picture was swapped out for an American flag teardrop image. Then on Oct. 14, 2022, the admin reposted the fake, AI-generated profile photograph of the woman.
The Group's Content: Posts That Stoke Division, Mock LGBTQ People
Posts by the three admins often focused on day-to-day news. They drove hatred of the state of California and Democratic Gov. Gavin Newsom; downplayed the effectiveness of face masks in preventing COVID and the virus' dangers; stoked racial divisions; made fun of people who identify as LGBTQ; and supported Republican politicians running in the 2022 election.
The groups promoted messages in favor of former U.S. President Donald Trump, referencing recent events.
On Nov. 8, 2020, one day after U.S. President-elect Joe Biden was projected by The Associated Press to have won the 2020 election against Trump, the "Ann Ambry" account posted an announcement: "I started a group yesterday. It's called Biden IS NOT MY PRESIDENT." That title appeared to be a reference to the false conspiracy theory that Biden had not won, and there was reason to believe the long-ago-debunked rumors of massive voter fraud in American elections.
The group referenced in this post was not accessible, as of October 2022. We asked Meta about the group but did not receive an answer.
'Engagement Drives Facebook's Business Model,' Says Ex-Facebook Official
On Oct. 4, after we reached out to Meta about the network, the "Ann Ambry" admin account posted a screenshot that said Facebook notified the account about the largest DeSantis group being "at risk due to continued community standards violations." In response, the admin asked members to migrate to a new group. No further details about that new group were available.
Despite making billions of dollars in quarterly revenue and having tens of thousands of employees, Facebook has for years apparently struggled to promptly remove content and accounts that violate its community standards guidelines.
"The issue here is misaligned incentives," Roger McNamee, a former adviser to Mark Zuckerberg, told us, speaking generally on the subject. McNamee is also the author of The New York Times best-selling book, "Zucked: Waking Up to the Facebook Catastrophe."
"The most valuable content on Facebook is harmful, so the company does everything possible to avoid removing it," he said. "Why? Engagement drives Facebook’s business model. The most universal way to generate engagement is to trigger fight or flight. The content that does this best is hate speech, disinformation, and conspiracy theories."
Authentic or Inauthentic?
Considering Meta's statement that said it hadn't found "evidence of policy violations," our own inconclusive findings, and the fact that our attempts to reach any human involved with the network were unsuccessful, we're back to where this investigation started: asking how to definitively find out if activity on Facebook is authentic or inauthentic. The possibility always remains that even the Facebook groups that seem most inauthentic might simply be managed by Americans who would rather not reveal their true identities to the platform. Our evidence shows signs of inauthentic behavior — but, again, we don't know for sure.