FACT CHECK:  Did a study prove that Republicans (primarily supporters of Donald Trump) have weaker grammatical skills than Democrats?

Claim:   A study proved Republicans (primarily supporters of Donald Trump) have weaker grammatical skills than Democrats.

   UNPROVEN

WHAT’S TRUE:   Grammar-checking web site Grammarly analyzed a small sample of Facebook comments and claimed Republicans and supporters of Donald Trump exhibited a higher rate of grammatical errors.

WHAT’S FALSE:   The assertion was proved by “science”; Grammarly’s data constituted what would be broadly recognized as a “study.”

Examples:     [Collected via Twitter, October 2015]

Origins:   On 6 October 2015 the web site Grammarly published a blog post titled “Presidential Debate Grammar Power Rankings,” claiming that with a “light heart and heavy-hitting algorithms” they parsed the Facebook pages of Presidential candidates to determine whose supporters were the most capable writers. The claim received wider attention when it was picked up for a 6 October 2015 USA Today article titled “Democrats crush Republicans in grammar; Chafee on top.”

Grammarly (which described their findings as a “study”) didn’t introduce the notion of a correlation between political affiliation and intellect; during election season in particular, ping-ponging accusations about the respective smarts of Republicans and Democrats inevitably appear on social media. Of course, definitive data of that description would be immediately and widely well-received by whichever political caucus wasn’t maligned by it — but thus far, claims that liberals or conservatives are provably wiser leaned largely on confirmation bias and blogosphere speculation.

Grammarly‘s infographic and blog post (both particularly popular among folks who didn’t identify as Trump supporters) explained their methodology:

We began by taking a large sample of Facebook comments containing at least fifteen words from each candidate’s official page between April, 2015 and August, 2015. Next, we created a set of guidelines to help limit (as much as possible) the subjectivity of categorizing the comments as positive or negative … We then randomly selected at least 180 of these positive and neutral comments (~6,000 words) to analyze for each candidate. Using Grammarly, we identified the errors in the comments, which were then verified and tallied by a team of live proofreaders. For the purposes of this study, we counted only black-and-white mistakes such as misspellings, wrong and missing punctuation, misused or missing words, and subject-verb disagreement. We ignored stylistic variations such as the use of common slang words, serial comma usage, and the use of numerals instead of spelled-out numbers.

The pool of data was small (180 samples per candidate), and the research conducted over a relatively short period of four months in 2015. Not much detail was provided about how random the sampling truly was, and there was no indication that the information collected was checked over or looked at in any meaningful way before the findings were presented to the media.

While Grammarly‘s research perhaps made for a compelling pop-politics read, it didn’t fit generally accepted criteria for a serious study. By their own admission, the data was collected and interpreted by employees of Grammarly (not data analysts or dedicated researchers) with the ultimate goal of promoting their grammar-checking app. The results could theoretically bear out in stronger research, but a less-divisive conclusion almost certainly would draw less attention (and traffic).

At least one site viewed the claim critically; ThinkProgress (a progressive outlet) noted:

Of course, there was more room for error on the Republican side, which has nearly three times as many candidates. In addition, many Republican candidates have a lot more Facebook supporters, meaning the pool from which Grammarly’s researchers picked its 180 comments was much larger. For example, Lincoln Chafee only has 9,526 Facebook followers, while Donald Trump has 3.8 million. Overall, the average number of Facebook followers for Republicans was 1.1 million, while the average for Democrats was about 591,000.

Studies” conducted by companies to garner media attention and promote products (not to be confused with research hoaxes and deliberately fabricated findings) aren’t uncommon, and social media has introduced additional incentive to lure visitors with click-baiting, share-friendly claims. But the assertion Trump supporters (or Republicans in general) possessed weaker grammatical skills than their ideological opponents hinged largely on a small set of comments collected over a short period by a grammar web site with a strong interest in generating press; the blog post in question concluded by encouraging readers to “[g]et more fun posts.”

Last updated:      7 October 2015

Originally published:      7 October 2015