On 17 June 2014, researcher Kiju Jung and colleagues published a study in the journal Proceedings of the National Academy of Sciences titled "Female Hurricanes Are Deadlier Than Male Hurricanes". Using a combination of historical data and survey experiments, the study argued that more people die from feminine-sounding hurricanes because of an implicit bias that leads to the public to underestimate their risk relative to masculine-sounding hurricanes:
We use more than six decades of death rates from US hurricanes to show that feminine-named hurricanes cause significantly more deaths than do masculine-named hurricanes. Laboratory experiments indicate that this is because hurricane names lead to gender-based expectations about severity and this, in turn, guides respondents’ preparedness to take protective action.
The report used archival data from Atlantic hurricanes in the United States spanning 1950-2012. To quantify the clearly subjective view of what constitutes a "masculine" versus a "feminine" name, the study relied on ratings derived by nine individuals with no knowledge of the study’s hypothesis:
To test this hypothesis, we used archival data on actual fatalities caused by hurricanes in the United States (1950–2012). Ninety-four Atlantic hurricanes made landfall in the United States during this period.
Nine independent coders who were blind to the hypothesis rated the masculinity vs. femininity of historical hurricane names on two items (1 = very masculine, 11 = very feminine, and 1 = very man-like, 11 = very woman-like), which were averaged to compute a masculinity-femininity index (MFI).
Based on this analysis, the researchers concluded that the perceived femininity or masculinity of a name had no real bearing on milder storms, but that for severe ones (defined by how much damage they caused) the change was "substantial, such that hurricanes with feminine names were much deadlier than those with masculine names."
The crucial link underlying the researchers’ hypothesis, however, cannot be tested with that data alone, and therefore the researchers also conducted survey "experiments" to see if there was quantitative support for the idea that gender bias clouds people's view of the potential destruction of a hurricane.
Their first experiment looked at expectations of hurricane severity based only on names:
Used five male and five female names from the official 2014 Atlantic Hurricane names. Three hundred forty-six participants predicted each hurricane’s intensity [...]. As expected, hurricanes with male names [...] were predicted to be more intense than those with female names [...].
The second experiment gave 108 participants a map and a written scenario of a hurricane’s future possibilities and uncertainties. Each participant had the same information save the name of the Hurricane. One group's materials identified the fictional storm as Alexander, another Alexandra, and a control group for whom the storm was identified simply as "hurricane". By their analysis, participants found hurricane Alexander riskier than Alexandra and the unnamed storm.
The third and fourth experiments aimed to demonstrate the final link of their hypothesis: that perceived risk caused by implicit gender bias affects the likelihood of someone taking evacuation orders seriously (which subsequently contributes to the deadliness of the storm). Using similar survey methods, they reported that people were more likely to follow evacuation orders for a masculine sounding storm. As a whole, the researchers concluded:
An archival study of hurricane fatalities established that severe storms with more feminine names are deadlier. Multiple experiments suggested that this is because feminine- vs. masculine-named hurricanes are perceived as less risky and thus motivate less preparedness. Although our findings do not definitively establish the processes involved, the phenomenon we identified could be viewed as a hazardous form of implicit sexism.
The study was widely and uncritically reported in a number of news outlets at the time, but as its stature as a viral story spread, so too did scrutiny of the methods utilized by the researchers, leading to calls that the conclusions were fatally flawed for two main reasons: 1) The study’s dataset included hurricanes from a time period (1950-1978) in which only female names were used; and 2) the statistical significance of the gendered trend relies, essentially, on only a small number of very deadly storms.
The fact that only female storm names were used before 1979 complicates Jung’s results, statisticians have argued, because it makes it more likely that a deadly storm be given a female name. As discussed in a National Geographic story based on an interview with the National Centre for Atmospheric Research’s Jeff Lazo:
This matters because hurricanes have also, on average, been getting less deadly over time. “It could be that more people die in female-named hurricanes, simply because more people died in hurricanes on average before they started getting male names,” says Lazo.
Jung’s team tried to address this problem by separately analysing the data for hurricanes before and after 1979. They claim that the findings “directionally replicated those in the full dataset” but that’s a bit of a fudge.
Indeed, this "fudge" was pointed out in a response to the study also published in PNAS:
In the light of an alternating male-female naming system started in 1979, the authors claim that similar results can be obtained by only considering the data after 1979 [...]. However, when reanalyzing [their model] for these years, no effect can be observed at all: neither [the perceived gender of the storms name] nor [other potential confounding factors] are significant on a 10% level.
The authors of the study responded to this line of criticism by saying that those analyses rely on a binary male or female gender, but noted that their study used perceived gender and presented that data as a degree of masculinity and femininity:
When we model the fatalities of all hurricanes since 1950 using their names’ degree of femininity, the interaction between name-femininity and damage is statistically significant. That is a key result. Specifically, for storms that did a lot of damage, the femininity of their names significantly predicted their death toll.
They also noted that Lazo's claim of decreasing deadliness of hurricanes was not borne out by their analysis:
Is this a statistical fluke? In some articles, Jeff Lazo says, “It could be that more people die in female-named hurricanes, simply because more people died in hurricanes on average before they started getting male names.” But no, that is not the case according to our data and as reported in the paper. We included elapsed years (years since the hurricane) in our modeling and this did not have any significant effect in predicting fatalities. In other words, how long ago the storm occurred did not predict its death toll.
The second complaint is the fact that any significant trend found in the data requires the presence of a few very deadly storms. Jung et al do note that there appears to be no trend for the less deadly storms, and that significance is only reached when adding a small number of very deadly storms. This, too, was noted in a separate published response in PNAS:
Inclusion of four storms with more than 100 deaths raises the average death toll for all female storms by 85%, to 23.5 [...]. Hurricanes Diane (1955), Camille (1969), Agnes (1972), and Sandy (2012) collectively accounted for 732, or 38.5%, of 1,900 total deaths in the study. Virtually all of the statistical difference between the deadliness of female vs. male storms is explained by the inclusion of these four events.
By some analyses presented online, including the National Geographic piece, the entire trend's statistical significance could be changed if only one destructive storm — Hurricane Sandy — was not included. The authors of the study, however, challenge this finding as well, arguing that these interpretations were based on a simplistic statistical approach that was not in line with their research:
We also noted a few very simple efforts to correlate masculinity-femininity of name with death toll in a subset of the data, without controlling for any other factors, and then drawing conclusions about whether the effect holds (e.g., on Slate.com). For multiple reasons, this type of approach is quite misleading. Appropriate analyses show that keeping versus deleting data from Hurricane Sandy does not change the conclusions, as claimed in some commentaries. They can be found here.
Though originally presented by news outlets as a simple and shareable story, this study asked a simple question using extremely complex statistical tools that also involved myriad assumptions which have been challenged by a number of other researchers.
Further, the mechanism (implicit bias) by which the researchers hypothesize their contested findings is only addressed indirectly, and requires agreeing with the conclusion of multiple online surveys. For these reasons, this seemingly simple question does not currently have a simple answer.