Gender differences in recommendation letters for postdoctoral fellowships in geoscience

This article is part of an ongoing blog series, titled Inequality in STEM: a Dive Into the Data. In this series, we cover recent research exploring and quantifying inequality in STEM. We'll discuss different aspects of inequality, including barriers to career advancement and a chilly social climate, as well as the efficacy of various interventions to combat bias. Our goal with these pieces is to provide clear summaries of the data related to bias in STEM, giving scientific evidence to back up the personal experiences of URMs in STEM fields.

It is impossible for young scientists to advance in academia without the support of their advisors. In particular, letters of recommendation give hiring committees a personal picture of applicants; letter writers have the opportunity to provide specific examples of scientific acuity that may not come across in the rest of the application. Moreover, these letters can highlight the contributions of trainees; as science becomes increasingly collaborative, it is important to highlight the specific contributions of a student to the projects on which they worked.  As a result, these letters of recommendation are absolutely key to securing admission to graduate programs and access to postdoctoral or tenure track positions. However, because letters tend to be personal in nature, they are also more susceptible to implicit biases[i].

Previous studies have demonstrated that letter-writers describe men and women in qualitatively different ways. Women are often described as being less confident and forceful as men, but more nurturing and helpful[ii]. Word choice matters; in an increasingly competitive academic climate that values independence and creativity, it is essential that letters emphasize when top students demonstrate these key traits. In the largest study on gender bias in recommendation letters to date, Dutt et al. quantitatively characterize gender differences in letters of recommendation across countries and institutions[iii].

View fullsize

Dutt and colleagues' coding scheme to characterize the language in over a thousand letter of recommendation for early-career scientists. Women are less likely to receive "excellent" letters describing them as "trailblazers" or "brilliant scientists," traits prized in the academic job market.

To do this, the authors took advantage of a dataset of over a thousand letters of recommendation for a competitive postdoctoral fellowship in geoscience. These letters were written by male and female principle investigators (PIs) from all over the world. In order to study the language in these letters, the authors came up with a comprehensive coding scheme to determine whether the letters were “excellent,” “good,” or “doubtful.” Excellent letters contained superlatives and indicated that the applicant was a “trailblazer”, “independent” and an “excellent role model/leader.” Good letters indicated that the applicant was a competent scientist, but not necessarily the top in the field. Doubtful letters either expressed concern or a lack of knowledge of the applicant. The authors also evaluated length of letters as a potential metric for letter quality.

The authors demonstrated that women candidates were significantly less likely to receive an “excellent” letter as compared to male candidates (Table 4 and 5). In other words, recommenders were far less likely to describe female applicants as “leaders” or “trailblazers.” This discrepancy was not because letters for women were shorter; letters for men and women were on average the same length, which is not surprising given the variation in letter length as well as the regional diversity in this data set (Table 6). Finally, the authors found that male and female PIs did not differ in their likelihood to write excellent versus good letters for applicants (Table 5). This is important to note because this suggests that it is not only male PIs that are writing better letters for their male trainees; female PIs also contribute to the effect seen in this paper.

Together, this paper demonstrates that women are much less likely to receive the excellent letters of recommendation they need to stand out in the competitive academic job market and transition to the next stage of their career. These findings point to a specific case in which bias may affect womens’ access to academic careers, potentially contributing to the existing gender disparities in geosciences and other STEM fields. Due to the quality of archival data, the authors were unable to control for applicant qualifications. However, given previous studies that control for applicant qualifications and the enormous number of letters reviewed here, it is unlikely that the effects seen in this study are due to deficits in female applicant qualities. This study and others suggest that as recommenders write their letters, they must think critically about their own implicit biases. Careful consideration of bias in letter writing will ensure that women receive the letters of recommendation that they deserve.

View fullsize

"Gendered Language in Teacher Reviews," a tool developed by Dr. Schmidt to explore gender differences in reviews from RateMyProfessor.com, shows striking differences between reviews for male (blue) and female (orange) professors. Men are more likely to be described as "brilliant" or "genius," while women are described as "helpful" or "shrill." Try it out here!

Importantly, this paper is not able to explore why women are less likely to get the type of letter they need to receive a competitive postdoctoral fellowship.  While there could be many reasons for this disparity, one possibility is that the language used to describe men and women is gendered; other studies have demonstrated that this is the case. For example, Dr. Benjamin Schmidt, an Assistant Professor of History at Northeastern University, created a powerful online tool for visualizing the gendered ways in which men and women are described, called “Gendered Language in Teacher Reviews.” To build this tool, he gathered 14 million teacher reviews from the website “RateMyProfessor.com” and found striking differences in how students describe male and female professors. For example, across all fields, men are far more commonly described as “genius,” while women are “shrill” or “helpful.” This effect could also play into letters of recommendation, and demonstrates how important it is for recommenders to think carefully about the ways they describe their trainees.  

Finally, it is worth dwelling on the characteristics that the current academic system values when hiring graduate students, post-doctoral fellows, and tenure track faculty. Certainly, “independent trailblazers” are essential to moving the science forward, but do we necessarily want to select such individuals over “highly collaborative” and “helpful” candidates? As science becomes increasingly collaborative, I would argue that the best scientists not only think creatively and independently, but also work well with others and want to bring people with differing expertise together to solve scientific problems.

[i] Sheltzer, J. M. & Smith, J. C. Elite male faculty in the life sciences employ fewer women. Proc. Natl Acad. Sci. USA 111, 10107–10112 (2014).

[ii] Madera, J., Hebl, M. & Martin, R. Gender and letters of recommendation for academics: agentic and communal differences. J. Appl. Psychol. 94, 1591–1599 (2009).

[iii] Dutt, K., Pfaff, D.L., Bernstein, A.F., Dillard, J.S. & Block, C.J. Gender differences in recommendation letters for postdoctoral fellowships in geoscience. Nat. Geo. 9, 805–808 (2016)