The epidemiology of peer review

Editors, authors, and reviewers are influential in shaping science.

Public health reformer Florence Nightingale had a penchant for pie charts.

So starts a 1998 JAMA paper by Dickersin et al. who asked “Is there Sex Bias in Choosing Editors?” The five authors decided to focus on four US journals in one field. By hook and by crook they found out how many women were editors, reviewers or authors for American Journal of Epidemiology, Journal of Clinical Epidemiology, Annals of Epidemiology, and the succinctly named Epidemiology in 1982, 1987, 1992 and 1994. The field of epidemiology and public health research is an interesting sector as it has traditionally been one that attracts a high proportion of women, many of whom have become remarkable role models.

To support this, the authors cited statistics on membership of learned societies (46% of Society for Epidemiology Research members were women in 1996), enrollment in US graduate courses (women outnumbered men in 1984) and increasing representation on faculties (from 23% in 1976 to 36% in 1991).

Michelle Bachelet, former president of Chile and executive director of UN Women, specialised in epidemiology and public health. Copyright 2012 A Celebration of Women™

The value of this approach is that it compares journals within one field. That means that all journals were dealing with broadly the same pool of researchers as readers, authors, reviewers and editors. So differences in the proportions of women acting as reviewers is likely to directly reflect differences in journal policy and practice in any one year.

For Journal of Clinical Epidemiology, the number of men and women reviewers could not be estimated as reviewers’ names were not published. Could this lack of transparency be in any way related to the low proportion of women who served as editors at this journal, rising from none in 1982 to just 7.8% in 1994?

During the same period, the number of women reviewers at American Journal of Epidemiology doubled from 14.3% in 1982 to 31.3% in 1994. The number of women editors doubled too reaching just 15% in 1994.

Two of the journals were established in 1990 in the middle of the 12-year span. Annals of Epidemiology did not list reviewers’ names in 1992, but did in 1994 when 37.4% of reviewers were women chosen by a corps of editors, 30% of whom were women. Meanwhile at Epidemiology, 5% more women were reviewers in 1994 than in 1992, a rapid change bringing it on a par with the older American Journal of Epidemiology.

As the authors say

If women in general are not perceived to have the same stature as men in a field or are not part of the existing informal or informal networks involved in the nomination process, there may be selection bias against them.

So what has happened in the last 18 years? How many women today review for journals in epidemiology or other fields? If enough journal editors within a sector take the APEER survey we may be able to assess whether bias is caused by particular journal practices, old or new.

Take the survey now or enrol here for a private key to the survey.

The original paper can be found here.

Kay Dickersin, Lisa Fredman, Katherine M Flegal, Jane D Scott, Barbara Crawley (1998) Is there sex bias in choosing editors? Epidemiology journals as an example. JAMA 280 (3), 260-264