Skip to main content

NIH holds contest for fixing bias in peer review (with cash prizes)

Recent studies have demonstrated that African American researchers are less likely to receive NIH grant funding than are white researchers. The NIH wants to know if grant-reviewer bias is causing this disparity and wants to improve the overall fairness of peer review. With these aims in mind, the NIH is issuing two challenges for improving the grant review system.

A 2011 study by Donna Ginther and colleagues published in Science examined the relationship between grant applicants' self-identified race and/or ethnicity and the probability that their NIH grant applications would be  funded. The study found that black applicants are 10 percent less likely to have their grants funded compared to white applicants, even after controlling for "educational background, country of origin, training, previous research awards, publication record and employer characteristics."

NIH Director Francis Collins and deputy director Lawrence Tabak immediately offered a response to the study, stating that "residual cultural biases may have disproportionate adverse consequences on minority subgroups of our scientific community." Collins and Tabak provided some steps that the NIH was taking to deal with the problem, such as increasing the representation of early-career investigators in grant-review study sections, increasing mentoring, and investigating whether a different review system could erase racial disparities in grant funding. In December 2012, the NIH announced a $500 million initiative for increasing the diversity of the biomedical workforce.

While these initiatives sound great, some have criticized the NIH for not tackling head on the role of reviewer bias in current study sections. According to an article in the Chronicle of Higher Education published in January 2014, "a project promised by the NIH's director, Francis S. Collins, to directly test for bias in the agency's grant-evaluation systems has stalled, with officials stymied by the legal and scientific challenges of crafting such an experiment."

Now it is May 2014 and the NIH is responding to criticism by offering two open challenges:

Challenge #1

New Methods to Detect Bias in Peer Review

Submit your idea on how to detect bias among reviewers due to gender, race/ethnicity, institutional affiliation, area of science,  and/or amount of research experience of applicants. First and Second prizes will be offered in two categories, best empirically based idea and most creative idea. Additional details can be found at FRN Doc.2014-10196.

Challenge #2

Strategies to Strengthen Fairness and Impartiality in Peer Review

Submit your idea on how to strengthen reviewer training methods to enhance fairness and impartiality in peer review. First and Second prizes will be offered for the best overall ideas. Additional details can be found at FRN Doc.2014-10203.

Applicants have until June 30 to submit ideas. First-prize winners for each category receive $10,000; second-prize winners receive $5,000.

This contest might be evidence that the NIH is unofficially throwing up its hands when it comes to addressing potential racial bias in peer review. Admittedly, bias—particularly unconscious bias—can be a difficult problem to probe and to solve. While it is disappointing that the NIH still doesn't know what to do after three years of considering reviewer bias, the contest itself is heartening. After all, what does a good scientist do when she doesn't know how to solve a problem? Ask for help.


Blog Name