Interdisciplinary research team examines racial disparities in NIH peer review
The National Institutes of Health (NIH) is the largest biomedical funder in the world. The R01 grant mechanism is their original and oldest grant mechanism. The peer review process used to allocate these grants strives to be fair, equitable, and free of bias. In 2009, NIH introduced criterion scores—for significance, investigators, innovation, approach, and environment—to complement overall impact scores as a way to increase transparency for applicants. Research studying applications submitted before and after this change revealed substantial Black-white funding gaps.
Elena Erosheva (Professor in Statistics and Social Work, and CSSS core faculty), Carole Lee (Associate Professor in Philosophy and CSSS affiliate faculty), Sheridan Grant (PhD student in Statistics) and colleagues from NIH are the first to examine the peer review scores from the earliest stage of the process, which are used to determine which proposals move on to panel discussion—a decision point in the process that previous work identifies as the largest contributing source for funding disparities in NIH peer review. They present their findings in a new paper published by Science Advances titled “NIH peer review: Criterion scores completely account for racial disparities in overall impact scores.”
The idea for the project was initially submitted to a competition held by the NIH Center for Scientific Review, where it won first prize for Most Creative Idea. The researchers later received an NIH contract, which provided access to the data needed to study their prize idea, followed up by an NSF grant, both administered by CSSS.
The researchers found that during fiscal years 2014-2016, Black applicants had about half the success of their white colleagues in getting their R01 grant applications funded. Even after controlling for demographic, education, and career-stage variables, Black applicants were found to be only about 75% as successful as their white peers. Black researchers received worse scores on all five criteria, even after controlling for key application and applicant characteristics such as career stage and area of science. Furthermore, these disparities in criterion scores fully explain the Black-white gaps in overall impact scores. In contrast, it does not appear that differences in how reviewers weigh the importance of different preliminary criterion scores explain Black-white differences in preliminary overall impact scores.
The Chronicle of Higher Education featured the paper and interviewed Erica Warner, an assistant professor at the Harvard Medical School whose research focuses on studying risk factors for breast cancer. As she said there, “[t]he approach that the NIH has taken in the enhanced peer-review system is to say, If we give reviewers clear, objective criteria, then the decisions they make will be solely merit-based. It doesn’t appear that that’s enough.”
“This is groundbreaking research that identifies where in the NIH review process the Black-white funding gap originates,” said Darryl Holman, CSSS Director and Associate Professor in Anthropology. “Ultimately, this work will help eliminate that gap and lead to greater transparency and equity for researchers.”
