A measure developed to assess the quality of scientific journals has distorted how research is evaluated, and should be not be used to judge an individual’s work, Science Editor-in-Chief Bruce Alberts writes in the 17 May issue of the journal.
The editorial coincides with the release of the San Francisco Declaration of Research Assessment (DORA), which grew out of a gathering of scientists at the December 2012 meeting of the American Society for Cell Biology (ASCB). More than 150 scientists and 75 scientific organizations including Science’s publisher AAAS have endorsed DORA, which recommends specific changes to the way scientific journal rankings are used in hiring scientists, funding research and publishing papers.
One of the most popular ranking measures, called Journal Impact Factor or JIF, ranks research journals based on the average number of times its papers are cited by other papers. (The higher the JIF score, the more often its research papers are cited by others.) JIF was devised to rank journals, but is now often used to evaluate an individual’s research, by looking at whether she or he has published in high-score journals.
This misuse of the JIF score encourages far too much “me-too science,” Alberts writes. “Any evaluation system in which the mere number of a researcher’s publications increases his or her score creates a strong disincentive to pursue risky and potentially groundbreaking work, because it takes years to create a new approach in a new experimental context, during which no publications should be expected.”
Alberts notes that an unhealthy obsession with journal ranking scores may also make journals reluctant to publish papers in fields that are less cited, such as the social sciences, compared to papers from highly-cited fields such as biomedicine.
The DORA guidelines offer 18 specific recommendations for discontinuing the use of JIF in scientists’ hiring, tenure, and promotion, along with ways to assess research on its own merits apart from its place of publication.
“The Journal Impact Factor was developed to help librarians make subscription decisions, but it’s become a proxy for the quality of research,” said ASCB Executive Director Stefano Bertuzzi in a recent news release about the DORA recommendations. “Researchers are now judged by where they publish not by what they publish. This is no longer a question of selling subscriptions. The ‘high-impact’ obsession is warping our scientific judgment, damaging careers, and wasting time and valuable work.”