Inflation of the journal impact factor

In 1975, the inaugural Journal Citation Reports (JCR) included the widely used bibliometric measure, the journal impact factor (JIF). The JIF is now reported in the annual JCR by Clarivate Analytics. It is calculated by dividing the number of current year citations by the number of items published in that journal during the preceding 2 years. In 2017, the JIF began including early access (EA) items using the EA date of publication, but since 2020, the JIF calculation uses the final publication date instead. The JIF is an objective and globally familiar indicator. It allows comparison across similar fields and is normalised for journal age and size. The JIF is criticised due to technical imperfections and frequent misinterpretations. Disadvantages include skewness in the citation distributions, biases in favour of English language journals, the inclusion of self-citations and the fact that citations do not indicate quality or importance. Despite these shortcomings, the JIF is widely used as a metric of productivity, influencing hiring, promotion and grant applications. Over time, the number of journals and publications has increased, but importantly, the number of articles in citation lists has increased at a greater rate.1 Consequently, there has been an inflation in the JIF over time.

We searched the JCR for the JIF of top ranked journals from high-income countries (HICs) and low- and middle-income countries (LMICs) as defined by The World Bank in the category Medicine, General and Internal from 2006 to 2023 (methods in online supplemental information 1).2 The results show that during this period, the JIF for the top-ranked journals has increased over 400% and doubled from 2020 to 2021 alone (figure 1). The New England Journal of Medicine JIF in 2006 was 51.296 and by 2021 it was a staggering 176.080. The stark increase in JIF from 2020 to 2021 is attributable to the publication surge during the COVID-19 pandemic and exemplifies many JIF biases. The COVID-19 infodemic was riddled with an overwhelming number of inferior quality publications in an emotionally charged environment. COVID-related publications were more likely to be case series or observational trials, compared with non-COVID-related studies. Manuscript retraction rates for COVID-related research were high, including in high-impact journals.3 The rate of dissemination of COVID-related publications was unprecedented with a rapid rise in citations per publication. Between 2020 and 2021, COVID-related publications accounted for 4% of all publications but 20% of all citations.4 This rapid escalation in citations and hence increases in the JIF numerator contributes to a rise in JIFs during the pandemic years. Since the pandemic has ended and the high volumes of COVID-related publications have ceased, the JIFs have begun to decline to prepandemic levels. However, the overall JIF trend of HIC journals is still increasing.

The pandemic brought to light many research and healthcare disparities between HICs and LMICs, with vaccine development dominated by HIC researchers and most of the initial global vaccine supply funnelling to the HIC population. Journals published in LMICs have not seen the same sharp ascent in JIF inflation, nor the large surges in JIF with the COVID-19 pandemic (figure 1). The Libyan Journal of Medicine, one of the top journals in Africa, has seen its JIF rise over 10-fold from 0.169 in 2010 to 1.8 in 2023.2 Yet, this is still far off from that of comparable journals from HICs, such as the Canadian Medical Association Journal which had a JIF of 9.4 in 2023. The JIF gap between journals from HICs and LMICs has important implications for researchers from LMICs stemming from the widespread misuse of the JIF as a measure of research productivity, quality and scholarly value. Researchers from LMICs are already under-represented in the medical literature and encounter significant barriers such as inadequate funding, limited resources and language barriers when it comes to research publication and the widening of the JIF gap only adds to the inequities. Researchers from LMICs are more likely to publish in lower-ranked LMIC medical journals which have less global reach and less likelihood of being cited.5 Journals with lower JIFs are more likely to be dismissed by readers as ‘lower-quality’, compared with research published in higher impact journals, regardless of study quality.6 HIC journals also favour globally visible topics, so research on locally relevant or neglected tropical diseases is under-represented in HIC journals.7 LMIC researchers often face geographical biases with publishing and peer review, further impairing research translation.8 Editorial boards and peer reviewers for high-JIF journals are predominantly based in HICs, introducing potential biases in topic relevance and methodologies. Lastly, journals from HICs have expansive marketing campaigns that further inflate their metrics. Funding for this is partially supported by the high article processing charges, which may be unattainable for LMIC researchers, despite fee reduction schemes. While these research disparities cannot be solely blamed on the JIF, we believe that academia’s over-reliance on the JIF as a measure of quality and productivity indirectly increases the inequities between researchers.

The JIF is fraught with biases, as illustrated by the surge of JIFs during the pandemic. Researchers and academic institutions alike should move away from using it and other journal-based metrics as measures of productivity. The over-reliance on the JIF and the lack of understanding of its limitations and biases has further disadvantaged researchers from LMICs. Already in 2023, the JIF from many journals is rapidly declining, and we hope that the JIF inflation stabilises and the JIF gap between journals from HICs and LMICs closes. Global collaborations and partnerships are integral to closing the gaps faced by researchers and translating important global health issues. Several organisations and partnerships already exist to help researchers in LMICs publish work, such as Pre-Publication Support Service, but further support and recognition of these issues is paramount. We believe that unless publishers and academia can continue adhering to equitable and transparent principles, the over-reliance on the JIF will only further disadvantage researchers.