Field-Weighted Citation Impact affects rankings – Study

Field-Weighted Citation Impact (FWCI), which is used to represent research quality in the Times Higher Education (THE) World University Subject Rankings, can be exploited strategically for maximising measured performance, rather than true research quality, according to a new study.

“FWCI-driven indicators in global university ranking can amplify positioning, thus distorting institutional comparisons,” states a study that analysed FWCI in a dataset of 230,357 publications from ten highly ranked universities over the 2021 to 2025 period.

The “Field-Weighted Citation Impact” (FWCI) measures how well a publication performs in terms of citations compared to similar publications in the same field, year, and document type.

The study, titled “Is field-weighted citation impact fit for purpose? Sensitivity, structural bias, and implications for global university rankings”, published in Scientometrics on 22 June, states: “Despite its prominence, the underlying mechanisms of FWCI and its influence on ranking outcomes remain insufficiently examined at the institutional level.”

Therefore, the study investigates internal logic, empirical behaviour, and possible inflation (disproportionally high values) reasons of FWCI, which can substantially affect institutional ranking outcomes.

Artificially inflating institutional rankings

The study showed that “even though FWCI is conceptually attractive as a field-normalised indicator, it is highly sensitive conceptually to publication type, subject classification and citation baselines”.

Additionally, the study presented evidence that suggests that selective emphasis on some types of publication, disregard of self-citation, collaboration in mega-papers or other strategies of publication may artificially inflate FWCI and, therefore, institutional ranking positions.

“This concern is particularly acute in institutions where conference papers, books or chapters constitute a large share of total output,” the study indicated.

“100-institution analysis, portfolios where these types can reach up to 30% of total publications are not exceptional, and in such cases the structural amplification mechanism, whereby non-article types generate up to five times more FWCI per citation unit than articles, can elevate institutional FWCI independently of any genuine improvement in research quality,” the study explained.

The findings indicate that FWCI behaviour is influenced by factors independent of research quality, including access models and subject classification frameworks, which affect institutional-level aggregation.

Additionally, the study highlights that, aside from expected factors like citation baselines and publication maturity, subject classification systems and document types significantly impact FWCI values.

For example, conference papers and chapters in some fields can have up to 5 times higher average FWCI-to-citation than articles, with maximum ratios above 2.2 and 2.8, respectively, for the most mature years, according to the study. Additionally, the study revealed that for some fields and publication types, open-access outputs can exhibit up to two times higher FWCI values than non-open-access ones.

The findings were confirmed by a validation conducted on a broader set of 100 institutions from the top 1,000 of the Times Higher Education (THE) World University Subject Rankings 2025.

The study found that conference papers with books and chapters reaching up to one third of total output can create conditions where the amplification translates into unbalanced institutional FWCI performance. To address this, the study proposed a simple adjustment through reducing FWCI inflation and improving interpretability, reducing conference paper and chapter FWCI by around 50%.

Exploiting, inflating and ‘gaming’ metrics

Dr Savo Heleta, a researcher, internationalisation scholar, and analyst, told University World News: “The findings from this paper are yet another set of evidence showing the problematic nature of university rankings.

“In this specific case, the study illustrates how the Field-Weighted Citation Impact can be exploited by institutions to inflate and ‘game’ their metrics in order to move up the university rankings.

“All this is highly problematic and worrying, showing that there is a lack of ethics and values in many higher education circles around the world. The urge to be ranked high is clearly leading to some very unethical behaviour in many institutions, including the ‘top’ ones.”

Expanding further, Global higher education expert and director of strategic insights at RMIT University in Australia, Angel Calderon, told University World News: “This study is both timely and important, as it examines a metric that remains underexplored and often misunderstood in the field.

“The FWCI carries significant weight in global rankings such as THE World University Rankings and, therefore, has critical implications for institutional standing.

Beyond institutional impact, the FWCI can also shape individual academic outcomes, including recognition, career progression, and promotion,” said Calderon, who is the author of “Sustainability Rankings: What they are about and how to make them meaningful” .

Calderon noted that the study emphasises how citation impact is affected by publication type and disciplinary differences, highlighting concerns about the inconsistent operation of FWCI across different contexts.

It specifically affirms that FWCI may disproportionately advantage smaller institutions and can be strategically optimised by research teams in niche fields.

Research evaluation is ‘inherently complex’

“I commend the authors for opening an important line of inquiry, particularly in relation to the influence of publication types, the imbalance both within and across fields of research, and the question of whether publications in top-quartile journals are consistently associated with higher FWCI,” Calderon noted.

Calderon emphasised that the relationship between top-tier journal publications and citation impact is complex, challenging common assumptions. Multiple interacting factors influence whether articles in highly ranked journals achieve higher Field-Weighted Citation Impact (FWCI) compared to those in lower-tier venues.

“More broadly, the study highlights the inherent complexity of research evaluation. Universities operate in multifaceted environments where a wide range of structural, disciplinary, and behavioural factors influence research performance,” Calderon noted.

Because they represent the interaction of several and occasionally “opaque dynamics”, citation-based metrics like FWCI cannot be read in a vacuum, Calderon noted. The study also highlights a crucial knowledge gap about the connection between the subject classifications utilised in ranking algorithms and the All Science Journal Classification (ASJC) system.

According to Calderon, this is an issue that needs more attention because many academics are still not familiar with these classification typologies, and as a result, they are unaware of how ranking systems map and assess their scholarly outputs.

He proposed that “improving transparency and literacy in this area would contribute to a more informed and equitable use of bibliometric indicators”.

Understanding citation performance comprehensively

“The authors’ proposal to refine THE’s use of FWCI by adjusting for publication type is a valuable contribution and deserves further investigation,” Calderon indicated.

“However, the analysis could be strengthened by incorporating additional dimensions known to affect citation impact,” Calderon pointed out. These include the extent of international collaboration (for example, publications co-authored across multiple countries), the number of contributing researchers, and the number of participating institutions, according to Calderon.

“Accounting for these factors would provide a more comprehensive and nuanced understanding of citation performance,” Calderon pointed out.

Independent researcher and consultant Dr Vladimir Moskovkin told University World News: “The significance of this study lies in uncovering the hidden structural vulnerabilities of the Field-Weighted Citation Impact (FWCI), a cornerstone metric used by major global rankings like THE to evaluate research quality.”

“The most critical finding is the massive structural bias within the FWCI calculation framework, where conference papers and book chapters in certain fields exhibit up to 5 times higher average FWCI-to-citation ratios than traditional journal articles,” said Moskovkin, who is the author of the 2026 study titled “Pandemic of ‘Predatory’ Rankings: Why academic integrity fails the stress test”.

Methodological scrutiny is needed

He noted that the study’s revelation that open-access (OA) publications can significantly inflate the FWCI values indicates that FWCI is influenced more by institutional publication strategies than by intrinsic scientific merit.

This discrepancy creates a misleading representation of a university’s research quality versus its bibliometric prestige, prompting a shift in the ranking debate towards methodological scrutiny rather than general criticism.

University World News also reached out to ShanghaiRanking, the Webometrics Ranking of World Universities, QS World University Rankings and THE World University Rankings for their views on the study’s findings but only received a response from the Webometrics Ranking of World Universities.

Isidro Aguillo, editor of the Webometrics rankings, and head of the Cybermetrics Lab, a research group of the Institute of Public Goods and Policies, which is part of the Spanish National Research Council, told University World News: “I ‘fast-read’ the study and it looks sound. Personally, I am not sure about the use of size-independent variables for ranking large groups like universities. I think they are valuable for individuals.

“Regarding the possible manipulation, my position is clear: Those universities involved in bad practices should be penalised.”

On the other hand, Duncan Ross, former chief data officer of the Times Higher Education (THE) from 2015 to 2025, told University World News: “It is always welcome when core institutional measurements such as FWCI are explored.”

“However, unfortunately this study displays fundamental misunderstandings of how Times Higher Education(THE) calculates FWCI, which significantly weakens the conclusion. THE has, over recent years, moved away from relying on FWCI as the sole measure of research quality, recognising that a broader basket of measures provides additional stability and provides universities with additional insight,” Ross added.

Reacting to Ross’ view, Calderon said: “It is worth noting that THE introduced changes in 2023 that revised the use of FWCI as the sole indicator within the Research Quality pillar, which accounts for 30% of the overall score. Rather than relying exclusively on FWCI, THE added three citation-based measures: research strength, research excellence, and research influence.”

Research strength assesses whether an institution’s scholarly output reaches the 75th percentile of FWCI performance; research excellence measures the proportion of publications that fall within the top 10% worldwide based on FWCI; and research influence, meanwhile, evaluates whether an institution’s research is recognised among the most influential scholarly outputs globally, according to Calderon.

“At first glance, these changes suggest a reduced reliance on FWCI. However, the revised measures remain fundamentally citation-based and continue to depend heavily on the influence and impact reflected through citation performance,” Calderon pointed out.

Take-away messages for HE policymakers

“Higher education policymakers and university leadership must stop treating global rankings as objective reflections of academic excellence,” Moskovkin said.

He recommended that universities should not hastily allocate resources to easily manipulated metrics like FWCI but instead focus on sustainable research integrity and strong internal peer-review processes. He emphasised the need for future research to conduct a direct sensitivity analysis of global ranking systems, addressing the inflation of FWCI highlighted in the study.

“We need to quantify the exact operational impact – for instance, measuring how a 1% artificial surge in an institution’s FWCI translates into a specific percentage jump in its overall THE ranking position,” Moskovkin indicated.

“This would definitively prove how vulnerable international ranking brackets are to mathematical anomalies and minor shifts in publication patterns.”

Expanding further, Heleta said: “The most important takeaway is that research evaluation, research metrics, and university rankings are highly complex and often unethically exploited and ‘gamed’ by academics, administrators and institutions.

“Critical discussion and engagement around all this is needed on the global, national and institutional levels, with ethics and values guiding the production of knowledge rather than the urge to game the system and move up the rankings.”