Are the sustainability rankings measuring sustainability?

Times Higher Education (THE) recently published a new edition of their University Sustainability Rankings, now renamed Sustainability Impact Ratings, which evaluate universities according to their alignment with the UN’s Sustainable Development Goals (SDGs).

THE are presenting this as a new departure, creating a community, the Sustainability Impact Network, dedicated to the cause of global sustainability.

Membership in the community does not come cheap. A basic subscription is £3,620 (US$4,807), which provides entry into the network.

An advanced subscription, which gives “deeper data” on four SDGs, is £8,500, and a premium subscription for “institutions leading sustainability across their full portfolio and [who] want clarity across all SDGs, access to historical data and dedicated support” costs £18,300.

If that is still not enough, then there is an “Impact Ratings Enhanced Brand Package”. There are discounts for developing economies.

THE says that they will eventually stop publishing ranks and scores and classify institutions in bands or grades, but they are rather coy about when this will happen.

Although THE have presented their new “greater mission” with the usual fanfare, it looks like a bit of a flop. According to THE, 1,646 universities have been evaluated this year, and results are displayed for 1,603, compared to 2,526 and 2,318 in 2025.

That is a very significant fall, although THE claims to be satisfied, and it is possible that there will be more participants next year.

That number is now lower than the QS sustainability rankings, which include 2,021 institutions, although there is no option to withdraw from the QS rankings, while participation in THE’s remains voluntary.

More significant is that the number of participants has fallen below those included in the Universitas Indonesia GreenMetric rankings, 1,745, something that has been ignored by university leaders and the media. Indeed, the mainstream media seem to have forgotten that GreenMetric has published sustainability rankings since 2010, long before THE or QS.

Falling below GreenMetric ought to be an alarm call for THE, although their self-assurance is probably so great that they will not notice it.

Are the rankings reducing impact?

To briefly review the new rankings – it would be premature to call them ratings – the basic methodology is that universities are assessed, as before, on their compliance with the UN’s 17 Sustainable Development Goals.

That assessment is based on a combination of research metrics and various data submitted by institutions or gleaned from public sources. Universities are ranked according to their three best scores and one mandatory indicator, Partnership for the Goals.

One result of this is that universities are encouraged to focus on just a few of the goals. If participation in these rankings has any relation to contributions to sustainability, then THE’s methodology will have the perverse effect of reducing the overall impact of many universities.

Looking at this year’s results in detail, on average each university submitted data for 10 goals, leaving seven unassessed.

Rankings and AI

These ratings now make explicit and intense use of AI, as THE explains: “As part of our ongoing commitment to transparency and continuous improvement, we are incorporating large language model (LLM) technology into the validation of qualitative evidence submitted for the Impact Ratings. This section outlines how LLM is used, what it evaluates, and what universities should keep in mind when preparing their submissions.

• All submitted evidence is reviewed and scored by our LLM model. The model determines whether each piece of evidence is specific, generic, or not relevant to the indicator question.

• Although the LLM is the primary tool for validation, our human validation team remains in place to conduct quality assurance and review evidence as required.

• If a piece of evidence cannot be processed by the model (for example, due to inaccessible formats), it is not automatically scored as not relevant. Instead, the system flags it and forwards it to human validators, who review and score it directly.

• To ensure fairness, human validators follow the same visibility constraints as the AI – no clicking links or navigating beyond the visible content on a page.

• Evidence must be self-contained. The reviewer (AI or human) does not click on links or explore additional pages to locate information.

Documents or web pages containing multiple links pose a challenge in ensuring fairness across institutions. In line with our policy of evaluating a single piece of evidence per indicator, such files are now being rejected and will no longer be accepted as valid evidence.”

I suspect that not everyone will find this reassuring. It is true that LLMs are becoming less error-ridden and delusional. However, it is unlikely that they will become completely error-free any time soon, although they probably make the verification process more cost-efficient. The rejection of documents with multiple links seems unduly onerous and will probably penalise many unsuspecting institutions.

Also, THE is now demanding one item of evidence for each data point instead of three. On the surface, that seems to reduce the burden on universities but given the inaccuracy of AI and its human supervisors, it may well undermine the fairness and stability of the rankings.

How global are the rankings?

Representation in the current edition is very uneven. There is minimal participation from China, with only three universities listed. Germany is also represented by three universities and the Netherlands, Belgium and Israel by two each. Iran is totally absent this year and so is Russia.

In marked contrast, there is a substantial response from the Philippines with 160 universities, India with 110, Turkiye 74, Thailand 69, Indonesia 67, Pakistan 52, Egypt 45, Iraq 43, Malaysia 31, Japan 30, Saudi Arabia 28, Kazakhstan 24 and Bangladesh 22.

The turnout for the UK is quite good at 59 universities. For the US, it is very sparse. There are just 16 American institutions, only one of them in California, and the highest ranked of these is MIT. The Ivy League, Oxbridge, the elite écoles of France and the Indian Institutes of Technology are all absent.

Wide variations in rankings

These rankings consist mainly of countries and universities that could not get anywhere in objective research-based rankings. They also fail an important test of validity, which is reasonable stability across successive iterations.

Since the first edition of the THE sustainability rankings in 2019, there has been a massive turnover, partly due to methodological changes and partly to high-ranking institutions dropping out.

Consider what happened to the top 10 in the 2019 rankings. Table topper University of Auckland is barely in the top 100 in 2026, in 96th place, McMaster University has gone from second to 18th, King’s College London from 5th to the 101-200 band, Bologna from 9th also to 101-200.

Five universities – British Columbia, Gothenberg, KTH Royal Institute of Technology, Université de Montréal and the University of Hong Kong – have left the rankings altogether. Only one 2019 top 10 university is still there in 2026: Manchester, third in the first ranking, is now first this year.

This is extraordinary. For so many universities to rise or fall or disappear altogether casts serious doubt on the validity of the whole exercise. Comparing the results of the THE rankings with other sustainability rankings, of the top 11 in the THE rankings, exactly one can be found anywhere in the GreenMetric ranking, Universiti Sains Malaysia (USM).

What about the QS sustainability rankings? There seems to be almost no relationship at all. Manchester is 10th in the QS sustainability rankings.

The others are all much lower. USM is 247th, Griffith 76th, Queen’s University in Canada 59th, Western Sydney 172nd, Hanyang 162nd, Hokkaido 229th, Universiti Kebangsaan Malaysia 208th, National Taiwan University 72nd, Alberta 74th. Institut Agro in France is not there at all.

Now compare the top five UK universities with People and Planet, a ranking of the best British universities for sustainability which has been incorporated into the Good University Guide. Again, there is little connection. Manchester is at 34th place, Glasgow at 96th, Exeter at 28th, Northumbria at 16th and Queen’s University Belfast at 50th.

Looking from the other direction at the top five British universities in People and Planet. Manchester Metropolitan is 23rd among British universities in the THE rankings, Bath Spa is 58th, next to the bottom, and the University of West London is 50th. Reading is not ranked at all by THE, and Salford is 50th.

No doubt, THE would assert that their methodology is the best, even if it changes a lot, and the others are inferior.

Things that can be measured

A major problem with these rankings is that THE is looking at things that can be measured and may have little or no relationship with the stated goals.

So, for example, SDG 16 Peace, Justice and Strong Institutions is measured by graduates in Law and Civil Enforcement, No Poverty by the proportion of students receiving financial aid, Zero Hunger by graduates in food sustainability, Good Health and Well-Being by graduates in Health, Quality Education by the proportion of first-generation students, and Gender Equality by “Non-discrimination policies for transgender”.

A particularly ironic criterion is the measurement of Reduced Inequalities by the proportion of staff and students with disabilities. In the US and the UK massive numbers of students are claiming accommodations such as extra exam time, lenient deadlines and help with note-taking for a variety of conditions, including depression, autism, anxiety and learning disabilities.

It seems that the students who benefit from such privileges are disproportionately from wealthy families. It is irresponsible of THE to encourage such practices.

The new sustainability rankings may prove profitable for THE, but they are unlikely to be helpful for universities facing new and dangerous challenges over the coming decades.

Richard Holmes is an independent writer and consultant and the producer of the blog University Ranking Watch. This article was first published on his Substack here.

This article is a commentary. Commentary articles are the opinions of the author and do not necessarily reflect the views of University World News.