News Details

img

OfS Warns on AI Grading

University leaders lack strategic approach to AI use – OfS

New research from the Office for Students (OfS) and Advance HE explores how artificial intelligence (AI) is reshaping students’ experiences of higher education and have found that university leaders are failing to address AI issues strategically.

A frequent example is that many higher education institutions are continuing to use unsupervised assessments that can be completed to a high standard using AI tools without substantive engagement from students, according to the new report released on 11 November.

In the roundtable discussions and interviews that informed the research, students frequently described peers who had completed assessments using large language models, sometimes in just a few minutes, and received a similar grade to those who had completed the assessment themselves.

“Some students described their peers being unable to recall basic details about the assessment they had just submitted,” the report says.

“I’ve had people admit, peer-to-peer, ‘I wrote this whole essay with ChatGPT.’ […] Then they don’t get caught, they get a good grade probably, and then because it worked the first time, they do it on their next assessment,” one student told an online roundtable.

“… It is incredibly disconcerting when I plan, and write an essay completely by myself, and get the exact same grade as someone who used AI to write it,” one student told an online survey.

The research demonstrates the critical role of universities and colleges in equipping students with the AI skills they will need to thrive in life beyond higher education.

It also highlights the continuing risks AI poses for the effective assessment of students’ work and the credibility of qualifications, as well as the potential for AI to benefit higher education.

Josh Fleming, director of strategy and delivery at the OfS, which is the higher education regulator, said: “The scale and pace of both the challenge and opportunity presents a novel challenge for the sector and us as the regulator.”

Kathryn Harrison-Graves, deputy chief executive at Advance HE, said the research highlights the “real energy and creativity across the sector, with staff and departments leading the way in exploring what AI can offer”.

Staff concerned about assessments

Staff participants raised concerns that assessments do not consistently and reliably assess students’ knowledge and skills.

A key challenge is designing assessments that reliably demonstrate whether students have achieved the intended learning outcomes.

In many cases, assessments designed to prevent AI use, such as closed-book exams, presentations, practical assessments and performances, are likely to provide the most valid measure of student attainment,

However, discussions with students and staff also found that:

• Use of AI among teaching and professional services staff varies significantly, both within and between institutions. Some staff are developing innovative approaches, while others use AI rarely or not at all.

• Many students use AI in sophisticated ways to support their studies. However, students and staff expressed concerns that some uses displace the cognitive effort required for deep learning.

• Many students find their institution’s or department’s policies on AI difficult to understand. Some also felt that staff use of AI had reduced the quality of learning materials.

• AI literacy training for staff is often limited and generally voluntary, with levels of participation typically low.

• Students see AI literacy as important to their future employment prospects and would welcome more opportunities to develop AI skills.

Commenting on the research, Fleming said: “AI is rapidly reshaping higher education. This is a huge opportunity for universities and colleges to leverage the technology for the benefit of students and society.

“It also presents a generational risk to the system as it stands, threatening to undermine students' learning and institutions' ability to assess them. We need to see far greater efforts to tackle these risks, including far greater use of AI-free assessment.”

Lack of strategic approach

The research found that in many institutions, approaches to AI have been driven by the experimentation of individual staff rather than strategically by senior leaders.

Some institutions have established governance groups or transformation programmes, while others have appointed AI leads or ‘champions’ to help support the development of staff AI literacy and steer departmental approaches to AI.

“However, relatively few institutions that contributed to the research have adopted strategic approaches to AI and embedded these throughout their institutions,” the report says.

Participants called for sector, regulatory and government bodies to adopt a collaborative approach to AI, working with institutions to support improvements in their approaches.

Many highlighted the value of sharing effective practice in addressing common challenges. Participants generally suggested that new conditions of registration from the OfS on AI could be counterproductive and would risk constraining innovation.

Instead, some participants called for greater regulatory clarity on complying with existing conditions of registration in the context of AI.

Harrison-Graves said: “As institutions build on that work, there is a valuable opportunity to bring it together under a clear institutional direction. Students told us they would welcome greater clarity about what is expected of them, and staff are approaching AI with varying levels of confidence and support.

“Senior leaders are well placed to help by setting a shared vision for AI, supporting consistent and well communicated guidance, and making time, training and resources available to staff and students.

“Regular discussion at executive and board level, alongside giving staff the confidence to make good decisions in the interests of their students, will help institutions make the most of this work.”

Pledge to hold institutions accountable

In response to the research findings, the OfS said it will require institutions to ensure assessments are valid and reliable measures of students’ knowledge and skills.

In many cases, this will require the use of secure (AI-free) assessments. Institutions must also ensure courses are up-to-date and require students to develop relevant skills. Alongside the report, OfS has set out how institutions can comply with its regulation in the context of AI.

OfS has pledged to hold institutions to account when they fall short. Where evidence suggests institutions have not met its requirements, it will deploy its regulatory levers, which include imposing specific conditions of registration, monetary penalties or suspending access to student support funding and/or OfS grant funding.

It said it will support universities and colleges to strengthen their approaches to AI, including by clarifying how OfS regulatory requirements apply in the context of AI and sharing examples of effective practice.

In addition, OfS will continue to deepen sector understanding of how AI is reshaping higher education by undertaking further structured research to establish the scale and nature of change.

In response to the findings, Advance HE pledged to launch an AI focused Change Academy programme for member institutions early in 2027.

Designed as a peer-learning and action-learning programme connecting Advance HE members from across the globe, the Change Academy will “create space for leaders to pause, reflect and develop their institutional AI strategy”.

Advance HE also plans to convene sector leaders, industry experts and academics to explore effective approaches to responding to AI within teaching and learning, leadership, governance and inclusion.

It will also continually review frameworks and resources available to members via its Advance HE Knowledge Hub to provide guidance and shape discussion around ethical and appropriate use of AI in higher education.

  • SOCIAL SHARE :