Review of Research Assessment
Notes from the Joint Medical Advisory Committee (JMAC) meeting held on
27 November 2002
Present:
Members
Professor Sir Leszek Borysiewicz (Chair)
Principal, Imperial College School of Medicine
Professor Dame Ingrid Allen
Medical Biology Centre, Queens University Belfast
Mr Alec Cumming
Chief Executive, Grampian University NHS Trusts
Professor Richard Hobbs
Head of General Practice, Univ. of Birmingham
Professor Peter Holmes
Vice Principal (Research), University of Glasgow
Professor William J. Hume
Dean, Leeds Dental Institute
Professor David Levison
Mrs Ann Lloyd
Dean, Faculty of Medicine, Dentistry and Nursing,
University of Dundee
Director NHS Wales
Professor John Tooke
Dean, Peninsula Medical School
Professor Stephen Tomlinson
VC, University of Wales College of Medicine
Professor Ian Weller
Dept Sexually Transmitted Diseases, Royal Free &
University College Medical School
Head of Division of Nursing and Midwifery, King’s
College, London
Professor Jenifer Wilson-Barnett
Observers
Dr Jane Ashwell
National Assembly for Wales
Dr Joan Box
Medical Research Council
Dr Robin Cairncross
Scottish Executive, Home and Health Department
Professor John Murray
University of Newcastle Dental School
Professor Maggie Pearson
Deputy Director Human Resources, DH
Professor Peter Rubin
Dean, Faculty of Medicine and Health Sciences,
University of Nottingham, Medical School
UK Central Council for Nursing, Midwifery and Health
Visiting
Dr P Walter
General
1.
The Committee asked for clarification about how the review of research assessment related
to the Fundamental Review of Research undertaken in 1999. It was noted that a lot of effort had
been put into responses to the Fundamental Review. The Committee was reassured to learn that
the responses from 1999 would be used to inform this review, in addition to the responses to the
current consultation.
2.
It was generally acknowledged that the RAE process worked well, and was an effective
method of measuring research quality. It had also had the effect of raising the profile of research
and of increasing research quality within a competitive environment. The Committee felt that the
RAE did not need a radical over-haul, but that it would benefit from a number of smaller-scale
changes. The area that raised most concern, however, was the way in which the results of the
RAE had been translated into research funding.
3.
It was acknowledged that any research assessment system should not disincentivise
teaching or service responsibilities.
Clinical research
4.
The Committee highlighted the treatment of clinical trials’ research in the RAE. It was noted
that clinical trials were a hugely important area of research in the UK, but that their importance was
not necessarily accurately reflected because of the definition of the categories of research
excellence. The nature and subject matter of clinical trials often created difficulties in being able to
assess the research as being of ‘international’ importance. As a result, this could have a huge
impact on many areas within the health sector, e.g. nursing, physiotherapy.
5.
It was noted that the treatment of clinical research in the RAE ran counter to the NHS R&D
strategy to improve clinical research. Clinical academics within institutions often chose to engage
in academic medical research due to its perceived higher standing within the RAE, rather than
undertake clinical research that might only be recognised as being of ‘national’ importance.
Staff specific issues
6.
The reduced amount of time available to clinical academics to undertake research, in
addition to their service responsibilities, needed to be acknowledged within a research assessment
system.
7.
The Committee considered that the nature of the RAE could be discriminatory against
certain groups of staff, for example women who might have had career breaks during the
assessment period, or staff who had taken breaks due to ill-health. These staff were often not
returned to the RAE. The question of whether it should be a requirement for institutions to submit
all staff to the RAE should be considered.
8.
The concept of returning research groups to the RAE rather than individual research staff
was discussed. Assessment of the quality of a research group might also include assessment of
the quality of the research environment, thereby highlighting any erosion of the research
infrastructure.
Panels
9.
The role of sub-panels was discussed – allocating ratings to specific research sub-groups
was all well and good, but if these ratings did not affect the overall rating, there was no financial
benefit to the institution. The Committee considered that ratings awarded to research sub-groups
should carry a financial reward.
10.
It was agreed that the RAE had dealt with multi-disciplinary submissions well.
11. The issue of comparability of assessment between different units of assessment was raised.
The use of more umbrella panels in the future would help ensure consistency.
Research training
12. The importance of assessing the quality of research training was discussed. It was noted
that the Academy of Medicine’s response to the 1999 Fundamental Review of Research had
highlighted the need to assess the quality of research training within institutions as well as the
quality of research. It was agreed that the capacity to assess research training was important,
however, the consensus was that the quality research training should not be awarded a separate
score.
Use of RAE ratings
13. The purpose for which RAE ratings were used by other funders of research was a significant
issue – for example, the use of ratings by Research Councils to determine research grants. It was
noted that other funders of research needed some guidance about research strengths and
capacity in order to decide where to invest funding.
14. Some members of the Committee considered that it would be helpful for a research
assessment process to be able to disaggregate between retrospective and prospective investment
in research.
Timetable
15. There was some discussion about whether the RAE process should be an annual exercise
or retain its current frequency of once every 5 years. On balance, there was more support for
retaining an infrequent, but large-scale, assessment process, rather than introducing an annual
assessment process.