University of Derby
The HEFCE Review of Research Assessment
Response to the preliminary consultation
1. Introduction
The University of Derby is pleased to respond to the invitation to comment on the preliminary
issues for consideration in the forthcoming review of research assessment. The Research
Assessment Exercise (RAE) has progressively evolved over the past 15 years and so it is highly
appropriate to take a comprehensive review of research assessment. While there were
undoubtedly some serious shortcomings in the elaborate and complex methods employed in
RAE 2001 (notably in the assessment of the environmental sciences), devising a new improved
method of assessment, which satisfies the criteria of fairness, integrity, robustness and
transparency, is a huge challenge. There are certain key points that the University wishes to
contribute at this stage. Most of the sections which follow are linked to a question number in
the consultation paper to assist the HEFCE with the recording of these responses.
2. The definition of research excellence (Question 17c)
This is a matter of fundamental importance in the context of research assessment. The
University believes that the former Research Assessment Exercise adopted a relatively narrow
definition of research excellence. Excessive importance appears to have been attached to the
publication of works in international journals as an indicator of excellence. The University
believes that research excellence may be reflected by a number of different characteristics. A
research contribution of high quality might satisfy any one or more of the following.
(a)
(b)
(c)
(d)
The originality of the investigation, the approach, the method or the ideas.
The quality of the scholarship: the intellectual analysis and argument
The magnitude and depth of the study.
The utility of the research which may be expressed in its commercial application to endusers, its social value or its contribution to the quality of life.
3. Should each institution be assessed in the same way? (Question 17e)
Research assessment should recognise and accommodate the different emphases in the
missions of universities. It is as valid to conduct and assess applied research in terms of its
utility as it is to assess other forms of research in terms of the quality of the intellectual
analysis. Research assessment must recognise and accommodate the full spectrum of types of
research. Different universities, or even different departments in the same universities, may
demonstrate different types of excellence. Applied research of the highest quality may be
published in a regional or national journal rather than international journals because it may be
more appropriately placed there; not because it has a lower intrinsic value.
It would be gravely damaging to the reputation and functioning of UK universities if research
became the exclusive preserve of the research-intensive universities. Universities lacking a
research dimension would fail to attract postgraduate students from UK and overseas. Many
universities place emphasis on quality of teaching and widening access to students to higher
education. Engagement of staff in research at these institutions is of vital importance in terms
of maintaining the quality of the learning experience. In this type of organisation there is a
strong likelihood that research may be particularly geared to user-oriented outputs. HEFCE
should ensure that the assessment and funding of such research reflects the significant
contribution that it makes in each and every university.
2
4. The mode of assessment
The clear recognition of research excellence through its applied value necessitates a broader
approach to research assessment. Universities and/or departments should be required to present
the case for the types of research that they are engaged in and then to present the evidence
necessary to support the application for the assessment of research excellence. The University
believes that the assessment of applied research should draw upon evidence from the key
stakeholders such as the regional development agencies, small and medium sized enterprises,
corporation and professional bodies. (See the attached figure).
We concur with the Modern University Research Group that the following modes should make
a contribution.
(a)
(b)
(c)
The expert review (including users of research)
Metrics data based on recent performance should provide supporting data
Self assessed quality
We also agree with MURG that the assessment of research submissions on the basis of
historical ratings could be prejudicial in its effects. The panel might fail to recognise
significant changes in the quality of research undertaken by departments, either in the form of
improvement or decline. Equally, the factors involved in the assessment of research are too
complex to be reduced to a series of relatively insensitive algorithms. The lead mode of
assessment is most appropriately with an expert panel that takes into account statistical
measures but has the capacity to apply discretion. The research assessment should focus on
recent performance of the research and on the strategic plans for future developments. All
criteria need to be set down in advance in order to achieve fullest possible transparency.
5. Expert review
The University supports the proposal to include an important component of expert review.
This would need to operate at the individual researcher level. While it would be inappropriate
to aim for an identical procedure in each area of assessment, the RAE was characterised by too
much inconsistency. Some units of assessment indicated an intention to read 10% of cited
material; others indicated the intention to read a considerably higher proportion (even 100%) of
the cited material. It is difficult to understand how sound judgements can be founded on
assessments which exclude 90% of the evidence.
The University believes that this review needs to solve the problem of inconsistency in the
selection of areas of assessment. There were some notable lapses in consistency in the 2001
RAE. Sub sections of areas of assessment are not satisfactory if they are treated as a
Cinderella.
For example, this university experienced difficulty in locating appropriate areas of assessment
for some of its most significant areas of research. It was little surprise that the considerable
productivity and standing of such areas was barely recognised. The review is urged to tackle
the problem of the assessment of interdisciplinary research because it is through
interdisciplinarity that some of the most significant advances are made in research. This
understanding of the importance of interdsiciplinarity is now reflected in the European Union’s
Framework programme.
6. The purpose of Research Assessment
The University believes that the assessment of research serves three purposes.
3
(a) To provide a process by which universities through internal and external assessment
enhance the quality of their research.
(b) To provide outcomes of assessment which inform the funding of research through the
HEFCE core and core-plus mechanism.
(c) To provide universities and other stakeholders with objective external indicators of the
standing of the research performance.
The university believes that to restrict research assessment to one major national exercise once
every six years is far too limiting in terms of recognising change. There have to be
mechanisms by which new interdisciplinary research centres or departments can gain a
foothold on the slope leading to recognition of research excellence. The University can
demonstrate the considerable benefit of relatively small amounts of funding to new and
developing research areas. In terms of value for money it is difficult to conceive of any
research monies that achieve more. Modest funds found for new areas of research may achieve
significantly more, pound for pound, than when the same sums are advanced to top-rated
departments which already enjoy a very high QR income. Such departments are already wellpositioned to use their rating and QR income to lever further external funding.
7. The mechanism for research funding needs to evolve in parallel
This review is focused on the methods and procedures applied in the assessment of research.
However, our principal criticism of the HEFCE is in relation to its policy for the funding of
research. We supported selectivity in funding of the extent that was applied after RAE 1996.
This level of selectivity was necessary in order to ensure adequate funding for the departments
attaining national and international excellence. The policy introduced in December 2001
brought about excessive selectivity, which had a major adverse effect on many universities
across the sector (All Parliamentary Meeting at the House of Lords, July 2002). This University
achieved a 30% improvement in its average rating, increasing from one to four the number of
units of assessment rated at 3a, but this improvement was rewarded with a 66% reduction in
funding.
The University urges the HEFCE to consider the mechanism for the future funding of
universities in respect of their research, in parallel with the development of the method of
assessment. Universities must be in a position to know the rules and the consequences of the
outcomes in funding terms before they make submissions applications in the future. Can
organisations be reasonably expected to devote significant resource to preparing submissions
without knowing the potential consequences of the outcomes? It is simply unacceptable to
move the goal posts after the exercise.
The University believes that there would be merit in considering a two-tier mechanism.
Institutions wishing to gain a threshold status for a research centre would submit a selfassessment which is subject to verification. If successful the centre obtains the threshold status
and a basic level of funding which is included in the institutional core funding. If the institution
wishes to obtain QR funding for a research centre it will be necessary to submit as part of a
periodic co-ordinated research assessment. The outcome would determine the research rating
for the centre and the QR funding that it is awarded.
8. Further points on funding


The review is recommended to consider best practice in business development in industry.
The HEFCE should ensure that it has sufficient funds to support the funding mechanism
that it applies. It did not do this in 2001.
4




The HEFCE may find it necessary to increase the number of grades in order to overcome
the bunching at 5 and 5* but it should not introduce so many grades that the panels are
unable to grade the submissions with reasonable confidence.
The HEFCE could use grades for report purposes but it could use a scale gradient for the
purposes of funding to avoid the major steps in funding.
The HEFCE should avoid excessive selectivity in the scale of funding. Much can be
achieved with moderate levels of funding. Modest funds for areas operating at national
excellence (RAE 2001 = 3a and 3b) represent excellent value for money.
Some seed funding should be available to new entrants with evidence for potential.
Robert M Faithorn
Pro Vice-Chancellor
Chair of the University Research Committee
Revised PHB/PW/DC/MC/MG/WM/AG 28th November 2002