Southampton Institute
Departmental Response from the School of Law,
Southampton Business School.
Background
An invitation from the joint funding bodies’ review of research assessment was
published in the Times Higher Education Supplement, in addition to an invitation
being sent to the Southampton Institute to contribute to the discussion on how
research in the higher education sector should be assessed in the future.
The School of Law have enjoyed funding from previous RAE assessments by
achieving grade 3B in the 1996 round and a 3A in the 2001. The research community
in the School of Law therefore expressed the wish to respond to the invitation from a
departmental level as recipients of funding in the past. This was agreed to by the
Chair of the Institute Research and Scholarship Committee. Southampton Institute is
also responding from an institutional perspective.
Introduction
The School of Law would like to make the following general comments:
The consultation paper is premised on the continuation of a system, which was
originally developed to support the needs of the science based subjects, in order that
they could compete with other institutions internationally. The method of assessment
and the funding thereafter reflected this. The School of Law would challenge this
basic assumption and consider that different criteria could be used for subjects
broadly within the scope of the humanities and perhaps even more specifically subject
based. With the drive to wider participation, credit should also be given to those
departments who can demonstrate a realistic synergy between the quality of the
subject based research and the promotion of high quality teaching and the student
experience, both undergraduate and post graduate. In particular in those HEIs who
consider themselves to be research institutions.
It is recognised that the Government are considering a new formula for the funding of
HEIs which includes the concept of a ‘core plus’ model of institution. It would be
unfortunate in the extreme if this model only supported research in those institutions.
which had identified themselves as ‘research institutions’. As the definition of an HEI
includes the carrying out of research every HEI should be funded so to do. Additional
funds could be allocated to Departments that could demonstrate emerging excellence
in research. It was an unfortunate and discriminatory outcome from the 2001 RAE
that those UOAs who had achieved a 3A or B were no longer to be funded when they
had a legitimate expectation so to be. To demonstrate excellence in research a more
robust model of assessment should be employed, with each piece of work assessed in
the same way as a student thesis would be marked using published criteria rather than
reflecting the reputation of the institution.
Frequency of the reviews should be adjusted to allow for the completion of longer
projects and to avoid the ‘bunching’ of submission for publication. Perhaps a number
of like HEIs could be assessed in any one year in order that various types of
institutions could be assessed against each other. One suggestion is that selfassessment could take place with institutions pairing up to verify each other’s results.
The pairs would be changed for each exercise.
It was the strong opinion of the School of Law that the number of pieces submitted
should be reduced in order that academics could concentrate on quality rather than
quantity.
The most important features of an assessment process are fairness and transparency.
Group 1: Expert review
a)
b)
c)
d)
e)
Prospective assessment should be an element, as this would have the
facility to support emerging centres within the new university/college
sector. Notwithstanding the government view that there should be a degree
of difference among HEIs, to comply with the legal definition of an
institution of higher education some research must take place within that
institution. The quality of this research should be recognised by the
funding bodies; particularly where there are centres of excellence within
an institution, which mainly concentrates on teaching and contacts with the
community. To establish credibility there must be an element of
retrospective assessment. Feedback from the UOA 36 Law in all the
previous assessments, the panel has commented that only RAE 2 was
considered. However, old universities should not be permitted to trade on
their reputation to assert that all future research carried out there will be of
high quality.
As to type of data to be considered there were two schools of thought. i.
That the entire research output should be submitted with one primary piece
submitted in full plus a list of additional publications. ii. That only one
piece per academic should be submitted. This would mitigate against the
rush to publish work, and would emphasise quality rather than quantity.
One problem with the 2001 RAE was the perception that there were no
experts in certain emerging bodies of law on the panel. The move toward
transparency by publishing the work entered by each institution was to be
welcomed. However, it was strongly felt that the pieces of work, which
were assessed at international level, should be identified, as work
submitted by some 5* rated institutions was considered to be of less
quality. In addition why is the piece considered to be a primary reference
rather than additional. This would also give a benchmark for other aspiring
groups.
The assessment should be made according to groups, as this is the only
way to reward those who had produced the work.
The assessment of subjects or themes is unavoidable if true comparability
is to take place.
It is a major weakness to assess output rather than outcomes.
Group 2: Algorithm
a)
b)
c)
d)
e)
No.
As indicated in the consultation paper.
No
Groups could cite each other by arrangement.
The problem with this approach is that it assesses research activity rather
than quality. A citation is not necessarily a sign of quality. It is too easy for
groups of associates to cite each other or themselves. Alternatively a
citation could be a criticism.
Group 3: Self-assessment
a)
b)
c)
d)
e)
f)
See response to Group 1: Expert review. In addition institutions could
include aspects of their research which support their particular mission, but
all institutions should indicate how their research relates to their teaching.
Assessments should be a combination of both retrospective to establish
credibility and prospective to allow for emerging departments to
demonstrate their potential.
Each institution will have a Quality Standards and Assessment department
which monitors all work carried out within the institution. Some
departments, it is accepted are more robust than others, but generally
accepted benchmarks could be issued. Each institutions could then add to
these benchmarks to reflect their mission.
Institutions could be paired to validate each other’s self-assessment against
a published set of criteria. The pairing would reflect the position of the
institution within the sector and each pairing would be changed for each
assessment exercise.
All assessment takes time and effort if done correctly but is necessary for
the transparency of allocation of public funds.
The institution could demonstrate the correlation of the research with the
mission of the institution.
Group 4: Historical ratings
This would not be an acceptable system, as it would only re-introduce the old binary
divide. In addition it would militate against current institutions following the
examples of Warwick and, more recently Portsmouth, Universities in becoming
respected institutions, which carry out good research, in a relatively short time.
Group 5: Crosscutting themes
a)
An assessment of the research base would validate the quality of academic
activity within an HEI. This in turn could be used to support undergraduate
teaching; post-graduate teaching and support for research degree candidates.
These activities could then be properly funded to include the research activity
necessary to support the teaching and research student support.
b)
c)
d)
e)
f)
g)
The assessment should take place on a rolling programme in order that there is
less competition for publication at any given time. This would allow for a
more considered allocation of space by the editors of academic journals in
order to include good well researched articles from academics from a more
diverse number of institutions.
Research should be assessed against a published set of criteria for each
subject. The quality of Articles should be assessed against published criteria.
It is an accepted fact that not all subjects need the same amount of funding to
support good research. For some subjects both time and a good library are the
basic essentials, whereas for the science based subjects major funding may be
needed to purchase expensive technology. However, it is the allocation of
funding between institutions, which creates the greatest inequality.
 If the quality of research is to be benchmarked against international
competition, the criteria for what constitute international should be
transparent. In the 2001 RAE some work which was assessed as 5/5* was
well below this standard and included chapters in undergraduate textbooks
which did not go beyond accepted theories and doctrine. How did these
qualify as primary reference point?
 If numerous departments produce quality research they should be
rewarded and the size of the subject pot should reflect this.
 Given that public money must reflect the public interest in the strategic
importance of areas of specific research then this may give rise to
additional funds in any ‘subject pot’.
 This would be difficult to support as it would militate against emerging
research interests and ‘blue skies’ research, which may or may not have an
economic benefit, which may or may not attract external funding.
 The needs of research active academics change with time and improved
technology. To base the ‘subject pot’ on historic distribution disregards
this fact.
We support the concept that the system should provide a ladder of
improvement so that all researchers and institutions have the opportunity to
demonstrate potential. This would necessitate the retention of core funding for
research in all HEIs with the re-introduction of the CollR grants in order that
each institution may use those grants selectively to achieve higher quality
ratings in future research assessment exercises. This also encourages
collaboration and the sharing of good practice.
Each RAE panel published it’s own assessment criteria for the 2001 RAE.
These differed substantially in a number of different ways. In addition, in the
informal feedback from the Law panel it was expressed that only RA2 was
assessed and the research culture and support was not included. This militates
against emerging departments who must set up the facilities and procedures to
encourage a research culture and output. But these efforts were not recognised
in the assessment. Each subject, therefore already use different assessment
criteria. If this is so then the criterion should be open and published in order to
create a proper benchmark against which departments can assess the quality of
the research articles they submit for assessment. If self-assessment is adopted
then each institution can publish its own criteria.
It should be the department or research group who determine the submission
of their research. This would reflect the knowledge and experience of those
h)
i)
academics within the department to put together a submission which best
reflected the research carried out therein. This would not be unfair to the
institution, as the departmental submission would in turn reflect the strategic
mission of the institution in the research carried out.
These issues should be addressed by the institutions under their Human
Resource policy.
If the assessment were both fair and transparent, then the remainder of the
points would naturally be included.
Professor Patricia Park
Head of Law Research Centre
Tel: 023 8031 9850
e-mail: patricia.park@solent.ac.uk