REF selection criteria for
communication to staff
Policy for Cass Business School
1. All researchers in the School that meet the REF definition of Category A staff with
the required number of outputs rated 3* and above will be considered for inclusion, in
line with the City University Code of Practice on the selection of staff for REF (see
extract at Annex A).
2. The final decision on inclusion rests with the School REF Panel which, in turn, will
take advice from internal and external experts. The School REF Panel comprises
Professor Steven Haberman, Director and Deputy Dean (Chair), Professor Igor
Filatotchev, Associate Dean for Research and Enterprise, Professor Cliff Oswick,
Deputy Dean and Head of the Management Faculty, Professor Lucio Sarno, Associate
Dean and Head of the Finance Faculty, Dr Ben Rickayzen, Associate Dean and Head
of the Faculty of Actuarial Science and Insurance and John Montgomery, Research
Manager (Secretary).
3. The School REF Panel will use the views of internal and external experts
supplemented as appropriate by reference to academic journal quality guides [see 4.
below] to determine the quality of research outputs, adopting the interpretations of
quality of REF Main Panel C (see Annex B).
4. Upon taking up a post at Cass Business School, Faculty members receive academic
journal quality guides of world-class high impact journals which are the premier
targets for publication. Each subject area also receives a list of top quality specialist
field journals that are also considered. Success in publishing in these outlets is a prime
criterion for Faculty performance.
5. In ambiguous cases, the School REF Panel and advisors will seek additional
experts to assess the quality of research outputs before coming to a judgement.
6. The School REF Panel expects co-authors to reach initial agreement among
themselves over who should be assigned a joint output. In all cases the member of
staff to whom the output is assigned must have made a significant contribution to it
(see 7(iv) below). The School REF Panel will make a final decision on the assignment
the output, taking account of the strategic implications for the submission. In
particular, if the assignment of the co-authored output results in one of the authors
having sufficient outputs that pass the Panel’s quality criteria (is 3* or better) to be
1
included in the REF while the other author still would not have sufficient outputs,
assignment will be to the former.
7. The inclusion of an output in the submission of the School to REF is subject to the
following conditions:
(i) Outputs deemed to be of a quality that does not meet the definition of 3* or 4*, or
which do not meet the published definition of research for the purposes of the REF,
will not normally be included unless there is a strategic reason for a member of staff
with no individual circumstances as set out in the University’s Code of Practice on the
selection of staff for REF to be included in the submission with fewer than four
outputs of 3* or 4* quality. Research outputs will also be required to fit the overall
strategy for the presentation of research activity within the relevant Unit of
Assessment submission in order for staff to be included.
(ii) The output must have no significant material in common with another output that
is also included in the submission of the University to the same Unit of Assessment
regardless of whether the latter output is submitted by the same or a different member
of staff.
(iii) The output must not have significant material in common with an output
published prior to 1 January 2008. Where an output is submitted which has significant
material in common with one published prior to 1 January 2008, it is only considered
eligible for submission if significant new material has been incorporated. In this case
an explanation has to be provided of how far the earlier work was revised to
incorporate new material.
(iv) In the case of co-authored outputs, the member of staff who claims the output
must have made a significant contribution to it. The University may be audited on this
and staff may therefore be asked to provide additional information in relation to
outputs to be included in the submission where their contribution is not sufficiently
clear from the output itself. This applies both to outputs where co-authorship is with
colleagues included in the School’s REF submission and to those where the
collaboration is with external colleagues.
8. The School may seek permission from the University during 2013 to strengthen the
outcome of its REF 2014 submission further by applying a higher threshold for
inclusion of staff than that already set out in the University Code of Practice. This
may mean that fewer staff are included in the submission than would otherwise be the
case. Any agreed change to the selection criteria will be applied equally to all staff
under consideration for inclusion in the Unit of Assessment. Staff will be notified of
any changes to be made to the selection criteria following University approval.
2
Annex A
CITY UNIVERSITY LONDON
Code of Practice on the selection of staff for REF 2014
Extract:
Establishment and communication of quality thresholds for inclusion
It is already known that, following the REF, recurrent funding for research from the
Funding Council (known as QR funding) will only be provided for activity rated 3*
and 4* (defined as quality that is internationally excellent and world-leading
respectively). The ratio of funding between 3* and 4* activity will not be known until
after the REF results are published but it is anticipated that activity rated 4* will be
funded at a level significantly higher than activity rated 3*. The University will be
seeking to maximise the proportion of activity rated at these levels in its REF outcome
and will only include staff with outputs considered to be of 3* or 4* quality in its
submission. The assessment of quality will be made by senior staff in the relevant
subject area within each School, with calibration of the assessments provided by
external reviewers as appropriate, primarily through the University’s Annual Research
Quality Monitoring (ARQM) process. Further details of the criteria for inclusion are
set out under ‘Criteria and terms of reference for REF decision-making bodies’
(section 5) of the Code.
While the REF guidance on submissions defines eligibility and the number of outputs
expected to be submitted for an individual member of staff, the determination of the
appropriate quality threshold for inclusion of staff in the REF submission is a matter
for individual universities. As indicated previously, this may vary across UoAs and
Schools have therefore been asked to provide further information on the basis on
which they will determine the inclusion of staff for each UoA, for approval by the
REF Steering Group and subsequent communication to the staff concerned. This will
include publication via the University REF microsite of the criteria to be used in the
selection process and the names of those involved in the process for each School as an
appendix to the Code of Practice in its final form. In drawing up this information,
Schools will refer to the published criteria to be used by the relevant REF Main Panel
but may wish to supplement this further according to the strategy to be adopted for a
particular UoA.
Staff may request that an external review of an individual output is carried out, where
this has not already been undertaken, if they reasonably consider that the relevant
specialist expertise is not represented within the group undertaking the assessment of
quality in the School.
3
Annex B
The criteria for assessing outputs will be interpreted as follows, in line with the
criteria to be adopted by REF Main Panel C:
• Originality will be understood in terms of the innovative character of the research
output. Research outputs that demonstrate originality may: engage with new and/or
complex problems; develop innovative research methods, methodologies and
analytical techniques; provide new empirical material; and/or advance theory or the
analysis of doctrine, policy or practice.
• Significance will be understood in terms of the development of the intellectual
agenda of the field and may be theoretical, methodological and/or substantive. Due
weight will be given to potential as well as actual significance, especially where the
output is very recent.
• Rigour will be understood in terms of the intellectual precision, robustness and
appropriateness of the concepts, analyses, theories and methodologies deployed
within a research output. Account will be taken of such qualities as the integrity,
coherence and consistency of arguments and analysis, such as the due consideration
of ethical issues.
In assessing work as being four star (quality that is world-leading in terms of
originality, significance and rigour), some of the following types of characteristics are
expected:
• outstandingly novel in developing concepts, techniques or outcomes;
• a primary or essential point of reference in its field or sub-field;
• major influence on the intellectual agenda of a research theme or field;
• application of exceptionally rigorous research design and techniques of investigation
and analysis, and the highest standards of intellectual precision;
• instantiating an exceptionally significant, multi-user data set or research resource.
In assessing work as being three star (quality that is internationally excellent in terms
of originality, significance and rigour but which falls short of the highest standards of
excellence), some of the following types of characteristics are expected:
• an important point of reference in its field or sub-field;
• contributing important knowledge, ideas and techniques which are likely to have a
lasting influence;
• application of robust and appropriate research design and techniques of investigation
and analysis, with intellectual precision;
• generation of a substantial, coherent and widely admired data set or research
resource.
In assessing work as being two star (quality that is recognised internationally in terms
of originality, significance and rigour), some of the following types of characteristics
are expected:
• providing valuable knowledge to the field or sub-field and to the application of such
knowledge;
• contributing to incremental and cumulative advances in knowledge in the field and
subfield;
• thorough and professional application of appropriate research design and techniques
of investigation and analysis.
4
In assessing work as being one star (quality that is recognised nationally in terms of
originality, significance and rigour), some of the following types of characteristics are
expected:
• useful knowledge, but unlikely to have more than a minor influence in the field;
• an identifiable contribution to understanding, but largely framed by existing
paradigms or traditions of enquiry;
• competent application of appropriate research design and techniques of investigation
and analysis.
Research will be graded as ‘unclassified’ if it falls below the quality levels described
above or does not meet the definition of research used for the REF.
5