PLANNING FOR THE
RESEARCH EXCELLENCE
FRAMEWORK AT GCU
page 1
The Research Excellence Framework (REF) is the new system for
assessing research in UK higher education institutions (HEIs).
It replaces the Research Assessment Exercise (RAE).
The REF will be undertaken by the four UK higher education funding
bodies:
• Higher Education Funding Council for England (HEFCE)
• Scottish Funding Council (SFC),
• Higher Education Funding Council for Wales (HEFCW)
• Department for Employment and Learning, Northern Ireland (DEL).
The exercise will be managed by the REF team based at HEFCE and
overseen by the REF Steering Group, consisting of representatives
of the four UK higher education funding bodies
page 2
The REF will:
o inform the selective allocation of research funding to HEIs on the basis of excellence
o provide benchmarking information and reputational yardsticks
o provide accountability for public investment in research and demonstrate its benefits.
The REF will be a process of expert review. Institutions will be invited to
make submissions to 36 units of assessment (UoAs).
The submissions will be assessed by an expert sub-panel to be established
for each UoA, working under the guidance of four main panels to ensure
common procedures and consistent application of the overall assessment
standards
page 3
Under current plans for the REF, three distinct elements will
be assessed for each submission:
o the quality of research outputs
o the wider impact of research
o the vitality of the research environment
The profile method of assessing research quality introduced
in RAE2008 will be used for REF2014:
4* World leading
3* Internationally Excellent
2* Internationally Recognised
1* Nationally Recognised
page 4
REF 2014 Main Panel Configuration
Main Panels covering 4 broad categories of “related” disciplines
A: Health, Medicine and Biological Sciences
B: Physical Sciences and Engineering
C: Economic and Social Sciences
D: Arts and Humanities
These Main Panels will take an overview of reports from
the work of subpanels reviewing specific Units of
Assessment (UoA)
page 5
REF 2014 Sub Panel Configuration
REF2014 will have fewer, larger sub panel UoAs.
There will be 36 Panels in REF2014 (c.f. 68 for RAE2008)
GCU made 14 submissions in 13 separate Units of Assessment
in RAE2008.
As a result of changes to Panel structure, the subject
submissions made by GCU in RAE2008 fall into 12 new
REF2014 Units of Assessment
page 6
REF2014 Main Panel A: Units of Assessment
1
Clinical Medicine
RAE2008 Sub-panels 1-5 (Cardiovascular
Medicine; Cancer Studies; Infection and
Immunology; Other Hospital Based Clinical
Subjects; Other Laboratory Based Clinical Subjects)
2
Public Health, Health Services
and Primary Care
RAE2008 Sub-panels 6-8 (Epidemiology and Public
Health; Health Services Research; Primary Care
and Other Community Based Clinical Subjects)
3
Allied Health Professions,
Dentistry, Nursing and Pharmacy
RAE2008 Sub-panels 10-13 (Dentistry; Nursing and
Midwifery; Allied Health Professions and Studies;
Pharmacy)
4
Psychology, Psychiatry and
Neuroscience
RAE2008 Sub-panels 9 and 44 (Psychiatry,
Neuroscience and Clinical Psychology; Psychology)
5
Biological Sciences
RAE2008 Sub-panels 14 and 15 (Biological
Sciences; Pre-clinical and Human Biological
Sciences)
6
Agriculture, Veterinary and Food
Science
RAE2008 Sub-panel 16 (Agriculture, Veterinary and
Food Science)
page 7
REF2014 Main Panel B Units of Assessment
7
Earth Systems and Environmental
Sciences
RAE2008 Sub-panel 17 (Earth Systems and
Environmental Sciences)
8
Chemistry
RAE2008 Sub-panel 18 (Chemistry)
9
Physics
RAE2008 Sub-panel 19 (Physics)
10
Mathematical Sciences
RAE2008 Sub-panels 20-22 (Pure
Mathematics; Applied Mathematics; Statistics
and Operational Research)
11
Computer Science and Informatics
RAE2008 Sub-panel 23 (Computer Science
and Informatics)
12
Aeronautical, Mechanical, Chemical and
Manufacturing Engineering
RAE2008 Sub-panels 26 and 28 (Chemical
Engineering; Mechanical, Aeronautical and
Manufacturing Engineering)
13
Electrical and Electronic Engineering,
Metallurgy and Materials
RAE2008 Sub-panels 24 and 29 (Electrical
and Electronic Engineering; Metallurgy and
Materials)
14
Civil and Construction Engineering
RAE2008 Sub-panel 27 (Civil Engineering)
General Engineering
RAE2008 Sub-panel 25 (General
Engineering and Mineral & Mining
Engineering)
15
page 8
REF Main Panel C: Units of Assessment
16
Architecture, Built Environment and
Planning
17
Geography, Environmental Studies and RAE2008 Sub-panels 32 and 33 (Geography and
Archaeology
Environmental Studies; Archaeology)
18
Economics and Econometrics
RAE2008 Sub-panel 34 (Economics and
Econometrics)
19
Business and Management Studies
RAE2008 Sub-panels 35 and 36 (Accounting and
Finance; Business and Management Studies)
20
Law
RAE2008 Sub-panel 38 (Law)
21
Politics and International Studies
22
Social Work and Social Policy
RAE2008 Sub-panel 39 (Politics and International
Studies)
RAE2008 Sub-panel 40 (Social Work and Social
Policy & Administration)
23
Sociology
24
Anthropology and Development Studies RAE2008 Sub-panels 42 and 43 (Anthropology;
25
Education
RAE2008 Sub-panel 45 (Education)
26
Sports-Related Studies
RAE2008 Sub-panel 46 (Sports-Related Studies
RAE2008 Sub-panels 30 and 31 (Architecture and
the Built Environment; Town and Country Planning)
RAE2008 Sub-panel 41 (Sociology)
Development Studies)
page 9
REF Main Panel D: Units of Assessment
Area Studies
RAE2008 Sub-panels 47-50 (American Studies and
Anglophone Area Studies; Middle Eastern and African
Studies; Asian Studies; European Studies)
28
Modern Languages
RAE2008 Sub-panels 51-56 and 58 (Russian, Slavonic and
East European Languages; French; German, Dutch and
Scandinavian Languages; Italian; Iberian and Latin American
Languages; Celtic Studies; Linguistics
29
English Language and Literature
RAE2008 Sub-panel 57 (English Language and Literature
30
History
Sub-panel 62 (History
31
Classics
RAE2008 Sub-panel 59 (Classics, Ancient History,
Byzantine and Modern Greek Studies
32
Philosophy
RAE2008 Sub-panel 60 (Philosophy)
33
Theology and Religious Studies
RAE2008 Sub-panel 61 (Theology, Divinity and Religious
Studies)
34
Art and Design: History, Practice and
Theory
RAE2008 Sub-panels 63 and 64 (Art and Design; History of
Art, Architecture and Design)
35
Music, Drama, Dance and Performing
Arts
RAE2008 Sub-panels 65 and 67 (Drama, Dance and
Performing Arts; Music)
36
Communication, Cultural and Media
Studies, Library and Information
Management
RAE2008 Sub-panels 37 and 61 (Library and Information
Management; Communication, Cultural and Media Studies).
27
page 10
REF TIMELINE
DATES
October 2010
November 2010
RESEARCH EXCELLENCE FRAMEWORK
(REF)
KEY MILESTONES
Main Panel Chairs Designate appointed
Announce panel structure and start
recruitment of expert panels
Deadline for applications for sub panel
chairs
Deadline for nominating panel
members
Sub-panel chairs appointed
Reports from the impact pilot exercise
December 2010
Early 2011
Mid 2011
Mid 2011
Panel members appointed
Panels begin meeting
Panels consult on criteria
Guidance on submissions published
Late 2011
Panel criteria and methods published
Early 2013
Late 2013
Jan- Dec 2014
December 2014
Submission system operational
Submissions deadline
Panels assess submissions
Outcomes published
July 2010
17 September 2010
8th October 2010
page 11
CHANGES IN REF2014 FROM RAE2008
Larger units of assessment, broadly of similar size (1000 FTE)
Fewer but larger submissions likely in each UoA (c.f. RAE2008)
Bulk of quality scores will be based on assessment of research
outputs (ca 60 % on publications)
Significant emphasis on research environment (ca 20 %)
Need to demonstrate wider research impact at research
grouping level (ca 20 %) based on report of pilot exercises*
Funding reward likely for research excellence (i.e. 4* and
3* International quality levels) only
page 12
REF RESEARCH IMPACT PILOT STUDY
http://www.hefce.ac.uk/research/ref/impact/
Main conclusion:
Expert review of case studies is an appropriate means
of assessing impact
It is essential that impact should be defined broadly to
include social, economic, cultural, environmental,
health and quality of life benefits.
Impact purely within academia should not be included
in this part of the REF
page 13
REF IMPACT PILOT STUDY (2)
Impacts from research typically develop over extended periods
of time and institutions should be able to submit impacts at any
stage of development, so long as some change or benefit
beyond academia has taken place:
The REF should only assess the impact that has taken place
during the assessment period and not attempt to anticipate
future or potential impact.
In selecting case studies, institutions should focus on those
impacts that are more fully developed or significant
‘interim’ impacts.
Institutions should be permitted to submit impacts that
evolve over long time-frames to successive REF exercises,
with each REF assessing the specific impacts that have
taken place during the assessment period.
page 14
REF IMPACT PILOT STUDY (3)
It is clear from the pilot study that impacts are not just economic or
STEM subject related.
The Expert Panels reported that arts and humanities subjects can be
assessed for impact using appropriate discipline specific indicators.
REF panels will develop more detailed guidance on what constitutes
impact in their disciplines.
This should include guidance about the types of impacts and indicators
anticipated from research in their disciplines, expanding on the initial list
provided by the funding bodies, and guidance on what constitutes ‘interim’
impact.
The guidance should be flexible enough to allow for a wide variety of
impacts and indicators, including impacts that panels may not
anticipate
page 15
REF IMPACT PILOT STUDY (4)
Case studies should contain all the relevant information and evidence
required by panels to come to a judgement
Case studies should include impact indicators within the narrative that are
meaningful, contextualised and relevant in demonstrating the particular case.
Case studies should be based on high quality research (2* level minimum).
Institutions must justify the quality of the underpinning research in their
submissions (Impact must be traceable directly to excellence in research)
Timescale for impact from original research is up to ca 15 years but may
vary with discipline.
There should be one case study per 10 members of staff but issues related to
small submissions may require more consideration
Criteria for assessing impact – ‘reach’ and ‘significance’ – are appropriate
and should be broadly applicable across all panels
Evidence will be assessed as presented to Panels but will be subject to
audit by REF and verified with key users of the research.
page 16
REF IMPACT PILOT STUDY (4)
Feedback from Pilot Institutions
Conveying meaning of REF non-academic impact to academic groups was
found to be non-trivial
There was an unforeseen need for a major communication exercise,
extensive discussion and bilateral exchange in developing impact case
studies .
There was a substantial amount of work and iteration involved. for many
staff across all of the institutions.
The development of illustrative case studies helped with the process
The need to obtain evidence relating impact to high quality research
retrospectively is problematic and will require the development of
appropriate information systems not currently in place.
However on the whole, case studies were endorsed by pilot institutions
as a good method of capturing research impact
page 17
PLANNING FOR REF2014
Scheduling of mock REF exercises with University Research Committee
Equality and diversity code of practice in relation to REF selection process
Alignment of research group themes with REF requirements and development
of possible UoA strategies (Panel guidance , 2011)
Modelling of REF UoA submission scenarios via RIMS database
Targeting of high quality new publications to achieve maximum academic
impact to support developing narrative in time remaining
Strategic focus in seeking high quality peer reviewed research grants
specifically to support the developing narrative
Development of case studies of research impact linked to previous
outputs /activity prior to RAE2008
Update of research website with developing research narrative
page 18
THANK YOU FOR YOUR ATTENTION
page 19