Research Excellence
Framework 2014: A briefing for
Staff
Professor Ros Foskett
Deputy Vice Chancellor
History of Research Assessment





Undertaken periodically on behalf of the Funding Councils
Determines the QR (Quality-rated) funding for each HEI
Previous exercises in 1986, 1989, 1992, 1996, 2001, 2008
Set up under Thatcher during tight budgetary restrictions.
Originally called the Research Selectivity Exercise, the
Research Assessment Exercise (RAE) and now the Research
Excellence Framework (REF)
 Submission for REF 2014 will cover the period January 1st
2008 – December 31st 2013.
What is it?
 Process of expert review by academic peers of an
institution’s research
 Assessment by four Main Panels covering the spread of
academic disciplines.
 Panel A:
Medicine, Health and Biological
Sciences
 Panel B:
Physical Sciences, Mathematics,
Computing, Engineering
 Panel C:
Social Sciences
 Panel D:
Arts and Humanities
By Government


How is the REF used?
To inform research funding allocations by the four UK HE funding bodies
(approximately £2 billion per year)
Provide accountability for public funding of research and demonstrate its benefits
By Universities


To demonstrate that funding and reputation are contestable
To help frame their strategic direction
External environment
 To provide benchmarks and reputational yardsticks
 To provide comprehensible ratings of research excellence in all disciplines in HEIs
across the UK
Key changes since the 2008 RAE
 Inclusion of assessment of impact
 Fewer Units of
Assessment (UoAs)/panels, operating more consistently
 Strengthened equality and diversity measures
 Revised eligibility criteria for staff
 Addition of (limited) use of citation data in some UoAs
 Removal of ‘esteem’ as a distinct element
 Revised approach to ‘environment’ and data collection
 Increased ‘user’ input; and an integrated role for additional assessors
 Publication of overall quality profiles in 1% steps
Timetable
2011
2012
2013
2014
• Panels appointed
(Feb)
• Final panel criteria
and methods (Jan)
• Panels assess
submissions
• Guidance on
submissions (Jul)
• HEIs submit codes
of practice (Jul)
• Launch REF
submissions
system (Jan)
• Draft panel criteria
for consultation
(Jul)
• Requests for
multiple
submissions (Dec)
• Close of
consultation (Oct)
• Survey of
submission
intentions (Dec)
• Recruit additional
assessors
• Staff census date
(31 Oct)
• Submissions
deadline (29 Nov)
• Publish outcomes
(Dec)
The assessment framework
Overall quality
Outputs
Impact
Environment
Maximum of 4 outputs
per researcher
Impact template and
case studies
Environment data and
template
65%
20%
15%
Assessment
Criteria for assessing:
 quality of outputs
originality, significance and rigour
 impacts
reach and significance
 Environment
vitality and sustainability
Assessment
Outputs
Impact
Environment
4*
World-leading
Outstanding impacts
Conducive to producing
research of world-leading
quality
3*
Internationally excellent
Very considerable
impacts
Conducive to research of
internationally excellent
quality
2*
Internationally
recognised
Considerable impacts
Conducive to research of
internationally
recognised quality
1*
Nationally recognised
Recognised but modest
impacts
Conducive to research of
nationally recognised
quality
U/C
Falls below quality of
‘nationally recognised’ or
does not meet the
definition of ‘research’
Impact of little or no
reach and significance or
ineligible according to
the definition
Not conducive to
producing research of
national quality
Objectives for UW REF2014
 To demonstrate an increase in research activity in terms of breadth
and depth for reputational enhancement
 To maximise the submission as far as possible in terms of QR funding
received (which will only be for 3* and 4* research in REF)
 To submit a greater number of members staff and a higher proportion
of the academic staff than in RAE 2008
 To make a submission to an increased number of Units of
Assessment than in RAE 2008
UW REF2014
We will aim to:
 Be as inclusive as possible yet aspire to maximise reputation and
income
 Maximise the fundable research 3* and 4* outputs and impact case
studies
 Include 1* and 2* research for reputational reasons
 Exclude anything which might attract an Unclassified score
RAE 2008 Results
UoA
Staff FTE % of research activity in the submission
returned judged to meet the standard for:
4*
3*
2*
1*
U/C
Allied Health Professions and
8.2
0
5.0
40.0
45.0
10.0
Studies
English Language and
5.4
5.0
20.0
35.0
35.0
5.0
Literature
Geography and
5.0
0
10.0
45.0
35.0
10.0
Environmental Studies
History
4.5
0
0
45.0
50.0
5.0
Units of Assessment for REF2014
Main
Panel
Potential Units of Assessment
Current REF Lead
A
3. Allied Health Professions, Dentistry, Nursing and Pharmacy
Professor Dominic Upton
A
4. Psychology, Psychiatry and Neuroscience
Professor Dominic Upton
A
5. Biological Sciences
Professor John Newbury
B
11. Computer Science and Informatics
Professor Howard Cox
C
17. Geography, Environmental Studies and Archaeology
Professor Nick Evans
C
19. Business and Management Studies
Professor Howard Cox
C
25. Education
Dr Richard Woolley
C
26. Sport and Exercise Sciences, Leisure and Tourism
Professor Derek Peters
D
29. English Language and Literature
Professor Antonia Payne
D
30. History
Professor Antonia Payne
D
34. Art and Design: History, Practice and Theory
Professor Antonia Payne
D
35. Music, Drama, Dance and Performing Arts
Professor Antonia Payne
UW Preparation







Submission being led by Deputy Vice Chancellor and Director of
Research Development (Dr John-Paul Wilson)
REF Working Group established 2011 (Institutes, Personnel, ASU)
Mock exercise has been undertaken on outputs to identify potential
Units of Assessment
Code of Practice for Submission of Staff (for 31st July 2012)
Impact Case Study workshops (June/July 2012)
Equality Impact Assessments (EIAs) undertaken at each stage
Reports to Vice Chancellor’s Advisory Group, Research and
Knowledge Transfer Committee, Academic Board
NOW









All outputs should have been put on the WRAP
Collect outputs and update the record of outputs
New staff will be asked whether they intend to submit
Survey of staff circumstances
Champions for each UoA to be finalised
Preparation of Impact Case Studies
Drafting of Research Environment templates
Preparation for Survey of Submission Intentions (5th December)
REF Submission Pilot
Outputs






Submitted staff will need to have produced 4 outputs (publications and
other assessable items) since January 2008 unless they have special
individual circumstances
Judgements need to be made about the likely quality of these outputs
and whether they adhere to the definition of research in the REF
Some external advice is being sought
Co-authorship is allowed
Citation data will be used to assess ‘at the margins’ in some UoAs
Double weighting of items can be argued
What is an ‘output’?
 Product of research “process of investigation leading to new insights,
effectively shared” (see handout)
 First brought into public domain during period 01/01/08 – 31/12/13
 By a member of staff (i.e. not a research student)
 Can include:
 printed academic work;
 new materials, devices, images, artefacts, products and buildings;
 confidential or technical reports;
 intellectual property including patents and other forms;
 performances, exhibits or events;
 work published in non-print media;
 reviews, textbooks or edited works (if they embody research)
Impact





‘an effect on, change or benefit to the economy, society, culture, public
policy or services, health, the environment or quality of life, beyond
academia’
Impact template (20% weighting) sets out the general approach to
supporting impact from research
Impact case studies (80% weighting) includes examples of impacts with
evidence of impacts already achieved
Impact relates to the period of the REF (2008-2013) and the
underpinning research (of at least 2* quality) must have been
undertaken since 1993
1 Case study per 10 staff ftes +1 extra per UoA
Environment

Environment data will include:
• Research doctoral degrees awarded
• Research income
• Research income in kind
 Qualitative information on the environment in the University and the
Unit of Assessment. The narrative will include:
• Overview of research environment
• Research strategy
• People (staffing strategy, staff development, research students)
• Income, infrastructure and facilities
• Collaboration and contribution to the discipline
Code of Practice







Submitted in July 2012 (available on the Research Portal)
Underpinned by principles of fairness and transparency
REF Equality and Diversity Panel will examine all CoPs
CoP will be published alongside our submission at end of REF
CoP covers: purpose; principles; legal framework; roles & responsibilities;
selection; disclosure of individual circumstances; feedback and appeals
Clearly defined circumstances (ECR; PT working; maternity, paternity or
adoption leave; secondments outside the HEI sector)
Complex circumstances (disability; ill-health or injury; mental health
conditions; constraints related to pregnancy or maternity; childcare or
caring responsibilities; gender reassignment; Other circumstances relating
to the above)
Submission System
 Pilot submission system open now until November 2012
 Full submission system will open from 1st January 2013 until 29th
November 2013.
 JPW and RF have login permissions – inputting can be extended to
others in the institution (maybe academic or administrative staff)
 Full manual is available from the REF website: http://www.ref.ac.uk/
 Research portal will be used to store useful information (including this
presentation and the Code of Practice) under ‘Research Excellence
Framework’
http://www.worcester.ac.uk/researchportal/
Audit and Results

Audit and verification
• All processes will be audited
• All institutions will be audited against at least one element
 Results:
• An overall quality profile (in 1% steps) will be published for each UoA
against each starred level
• Institution will also get a commentary for each UoA
• Results will be published December 2014
• Publication of submissions, panel overview reports and sub-profiles in
Spring 2015
ANY QUESTIONS?