Writing for REF
Introduction
The Research Excellence Framework is the mechanism by which the quality of research produced by
United Kingdom universities is evaluated. Allocation of government funding reflects that results of the
REF and the process is thus extremely important for the University. The last exercise was in 2014 and the
next is likely to be in 2020 and be conducted under very similar rules. 65% of the score was based on the
quality of research “outputs”. In the context of the School of Health Sciences these are papers accepted
for publication in peer reviewed journals.
The vast number of outputs submitted (191,150) were appraised by a relative small number of reviews
(898 members of the academic panels). On average therefore members reviewed over 400 papers (all
were sent to at least two reviewers). Most panel members will have been extremely conscientious but you
can see that your paper will be one among many that the member is looking at. The onus is thus on you
to write the paper in such a way that its quality stands out to the reviewer. This document is written to
provide advice to you on how to achieve this.
Grading
Each academic is allowed to submit a maximum of four papers each one of which is assessed by the
REF panels as being in one of five categories:
Four star
Quality that is world-leading in terms of originality, significance and rigour.
Three star
Quality that is internationally excellent in terms of originality, significance
and rigour but which falls short of the highest standards of excellence.
Two star
Quality that is recognised internationally in terms of originality,
significance and rigour.
One star
Quality that is recognised nationally in terms of originality, significance
and rigour.
Unclassified
Quality that falls below the standard of nationally recognised work or
does not meet the published definition of research for the purposes of this
assessment.
Future allocation of research funds was determined in REF 2014 on the basis of three star and four star
publications only. Following this the University has placed a very high premium on research active staff
publishing at least four three or four star papers before REF 2020. There are many valid reasons for
publishing one and two star papers but such you should make publishing at least four papers of three
or four star quality your first priority.
Note that publishing appropriate one and two star papers may be an important strategy to achieve this.
Publishing a one or two star technical note to establish a methodology may give you more space to
establish the three or four star quality of a subsequent definitive paper.
Criteria
Assessment of outputs for REF 2014 was based on three criteria, originality, significance and rigour
which we assume received equal weighting.
Originality1
The extent to which the output introduces a new way of thinking about a subject, or is distinctive or
transformative compared with previous work in an academic field.




What is different about your research?
What have you done that has not been done before?
Have you used a new technique or modified an existing one?
Have you studied a cohort of individuals that have never been studied before?
Significance
The extent to which the work has exerted, or is likely to exert, an influence on an academic field or
practical applications.



What will people do differently as a result of your work?
What are its implications for other researchers and for people in the wider world?
How has our knowledge of the world changed?
Rigour
The extent to which the purpose of the work is clearly articulated, an appropriate methodology for the
research area has been adopted, and compelling evidence presented to show that the purpose has been
achieved.




1
Have you chosen a sensible research question?
Have you used the best methodology to answer that question?
Have you applied standardised techniques robustly or properly validated innovative techniques?
Have you presented your results as clearly and convincingly as possible?
These terms were not specifically defined for the REF as a whole. Strictly speaking the italicised definitions here were only used
by main panel B (Science and Engineering). They do, however, appear a reasonable basis for thinking about the terminology.
Before you start
Waiting to the writing up stage to maximise your REF rating is too late. The primary determinant of the
originality and significance of your work is the quality of the research question you ask in the first place.
No amount of high quality writing will cover up for having selected your original question poorly. When
starting any research project then think about what the resulting publications might look like them and
assess the potential outputs against the bullet points on the previous page.
Another issue is how much to include in each paper. Most modern research projects will result in several
publications. In the current environment writing a small number of high quality papers should be a higher
priority than writing a large number of lower quality papers. The most effective way to do this will almost
certainly be to try and bundle the most original, significant and rigorous elements of your project into a
small number of papers, for moderate sized project this may mean just one.
Once you have decided on what the big hitting outputs of your research will be then you can consider
what to do with subsidiary elements. Publishing them as a series of lesser papers will help bulk out your
CV. As many papers now allow the submission of electronic appendices and these may often make a
paper look more substantial it may be sensible to publish subsidiary detail in this way to support the
primary publications to getting as high a star rating as possible.
Title, Abstract, Authors and Figures
With so many papers and so few reviewers it is inevitable that any reviewer is only going to spend a
limited time on any one paper. You thus need to make sure that what you write comes across as clearly
and effectively as possible. Think of the parts of a paper that you look at if you are skim reading. These
are the parts that are most important to catch the eye of reviewers. They will start off looking at the title so
make sure that this makes as much impact as possible.
Title
Titles that reflect that principle finding of a paper can sound much stronger than those that describe the
methods. “Gross motor function is an important predictor of daily physical activity in young people
with bilateral spastic cerebral palsy” is a stronger title than “A study of gross motor function and
physical activity in children in cerebral palsy”. Be careful with obscure titles which are as likely to
confuse as to inspire.
Abstract
First impressions are always extremely important and your reviewer will almost certainly start off by
reading your abstract. It can be argued, in the present environment, that the abstract is the most
important part of the whole paper. You thus need to make sure that all the bullet points listed on the
previous page are addressed in the abstract. Spend time working on the abstract so that it portrays the
strengths of your work as explicitly as possible.
Most abstracts have a structure of background, methods, results and conclusions. Make sure that,
through these sections, the originality, significance and rigour are clearly reflected. This is the place to
include any specific claims for the paper such as the first of its kind or the largest of its kind.
There may be a conflict between what will read well to a REF reviewer and what you feel stands the best
chance of getting published. You may feel that describing your work as a “pilot” may allow you a more
lenient review at the time of submitting to a journal, for example, but prominent use of such a term is may
be detrimental when it comes around to grading for REF purposes.
Co-authors
Regardless of what we like to think about the objectivity of science, subjective factors can be extremely
important in exercises such as REF. If you’ve been collaborating with a leading figure in your field or who
works at a well-recognised centre then make sure they are included in the author list (and that the most
impressive affiliation is clearly stated). Notice that grading of two star and three star papers is partially
dependent on their international significance so if you’ve been working with international partners then
acknowledge them as co-authors as well. Of course you shouldn’t be offering vanity authorships just for
this purpose. This is thus another thing you should be thinking about before you even start the research
and not leave until the writing up stage.
Figures
If people haven’t got time to read the text then they might scan the pictures. Clear pictures that illustrate
the method used or that reinforce the most important results may be much more effective in promoting
the impact of your paper than very detailed tables.
Where to publish
During REF2014 it was made very clear that a reviewer’s opinion was supposed to be based on the
contents of the paper rather than where it was published (impact factors for journals were not supposed
to be referred to). Most reviewers, however, will recognise the leading journals in the field and will be
influenced to some extent by the journal in which an article has been published. It is therefore
recommended that you try and publish in the leading journals. Impact factor is an important indicator
within a field but should not be the only criterion for selecting a journal to submit to.
Working with others
Making the most of any piece of research is the responsibility of all the co-authors and you should be
employing everyone’s knowledge and expertise to make the most of papers. Everyone is the School of
Health Sciences should be associated with at least one research programme lead and it is a particular
responsibility of Programme Leads to give advice on paper writing and submission. I’m also available to
give general advice if required (r.j.baker@salford.ac.uk).
Students
It is important that students be allowed to write their own papers as part of their research training and
personal development (the same is of course true for post-docs and other members of staff). This needs
to be balanced, however, against the requirement to ensure that paper is written as well as possible for
REF purposes. It will not be uncommon for this to require a very heavy editorial hand from supervisors or
other co-authors. To support this process it may be useful to make a distinction between a thesis chapter,
which clearly should represent the student’s own work, and a paper for publication where more
substantial input from senior authors is appropriate (and may be essential). Using this model the student
might be expected to complete the thesis chapter with a light touch from the supervisor. They would then
write the first draft of the paper but under the expectation that this is the start of a refinement process
which should involve the whole team and may require substantial editing of what they have drafted.
It may also be worth noting that not that many student driven publications will even have the potential to
be written up as three or four star publications. It might be useful to identify any papers that may fall into
this category early on and explain to students why this approach is being adopted. They can, of course,
be left with more freedom to prepare those papers that are unlikely ever to score highly for REF
purposes.
Active reviewing
There is nothing that teaches us as much about how to write our own papers than critical evaluation of
other peoples’. Research active staff should take every opportunity to engage in peer-review activities,
primarily through acting as reviewers for journals.
The School Research Outputs Monitoring and Evaluation Panel has been constituted to involve Research
Programme Leads in the process of reviewing papers for REF purposes. Most have found this an
extremely useful process in defining for themselves what a three or four star paper looks like. Recent
changes have been adopted to allow some more junior staff to participate formally in this progress as
well.
Maximising citations
Reviewers for REF 2014 were supplied with citation data to augment their personal opinions of papers
and it is anticipated that the same will occur for REF 2020. Whilst reviewers will have differing opinions as
to how to incorporate such information into their judgement it very will be very difficult for them to award
low star rankings to highly cited papers. There are a number of steps that can be taken to enhance your
citations.



The biggest single determinant of citation count is the length of time that a paper has been
available to cite. It follows that you should publish papers that may be highly cited as early in the
REF cycle as possible.
Papers can only be cited if people know about them. Make efforts to present your papers at
conferences so that people know what you are doing. If you use social media such as blogs
and/or Twitter then advertise the work you have published. If you belong to discussion forums
and the opportunity arises then draw people’s attention to your work and that of colleagues.
Contrary to popular opinion most citation counts include self-citations. Cite yourself and cite your
colleagues. All citations in peer-review journals count so publishing one and two star papers
which cite your potential three and four star papers can have a valuable role.