CHILDREN AND FAMILIES
EDUCATION AND THE ARTS
The RAND Corporation is a nonprofit institution that helps improve policy and
decisionmaking through research and analysis.
ENERGY AND ENVIRONMENT
HEALTH AND HEALTH CARE
INFRASTRUCTURE AND
TRANSPORTATION
This electronic document was made available from www.rand.org as a public
service of the RAND Corporation.
INTERNATIONAL AFFAIRS
LAW AND BUSINESS
NATIONAL SECURITY
Skip all front matter: Jump to Page 16
POPULATION AND AGING
PUBLIC SAFETY
SCIENCE AND TECHNOLOGY
TERRORISM AND
HOMELAND SECURITY
Support RAND
Browse Reports & Bookstore
Make a charitable contribution
For More Information
Visit RAND at www.rand.org
Explore RAND Europe
View document details
Limited Electronic Distribution Rights
This document and trademark(s) contained herein are protected by law as indicated in a notice
appearing later in this work. This electronic representation of RAND intellectual property is
provided for non-commercial use only. Unauthorized posting of RAND electronic documents to
a non-RAND Web site is prohibited. RAND electronic documents are protected under copyright
law. Permission is required from RAND to reproduce, or reuse in another form, any of our research
documents for commercial use. For information on reprint and linking permissions, please see
RAND Permissions.
This product is part of the RAND Corporation occasional paper series. RAND occasional papers may include an informed perspective on a timely policy issue, a discussion
of new research methodologies, essays, a paper presented at a conference, a conference
summary, or a summary of work in progress. All RAND occasional papers undergo
rigorous peer review to ensure that they meet high standards for research quality and
objectivity.
A Prize Worth Paying?
Non-standard ways to
support and reward excellence
in health research and
development in the UK NHS
Tom Ling
2008
Prepared for the Department of Health funded Centre
for Policy Research in Science and Medicine (PRISM)
EUROPE
This is an independent report commissioned and funded by the Policy Research
Programme in the Department of Health. The views expressed are not necessarily those of
the Department.
RAND Europe is an independent, not-for-profit research organisation whose mission is
to improve policy and decision making for the public good. RAND’s publications do not
necessarily reflect the opinions of its research clients and sponsors.
R® is a registered trademark.
© Copyright 2011 RAND Corporation
Permission is given to duplicate this document for personal use only, as long as it
is unaltered and complete. Copies may not be duplicated for commercial purposes.
Unauthorized posting of RAND documents to a non-RAND website is prohibited. RAND
documents are protected under copyright law. For information on reprint and linking
permissions, please visit the RAND permissions page (http://www.rand.org/publications/
permissions.html).
Published 2011 by the RAND Corporation
1776 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138
1200 South Hayes Street, Arlington, VA 22202-5050
4570 Fifth Avenue, Suite 600, Pittsburgh, PA 15213-2665
Westbrook Centre, Milton Road, Cambridge CB4 1YG, United Kingdom
RAND URL: http://www.rand.org
RAND Europe URL: http://www.rand.org/randeurope
To order RAND documents or to obtain additional information, contact
Distribution Services: Telephone: (310) 451-7002;
Fax: (310) 451-6915; Email: order@rand.org
Preface
This is a short paper scoping the issues involved in considering the merits of using prizes to
support the objectives of the Department of Health Research and Development
Directorate (DH R&D).
This document has been produced by RAND Europe’s Department of Health research
unit: PRISM (Policy Research in Science and Medicine), which works on the science of
science. PRISM aims to provide research, analysis and advice to support the effective
implementation of the Department of Health’s research strategy, and to improve research
to support decision making more widely in the health research sector.
RAND Europe is an independent not-for-profit policy research organisation that aims to
improve policy and decision making in the public interest, through research and analysis.
RAND Europe’s clients include European governments, institutions, NGOs and firms
with a need for rigorous, independent, multidisciplinary analysis. This report has been peer
reviewed in accordance with RAND’s quality assurance standards.
For more information about RAND Europe or this document, please contact Professor
Tom Ling.
RAND Europe
Westbrook Centre
Milton Road
Cambridge CB4 1YG
United Kingdom
Tel. +44 (1223) 353 329
reinfo@rand.org or tling@rand.org
iii
Contents
Preface ........................................................................................................................ iii
Summary ....................................................................................................................vii
CHAPTER 1
Introduction..................................................................................... 1
CHAPTER 2
The use of metrics and incentives in standard performance
management ..................................................................................... 3
CHAPTER 3
3.1
3.2
3.3
Beyond metrics and indicators: incentivising further
improvements in performance .......................................................... 7
Recognising organisational performance: organisation awards............................ 8
Recognising organisational performance: ‘beacon’ status for research
trusts? ................................................................................................................ 9
Moving beyond metrics: prizes for innovative research and development ........... 9
CHAPTER 4
Non-standard incentives in research and development in the
NHS ............................................................................................... 13
CHAPTER 5
Conclusions.................................................................................... 17
Reference List ............................................................................................................ 19
v
Summary
This is a short paper scoping the issues involved in considering the merits of using prizes to
support the objectives of the Department of Health Research and Development
Directorate (DH R&D). The paper concludes that there is indeed merit in developing
incentives to support excellence in health research in addition to ‘standard’ performance
management and routine inspection. These could act either to reinforce the signals created
by standard metrics (for example, awards recognising the best performers as measured by
standard metrics) or they could ‘fill the gaps’ to encourage behaviour not influenced by
conventional incentives. This would create an ecosystem to more effectively link reward
with motivation, which could deliver benefits for patients and the health care system more
widely. Prizes, it is argued, should play a more significant role in the UK health R&D
system than in the past but it is not suggested that they replace existing systems to support
high-quality research and development.
vii
CHAPTER 1
Introduction
This is a short paper looking at the issues involved in considering the merits of using prizes
to support the objectives of the Department of Health Research and Development
Directorate (DH R&D). Reviewing the use of performance measures and incentives, it
concludes that there is merit in developing non-standard incentives to support excellence
in health research in addition to ‘standard’ performance management and routine
inspection. These could act either to reinforce the signals created by standard metrics (for
example awards recognising the best performers as measured by standard metrics) or they
could ‘fill the gaps’ to encourage behaviour not influenced by conventional incentives. This
would more directly link reward with motivation, and could deliver benefits for patients
and the health care system more widely. Prizes, it is suggested, should play a more
significant role in the UK health R&D system than they have to date but it is not
suggested that they should replace existing systems to support high-quality research and
development.
The paper is divided into three parts. In Chapter 2 we consider the use of performance
measures and incentives more generally and note that, although helpful in many respects,
they are incomplete. In Chapter 3 we look at non-standard incentives and in Chapter 4 we
consider how these could apply to the case of health research and development, concluding
that ‘the prize is right’. This is a scoping paper identifying potential benefits but is not a
detailed policy appraisal and the arguments here should be seen as a stimulus for debate
rather than a developed position.
1
CHAPTER 2
The use of metrics and incentives in
standard performance management
To understand why prizes might play a more important role in the health and healthcare
R&D ecosystem, it is first necessary to consider the limitations of more standard
instruments of performance management. ‘Standard instruments’ are the metrics (typically
following the ‘balanced scorecard approach’ developed by Kaplan and Norton) that
identify important measurable targets, measure performance and use appraisal and
incentives to steer behaviour towards these. In our view, these are fundamental to the
efficient running of accountable and improving public services. However, they also have
limits relating to the potential gap between aggregated metrics and context-rich culturally
shaped motivations. These limitations leave a space for non-standard mechanisms to
flourish.
Standard instruments in health R&D reflect wider trends in results-based management.
Results-based management has become a prominent feature in the public and not-forprofit sectors in the UK. Across government in particular, departments, non-departmental
public bodies and arm’s-length bodies increasingly need to demonstrate that they
understand what their organisations are doing and what they have achieved. Furthermore,
it is argued, by creating incentives that reward the pursuit of measurable desired outcomes,
performance management can align the activities in an organisation to its strategic goals.
The theory of change implicit in this is that if goals are quantified and the contribution to
these goals measured and rewarded, then activities will be aligned.
Inevitably, results-based management has given rise to a need for more performance data.
Varying systems and approaches have developed to marshal the massive amounts of
internal performance data being generated by organisations. One of the most high-profile
of these in the UK was the report Choosing the Right FABRIC, published jointly by the
Treasury, Cabinet Office, National Audit Office, Audit Commission and Office for
National Statistics in 2001.1 The main goal of such approaches is to fashion a meaningful
performance management system out of organisations’ vast array of available and potential
data. This performance management system, it is hoped, will then render organisations
more tractable, more accountable and more able to learn.
1
HM Treasury, Cabinet Office, National Audit Office, Audit Commission and Office for National Statistics,
Choosing the Right FABRIC: A Framework for Performance Information, London: TSO, 2001.
3
A Prize Worth Paying
RAND Europe
In developing its own approach to performance data, in 2004 the Department of Health
produced ‘Standards for Better Health’, which became part of the performance assessment
by the Healthcare Commission. The Healthcare Commission consulted with UKCRC (the
UK Clinical Research Collaboration) and others involved in NHS research and, following
iterations, ‘The Better Metrics Project, Version 8’ was published by the Healthcare
Commission in November 2007.2 The current metrics cover three main areas:
x
Core standards assessment – where metrics relevant to core standards are
available from national data sources, they are included in the core standard
screening dataset for the annual health check.
x
Improvement reviews – any improvement reviews covering metrics topics have
included currently available metrics in their assessment tools (for example mental
health, heart failure and diabetes).
x
Developmental standards assessment – the Healthcare Commission extended
the annual health check to include a pilot assessment of trusts’ progress against
developmental standards in 2006/2007.
Nevertheless, the years since the publication of FABRIC in 2001 have seen growing
recognition of the possible perverse consequences of performance information (and indeed
these possible consequences are flagged up in the report).3 The Public Administration’s
Select Committee 2003 Report On Target? Government by Measurement,4 the Royal
Statistical Society5 and others have highlighted both the managerial and technical
limitations and the challenges of performance data. These anxieties reiterate concerns
expressed since the 1950s (if not earlier) and identified again in the discussion by Smith6
of the potential dysfunctional effects of the publication of performance data.7 It also
encouraged a view that micro-managing through performance indicators would, in many
contexts, encourage such perverse outcomes. Even amongst supporters of metrics, this has
strengthened a view that, for Key Performance Indicators (KPIs), more is not always better
and that, like the organisations they quantify, KPIs should be lean.8
2
P. Whitty, The Better Metrics Project, Version 8, London: NHS and the Healthcare Commission, 2007, p.8.
3
HM Treasury, Cabinet Office, National Audit Office, Audit Commission and Office for National Statistics,
2001, p.18.
4
House of Commons Public Administration Select Committee, ‘On Target? Government by Measurement’,
Fifth Report of Session 2002–03, Vol. 1, HC 62-I, London: TSO, 2003.
5
S. M. Bird, D. Cox, V. T Farwell, H. Goldstein, T. Holt and P. Smith, RSS Working Party on Performance
Monitoring in the Public Services, London: Royal Statistical Society, 2003.
6
P. Smith, ‘On the Unintended Consequences of Publishing Performance Data in the Public Sector’,
International Journal of Public Administration, Vol. 18, Nos. 2&3, 1995, pp.277–310.
7
These might be summarised as: tunnel vision; suboptimisation for the whole system; myopia; measure
fixation; misrepresentation; misinterpretation; gaming; ossification.
8
Or, as Lenin once wrote in his appropriately named ‘Better Fewer, But Better’: ‘In the matter of improving
our state apparatus, the Workers’ and Peasants’ Inspection should not, in my opinion, either strive after
quantity or hurry’; V. I. Lenin, Collected Works, 2nd English ed., Vol. 33, Moscow: Progress Publishers, 1965,
pp. 487–502.
4
RAND Europe
E rror! Reference source not found.
There are, of course, limitations to this approach. The ‘information architecture’ based on
KPIs will almost inevitably miss things that matter to researchers, managers and deliverers
of DH R&D. It is particularly vulnerable to the ‘fallacy of aggregated representation’, in
which not only do aggregations obscure variety but also quantitative representations are
confused with real experiences. The Nobel Prize-winning George Stigler argued that when
incentive systems become de-aligned with the goals of an organisation, suboptimal, or even
perverse, consequences will most likely follow. Similarly, health research and development
comprises a community of practitioners in health research and development and their
organisations, characterised by many and varied measures of success, interpretations of
quality and methods of inquiry. Under such circumstances, it is almost inevitable that
aggregated measures or performance will not feel closely aligned to the things that
incentivise high-quality research and a passion to translate findings into real improvements
for patients. This is not an argument against KPIs – which are an important part of the
DH R&D approach to performance – but it recognises that they leave gaps in the warp
and weft of motivation and reward that binds the activities of a community to its goals and
aspirations. The response of academic researchers to Research Assessment Exercise-style
measures of their performance may be partly due to the desire of any profession to avoid
external accountability, but it may also be due to a more legitimate sense of grievance with
a set of measures that do not adequately capture what they consider to be important.
There is therefore a clear possibility that performance indicators may not offer as complete
traction as has been hoped. Even worse, as Goodhart’s Law famously states: ‘any observed
statistical regularity will tend to collapse once pressure is placed upon it for control
purposes’.9 Thus, once bibliometrics become not only a means for monitoring research
activity but also a mechanism for directing funds, they may develop into the object of both
gaming and perverse behaviour. Furthermore, there is anecdotal evidence that the peer
review of publications encourages conservatism in research ambition and favours track
record over youthful enthusiasm. If this were true, the problems of perverse behaviour and
gaming might be increased.
As this brief discussion has shown, metrics might usefully support the effective steering of
R&D but for the fact that they provide an incomplete basis for a system of rewards and
incentives in a complex and varied system. The following chapter goes on to consider other
instruments that might be available.
9
C. A. E. Goodhart, ‘Monetary Relationships: A View from Threadneedle Street’, Papers on Monetary
Economics, Vol. 1, Sydney: Reserve Bank of Australia, 1975.
5
CHAPTER 3
Beyond metrics and indicators:
incentivising further improvements in
performance
In the preceding section we outlined some important strengths and weaknesses of a
metrics-based approach to the question of performance management for health research.
We saw that metrics provide us with measurements of outputs and outcomes against which
to measure performance. However, we also noted possible limitations to this.
In this chapter, we discuss ways in which it might be possible to move beyond metrics as
the only basis of performance management, towards policy instruments encouraging not
merely improved, but potentially outstanding, practice. We have identified three major
‘non-standard’ (in the sense that they are outside the practice of traditional performance
management) forms of policy instrument, each of which provides subtly differing
incentives for NHS Trusts, researchers and research-support workers to improve
performance: awards, beacon status and prizes. For clarity, we have separated these three
approaches but we recognise that in practice they could be combined.
A number of important considerations underlie the discussion of policy instruments that
follows. As with all improvement strategies, context is very important. First, we must
recognise that the time at which a non-standard instrument is deployed has implications
for its incentive effect (should a prize, for example, reward past outstanding practice or
create a goal to be pursued?). Second, we need to consider granularity – particularly for
awards and prizes, which may have quite variable incentive effects depending on their size,
cash value, duration and so forth. Third, prizes and awards, almost by definition, create
winners as well as losers and this has implications for the motivations of both (especially,
possibly, those who believe that they will always be losers). Finally – and perhaps most
importantly, given the complexity of healthcare organisations – we should acknowledge
that instruments may have variable incentive effects depending on the level at which the
rewards are focused: do we seek to reward individuals, groups or whole institutions?
Context may matter in other, subtle ways. For example, prizes are a way of confirming the
respect of a community and that presupposes that this respect is desirable and sought after.
Some cutting-edge researchers, for example, may not hold this respect from peers in such
high regard. Others might even regard such prizes with disdain. For them it may be that
the intrinsic rewards of research are sufficient and they would be immune to the lure of
recognition. In what follows, it should also be kept in mind that the systematic evidence
7
A Prize Worth Paying
RAND Europe
available around the effectiveness of these instruments unfortunately remains rather
patchy.
3.1
Recognising organisational performance: organisation awards
Organisation awards represent a natural extension of performance management based on
metrics. They amplify the influence of these existing metrics. Typically, they seek to
identify and celebrate excellent performance, defined as performance levels above a certain
threshold, and therefore they are targeted at high-flying institutions. A number of
important examples exist in the health sector in the UK, although at present these are not
tied to R&D. The healthcare intelligence provider CHKS Top Hospital Awards provides
perhaps the best example10. Here, an award is made annually to the most improved NHS
trust, and a list of the 40 best-performing trusts – of all kinds – nationwide is generated
according to a series of predefined metrics, most of which are forms of performance
measurement against which trusts are already reviewed by the Healthcare Commission.
These metrics include:
x
risk-adjusted mortality
x
emergency readmissions within 28 days
x
day case rate (trolley procedures)
x
diagnostics 13+ weeks’ wait
x
a staff survey
x
overall data quality.11
The annual Health Service Journal (HSJ) Awards recognise outstanding performance at
trust level in the NHS across a range of categories, ranging from the most outstanding
acute service trust and primary care trust, to those with the most effective chronic disease
management schemes, and IT systems. HSJ Awards are made on the basis of review of
trust performance by panels of expert judges for each of the categories12.
In both instances, the aim is to reward outstanding practice, over and above core standards.
However, in the absence of any formal evidence on the matter, it is difficult to say whether
these kinds of awards – in and of themselves – would act as effective incentives for
improved research performance. It is more likely that they would incentivise drives for
improved performance only among top-tier research institutions with a realistic chance of
winning the awards, or those near the bottom with the potential for substantial short-term
improvement. Trusts with a competent, but not outstanding, research record would be
unlikely to be strongly incentivised to improve performance by the distant prospect of
system-level recognition. They are therefore most appropriately used to encourage highperforming trusts to perform even better.
10
CHKS, homepage.
11
CHKS, Top Hospitals web page, 2007.
12
HSJ Awards web page.
8
RAND Europe
Beyond metrics and indicators: incentivising further improvements in performance
An important advantage of an awards system over other forms of incentives for improved
performance is its proximity to transparent metrics-based criteria (but this, for the reasons
listed above, could also be seen as its limitation). For the CHKS scheme in particular, the
decision to award seems to be based heavily on objective criteria – rather than the
subjective opinions of an ‘expert panel’. These data are relatively inexpensive to collect and
should be non-controversial. For the potential winners, the process requires little
additional effort (beyond attendance at the customary awards dinner or similar). An
alternative would be to try to gain some added value by using the awards participation
process itself to encourage further reflection and development by requiring them to
produce a reflective account of some sort to demonstrate why they were worthy of winning
the prize.
3.2
Recognising organisational performance: ‘beacon’ status for research
trusts?
Although in this sense the practice of awarding ‘beacon’ status to organisations has not
been systematically extended to the health sector in the UK, it is gaining increasing
currency in other areas, most notably in local government and criminal justice.13 For those
judging the award of beacon status, satisfying broad metrics requirements is the bare
minimum. The beacon approach aims to identify organisations performing outstandingly
and, in particular, innovatively, across the board, and which might provide models for
similar organisations elsewhere. Under the model practised by the Beacon Councils
Scheme, it does so in rounds, each one tied to a set of themes covering everything from
welfare provision to programmes to combat antisocial behaviour, and performance is
evaluated by specially convened expert panels. The culmination of the review is the award
of beacon status to a council for a particular programme it administers.
In fact, the award of beacon status is merely the first stage in a lengthy process of exchange
intended to raise the standards of beacons and non-beacons alike. The newly identified
Beacon Council becomes a focal point for workshops, consultations and informal meetings
with similarly focused bodies elsewhere, at which ideas are exchanged and elements of best
practice noted. The principle of beacon schemes is to recognise the complexity of the
systems in which they operate and particularly the specificity of the contexts in which
interventions occur. In the context of health research, beacon schemes could provide some
important advantages, notably:
13
x
The potential to recognise outstanding performance and innovative practice in
terms of research organisation – highlighting institutions that have proved
particularly successful at providing a positive environment for health research.
x
Formalising networks of exchange between institutions so that successful
management principles can be disseminated.
See http://www.localinnovation.idea.gov.uk/idk/core/page.do?pageId=17522455
9
A Prize Worth Paying
3.3
RAND Europe
Moving beyond metrics: prizes for innovative research and development
Thus far, we have focused on instruments that reward strong performance at organisational
level – they could conceivably be targeted at high-flying research units. Typically, they are
awarded for achieving previously identified high standards of performance. Prizes, on the
other hand, could be awarded to teams, organisations or to individuals. Like beacon
schemes, they also tend to foster innovation by individual researchers or research groups.
However, they usually involve a sense of competition. Typically prizes are for being
perceived as the best at something or for being the first to achieve some prespecified target.
The degree of incentivisation depends critically on whether prizes are awarded ex ante or ex
post. Perhaps the best-known international prizes – chiefly the Nobel Prizes – are made ex
post, usually for contributions to a particular research field over the course of an entire
career. In this sense, they contribute little to incentivising innovative research in the shortterm. However, by raising awareness about an achievement they can draw attention to a
body of work, or a certain accomplishment, which is instructive. In this way, information,
rather than incentive, may be the key to further behaviour change. For example, by
providing a prize for the best example of researchers involving patients, this may highlight
to others the potential benefits of patient engagement as well as illustrating how it might
be done. Ex ante prizes are thought to offer more incentives to innovate. Best known is the
‘grand prize’, and this is currently receiving renewed attention as a means to stimulate
research and development14 (see below). Relatively little work has been conducted to
examine their longer-term impact however. Grand prizes offer arguably the greatest
potential as mechanisms for incentivising increased innovation in research at systemic level.
Historically quite common, they have fallen from favour over much of the past 20 to 30
years, but more recently there has been a growing interest in reviving them in certain
sections of the research policy community.
There is evidence to suggest that even small-scale prizes may provide significant incentives
to produce innovative research if they are targeted appropriately; Erren15 identifies two
prizes of $1,000 offered by the Nobel Laureate physicist Richard P. Feynmann in 1959 for
highly specific and original pieces of research – both of which were subsequently
completed by research students. However, large-scale prizes present additional problems.
Erren suggests that while big cash prizes have long been used in fields such as aeronautical
research to generate rapid and radical advances, it is important to guard against
confounding effects arising from awards made only for the ‘complete solution’. Since big
cash prizes encourage greater secrecy between individuals or research groups that are
effectively in competition, it may be preferable to break up large-scale awards, or at least
recognise research that makes significant contributions to solving the recognised problem
for which the grand prize was established.
Ultimately, the real value of prizes – as with the other policy instruments we have discussed
here – probably depends on finding a correct balance between incentives of various kinds.
14
L. N. Davis, ‘Should We Consider Alternative Incentives for Basic Research? Patents vs Prizes’, paper
presented at the DRUID Summer Conference, Copenhagen, 2002.
15
T. C. Erren, ‘Prizes to Solve Problems in and Beyond Medicine, Big and Small: It Can Work’, Medical
Hypotheses, Vol. 68, 2007, pp.732–4.
10
RAND Europe
Beyond metrics and indicators: incentivising further improvements in performance
The real costs include not only the actual cash awarded but the opportunity costs, as
researchers stop pursuing some avenues of research in pursuit of prizes. We have seen that
metrics provide useful, but limited, indicators of individual and organisational
performance. In much the same way, we should be mindful of the quite different
incentives for changed performance that various supplementary policy instruments offer:
some may do little more than reward particularly strong performance against agreed
metrics; others may offer real incentives for genuinely innovative, cutting-edge research.
11
CHAPTER 4
Non-standard incentives in research and
development in the NHS
In Chapter 3 we discussed three ‘non-conventional’ approaches that could be added to the
mix of rewards designed to align the research and development system with public and
patient benefits. In this chapter we look more specifically at the role of prizes.
Recently, much of the debate around offering prizes in medical research has focused on the
alleged inadequacies of the patent system. The argument can be simply put. The patent
system delivers public benefit by promoting innovation in medical devices, treatments and
drugs and it is accepted that these innovations should be rewarded. The mechanism of
offering a monopoly for a period of years to the inventor is seen to provide such an
incentive. However, once a patent has been granted, a price can be charged above what
would be the case in a competitive market. Even though other producers can enter the
market when the patent expires, the disadvantage is that purchasers pay additional money
until the patent runs out. The problem, it is said, lies not with the granting of a patent but
with the monopoly that goes with it.
An alternative approach is to award inventors a cash prize that would incentivise the
innovation but would mean that the knowledge created should be publicly available to any
potential producers. This approach separates the reward for inventing useful products from
the product’s price. Once freed to set competitive prices, it is claimed, the resulting
competition among producers would generate public savings greater than the cost of the
prize. (However, in practice the difficulties in estimating the appropriate value for the prize
might be considerable.) This illustrates how incentives might be aligned in creative ways
with wider benefits.
The DH R&D shares in the aim of rewarding the pursuit of knowledge which, once
achieved, could be used time and again with continuous public benefit and no additional
cost (so-called non-rival consumption). The incentive structures shaping the production of
knowledge in DH R&D are, however, very different. Researchers intrinsically value their
research as an end in itself and extrinsically value their careers, status, the sense of doing
good, and income. As argued above, metrics-based reward structures are unlikely to be
sufficient to steer all these motivations. Prizes, equally, are unlikely to be sufficient but they
can play an important role in the ecosystem of rewards and, if successful, monetary benefits
would far outstrip the cost of the prizes.
13
A Prize Worth Paying
RAND Europe
NHS trusts engage with Research and Development in at least three different ways,16 and
prizes could relate to each or all of these. First, trusts may be users of research evidence;
second, they may collaborate in research led by others; and third, they may be a research
centre giving expert support to lead investigators and their collaborators. So prizes could be
won for using research, collaborating in its production or leading it. Within each of these
three areas there is considerable diversity among and within trusts (which might lead
research in one clinical area but not others). This diversity probably argues against having
an award or beacon status for a whole trust (although presumably success in one area
would reflect well on the whole trust).
The particular aspects that might be relevant are also varied. They might include:
x
being a high-quality collaborator in clinical trials
x
sponsoring underserved themes in research
x
engaging patients or public in research
x
supporting nurse-led research
x
communicating research findings to practitioners and policymakers
x
using the research of others effectively either in research or in healthcare (the ‘steal
with pride prize’, perhaps).
Prizes could be combined with elements of awards or beacon status. For example, the prize
for nurse-led research could include, ‘beacon-style’, resources to run seminars, contribute
conference papers and so forth on nurse-led research. Indeed, the cash value of the prize
would be unlikely to change the incentive structure significantly unless it was of an order
that brought it onto the radar of the chief executive (because of the low probability of
winning). However, relatively small sums could be used within the trust to support further
improvement and to build on excellence.
Non-standard incentives in healthcare perhaps face an additional advantage. Prizes confer
status and raise profile. They gain attention from peers and allow winners to feel good
about themselves and to tell friends and family of their accomplishments. Status and
collegiality are entrenched in the UK healthcare system. The engagement of professional
bodies and Royal Colleges, trusted journals and leading professionals all may enhance the
impact (although the selection of these would require care).
Given the recognised limitations of standard performance measurement and management,
there is at least a good case to be made for considering non-standard approaches. These
would, obviously, sit alongside more standard approaches but could reinforce them and
support important activities that might be lost in the aggregating processes of performance
measurement. They could also be used to balance the apparent conservatism of peer review
and to raise the profile of health R&D within trusts and in the wider policy community.
However, they would need to be carefully managed to avoid perverse incentives,
uncovenanted benefits and undesirable opportunity costs.
16
See P. Whitty, 2007, p.152.
14
RAND Europe
E rror! Reference source not found.
In designing a prize system, it is necessary to consider the amount of the prize, constraints
that might be put on the use of prize money and who should pay. It is unlikely that DH
R&D would award prizes to individuals to use as they wish. More probably, prizes would
be given to research teams who have a number of shared interests such as attending
conferences, engaging additional support, visiting other research centres and so forth.
These interests are not necessarily expensive and relatively small sums could have a
disproportionate influence on group behaviour; this is because small marginal sums could
be used to fund high-interest activities that could not be paid for out of core funding. By
constraining the use of prize money to these interests, the amount of the prize could be
minimised.
Award systems could be more or less radical. One more radical model would be to have a
fixed budget for innovation prizes, in which innovators applied for funding on the basis of
the amount of public money their innovation could be calculated to save in one year.
These claims would be verified and funding allocated to all applicants in proportion to the
sums of public money potentially saved. This approach is a variant of the US Medical
Innovation Prize Fund as proposed (but not agreed) in a 2005 legislative proposal to
Congress.
‘Grand prizes’ have historically been used for high-profile, scientifically or technologically
significant discoveries and at first sight they may be less relevant for DH R&D. However,
there are apparently intractable problems that might emerge as candidates; for example, the
successful application of the best research evidence to successfully eradicate C. difficile
(after taking into account apparently random fluctuations) or the development of new
procedures or techniques. The feature of ‘grand prizes’ is that they must be of sufficient
value (monetary or otherwise) to change behaviour. However, if they do not change
behaviour then they would be an uncovenanted benefit (a reward for doing what would
have been done without the prize), but if they do change behaviour then the problems of
perverse incentives and opportunity cost may appear. A further weakness is the technical
difficulty in identifying the threshold at which point the discovery is thought to be
complete. Another risk is that it might discourage collaboration as researchers conceal
emerging findings from rivals.
A more likely model could focus on the needs of high-priority groups and award prizes
based on the number of quality-adjusted life years (QALYs) (or equivalent) claimable by
the invention. This could either be awarded to one winner or shared, as in the above
example. It might include, for example, a focus on unfashionable groups that are currently
underserved by the R&D system, or on groups that are most costly to treat currently.
A variant might be similar but focused on one high-priority aspect of the patient pathway
in the treatment of a particular disease. For example, it could be decided that low-cost,
accurate diagnostics in sexually transmitted diseases was particularly important. Once
again, this could be used to attract research towards areas currently underserved. Where
this could, for example, exploit existing treatments and devices more effectively by
removing a significant bottleneck, the benefits could be considerable.
A final model would be a more conventional approach, with a panel receiving bids from
applicants focused on issues that matter, such as high-impact collaborations with
demonstrable benefits for patients, patient-led research, etc. More conservative in design,
15
A Prize Worth Paying
RAND Europe
this could nevertheless provide an opportunity to identify and spread good and promising
practice.
Each of these models also presupposes that some prestige is attached to winning. By
linking the prize to the prestige of sponsors, this can not only provide a source for financial
support but, more importantly, for intangible incentives. Clearly, selecting such sponsors
would be important. The allure of the prize can be added to with high-profile speakers and
events at the prizegiving ceremony.
16
CHAPTER 5
Conclusions
In conclusion, therefore, there is at least a case for considering a more extensive use of
prizes in the ecosystem of rewards and supports that aligns the DH R&D system to better
health outcomes and an improved healthcare system. These should be used either where
there is a perceived need to amplify existing (weak) standard incentives or to ‘fill the gaps’
by appealing to researchers and researchers’ managers in ways that performance metrics
don’t touch. Designing the prize system requires considerable care if it is to incentivise the
desired behaviour and not disincentivise non-winners. Attention should also be paid as to
whether it is individuals, teams or organisations that are being incentivised. The
granularity of timing and prize size also matter, as does the relationship between prizes and
prestige, and professional pride. Consequently, if it were to be a sustainable and long-term
contribution to the eco-system of health research and development, it should be
introduced with care, following consultation and further analysis.
17
Reference List
Bird, S. M., D. Cox, V. T. Farwell, H. Goldstein, T. Holt, and P. Smith, RSS Working
Party on Performance Monitoring in the Public Services, London: Royal Statistical
Society, 2003.
CHKS. As of 10 May 2011:
http://www.chks.co.uk/index.php?home
http://www.chks.co.uk/index.php?tophospitals2007
Davis, L. N. (2002), ‘Should We Consider Alternative Incentives for Basic Research?
Patents vs Prizes’, paper presented at the DRUID Summer Conference, Copenhagen.
Erren, T. C., ‘Prizes to Solve Problems In and Beyond Medicine, Big and Small: It Can
Work’, Medical Hypotheses, Vol. 68, 2007, pp.732–4.
Goodhart, C. A. E., ‘Monetary Relationships: A View from Threadneedle Street’, Papers
on Monetary Economics, Vol. 1, Sydney: Reserve Bank of Australia, 1975.
HM Treasury, Cabinet Office, National Audit Office, Audit Commission and Office for
National Statistics, ‘Choosing the Right FABRIC: A Framework for Performance
Information’, London: TSO, 2001.
House of Commons Public Administration Select Committee, ‘On Target? Government
By Measurement’, Fifth Report of Session 2002–03, Vol. I, HC 62-I, London: TSO, 2003.
Health Service Journal (HSJ) Awards. As of 10 May 2011:
http://www.hsjawards.co.uk/
Lenin, V. I. ‘Better, But Fewer’, Collected Works, 2nd English ed., Vol. 33, Moscow:
Progress Publishers, 1965, pp.487–502.
Smith, P., ‘On the Unintended Consequences of Publishing Performance Data in the
Public Sector’, International Journal of Public Administration, Vol. 18, Nos. 2&3, 1995,
pp.277–310.
Whitty, P., ‘The Better Metrics Project’, Version 8, NHS and the Healthcare
Commission, 2007. As of 10 May 2011:
http://www.cqc.org.uk/_db/_documents/Better_Metrics_full_report.pdf
Beacon status. As of 10 May 2011:
http://www.localinnovation.idea.gov.uk/idk/core/page.do?pageId=17522455
19