NHS Confederation / LSE Health seminar, 27 April 2010
Performance management
in the NHS:
going beyond the metrics
Dr. Mark Exworthy
Royal Holloway-University of London
M.Exworthy@rhul.ac.uk
Performance: opening comments
• Performance has been a dominant narrative
within English health policy reform in the last
decade
• Focus is shifting over time*
• Performance management shortcomings*
• Research tends to…
– Focus on technical `performance products’
– Lack conceptual development
• Wider perspective on performance required…
– To go beyond the metrics
Performance management:
a shifting focus over time
Feature
Traditionally
Unit of analysis Organisational level
Increasingly
Individual level
Specificity
Anonymous
Named
Motivation
Intrinsic
Extrinsic
Focus
Inputs & outputs
Outcomes
Purpose
Developmental /
formative
Peer reviewed
Judgemental /
summative
External
Reference
group
Exworthy et al, 2010
Performance: opening comments
• Performance has been a dominant narrative
within English health policy reform in the last
decade
• Focus is shifting over time*
• Performance management shortcomings*
• Research tends to…
– Focus on `performance products’
– Lack conceptual foundations
• Wider perspective on performance required…
Performance management shortcomings
Adapted from Sheaff et al, 2004; Talbot, 2005
1. Incompleteness
2. Over-complexity
3. High transaction costs
4. Attribution difficulties
5. Quantity-quality imbalance
6. Gaming
7. Short-term focus
8. Performance churn
Performance: opening comments
• Performance has been a dominant narrative
within English health policy reform in the last
decade
• Focus is shifting over time*
• Performance management shortcomings*
• Research tends to…
– Focus on `performance products’
– Lack conceptual foundations
• Wider perspective on performance required…
Wider perspectives on performance
•
The influence of subjective views of
performance upon management
•
Conceptual perspectives on performance
•
Consequence (intended and otherwise)
of performance management
1. Formal & informal performance
 Formal performance
–
–
–
–
–
Hard information ~ official metrics
Quantitative measures; retrospective
Tends to focus on `poor’ performance rather than improving good
performance
Example: rankings, league tables, targets
`Safety net’ function
 Informal performance
–
–
–
–
Soft information ~ perceptions, founded on subjective
judgements
Qualitative measures, can be prospective; reputation, trust,
goodwill, tacit knowledge, credibility
Eg. `safe pair of hands’, `keep an eye on them’, `what is really
happening?’
Substitution & complementary functions
Formal performance
• Safety net: formal notions
of performance invariably
used as a `safety net’ to
address organizations
with poor (formal)
performance
– Incentive? Formal
performance may offer
`high performing’
organisations little incentive
to improve
– High performance did not
ensure freedom from
centre
Informal performance
• Substitute: informal
performance is often
deemed more
responsive, timely and
useful than formal
performance
• Complement: both formal
and informal performance
were seen as important in
assessing organisations
Adapted from Goddard, Mannion & Smith, 1999; Exworthy et al, 2010a
• Formal performance is insufficient to explain existing performance
patterns and to promote improvement
• Need to understanding better the interplay between formal and
informal performance
Example #1.
FTs exercising their autonomy
• FTs have not `performed’ as well as expected
– Is autonomy such a panacea after all?
• FTs have technical ability to exercise autonomy
• Many FTs lacked the willingness to do so:
–
–
–
–
–
Some de facto autonomy already exists
Greater risk
Uncertain rules of the new game
Legitimacy
Fear of negative impact on local health economy
• Need to explore FT managers’ motivations,
attitudes towards the award and use of autonomy
Exworthy et al, 2010a
Example #2.
Mid-Staffordshire NHS Foundation Trust
• An FT and a high performing organisation?
– “In the four years from 2002 until 2005 (the last year of the star
ratings system which ranked trusts from 0- to 3-star), Stafford had
got, respectively, 2, 3, 0 and 1 star. Yet it was encouraged or
"invited" to seek FT status ” (Paton, 2010
http://www.publicservice.co.uk/feature_story.asp?id=13870)
• Patients and staff knew about `poor’ performance:
– “I remember at the time when our staffing levels were cut and we
were just literally running around. Our ward was known as Beirut
from several other wards. I heard it nicknamed that. ITU used to
call us Beirut… I remember saying: this will have repercussions,
this can’t go on like this. Because relatives were regularly coming
up to us and saying: my Mum has been buzzing for this long, there
has been a buzzer going there for that long.” (p.197)
Francis Inquiry report, 24 February 2010
http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/@ps/docu
ments/digitalasset/dh_113447.pdf
Example #3. Performance management
in local health economies
Director, PCT:
“We’ve got a wealth of informal knowledge about [Hospital X] because there are a
number of people in the team who’ve been there before and the nature of
[Hospital X] is it’s been very open to it because they’ve been so desperate for
help.
[Hospital Y] is much more of a closed book to us. There are people that have come
into different roles in the PCT who may have known something about it but it
always feels quite antagonistic that relationship and the GPs are pretty
disenfranchised lot as well. So there’s not a lot [of PCT staff] that has a kind of
root of understanding and influencing this... isn’t there either.
[Hospital Z] was a fairly sort of collegiate comfortable sort of relationship and
there’s a fair amount of sort of, traffic, you know, informal networking stuff
through that, for a lot of people in the PCT or- there are in the Exec Team of the
legacy PCTs”
http://www.sdo.nihr.ac.uk/projdetails.php?ref=08-1618-125 p.153
Exworthy et al, 2010
2. Disclosure of clinical performance
• “The more closely we are watched, the
better we behave?’ (Jeremy Bentham)
• “Sunlight is the best of disinfectant; electric
light the most efficient policeman”
(Brandeis)
Disclosure of clinical performance:
unintended consequences
1. Quantification emphasis
2. Short-term objectives dominate
3. Manipulation of performance data and
behaviour adjustment
4. “Misleading inferences” could be drawn from
“raw performance data.”
5. Organisational inertia
Smith, 1995
Public disclosure of performance data
Problem
identification
Naming of
individual
Pawson et al, 2005, S1:23
Public
sanction
Recipient
response
Public disclosure of performance data
(with unintended consequences)
Problem
identification
Culprit
misidentification
Naming of
individual
Public
sanction
Recipient
response
Dissemination
dissimulation
Sanction
misapplication
Unintended
outcome
Pawson et al, 2005, S1:23
Public disclosure of performance data
i. Identification
• Initial focus on mortality rates of cardiac surgery
• Disclosure supported by profession
• 2-3% mortality rate neglects most patients
• Attribution issue with 30 day mortality rate
• Limited use of comparisons
ii. Naming
• Average age of cardiac surgery patient = 68 years (and
rising)
• Data accessibility and user literacy
• Named consultant `hides’ clinical team
Public disclosure of performance data
iii. Public sanction
• Strong normative pressure for clinicians to participate but not
compulsory
• 25% cardiac surgeons do not participate
• Little evidence of `choice’ as sanction
• Sanction mediated by user proxy (GP)
• Episodic nature of care & emotional concerns
iv. Recipient response
• Initially educational but increasingly judgemental
• Professional ownership and promotion
• Surgeons’ sense of autonomy threatened
• Danger of gaming but little evidence so far
• Risk aversion: high risk patients avoided?
• Some rejection of performance measures
• Junior surgeons less exposed to high risk cases
Disclosure and performance management
• Patients’ experiences:
– PROMs
– Financial incentive: “Hospital income will increasingly
be linked to patient satisfaction, rising to 10 per cent
of their payments over time” (DH, NHS 2010-2015: From good
to great. 10 December 2009)
• Danger that performance measures becomes
too esoteric
– Management remains loosely coupled with clinical
performance
– Patients unable to interpret performance data
A final thought on performance
“There is a need to recognise the
imperfections and limitations of
[performance] measures, and to use
them as a means of supporting
politically informed judgements”
Stewart and Walsh, 1994, p.45