EHMA Annual Conference, Athens, Greece
25 – 27 June 2008
Katrien Verleye
INCREASING IMPORTANCE OF PERFORMANCE
MEASURES IN THE HOSPITAL SECTOR
Equity under pressure?
1
Overview
1. Context
2. Research questions
3. Methodology
4. Results
5. Discussion
2
CONTEXT
REGULATORS AND
OTHER ENTITIES
Internal
members
HOSPITAL’S
PERFORMANCE
Suppliers
Consumers
and purchasers
Kanji & Sá (2003)
3
CONTEXT
Ensuring quality as important issue in hospital sector
Focus on controlling the financial performance
Increased attention for quality improvement
cf. lapses in quality
Need to control costs without losing quality
cf. increasing costs and need in budget
4
CONTEXT
Demand for transparency and accountability
PERFORMANCE MEASUREMENT (PM)…
» … translates the hospital’s performance in quantifiable metrics
» … informs different stakeholders on the hospital’s performance
» … enables stakeholders to follow up, coordinate, control and
improve (aspects of) the hospital’s performance
5
RESEARCH QUESTIONS
PERFORMANCE MEASUREMENT MODEL
development
performance
measurement
interpretation
management
selection
1.
How is performance defined?
2.
How is performance measured within the hospital sector?
3.
How is performance information used within the hospital sector?
4.
How do hospitals evaluate their performance measurement systems?
5.
Which are the implications of PM on equity in healthcare?
6
METHODOLOGY
Evaluation of PM in hospitals in Belgium
FLEMISH CASE
- subjected to the Belgian government
Cf. http://www.belgium.be/en/health/index.jsp
- subjected to the Flemish government
Cf. http://www.zorg-en-gezondheid.be/HomeEN.aspx?id=1182
7
METHODOLOGY
Flemish Region
Flemish Communtiy
Population: 6,117,440
Total area: 13,522 km²
5 Flemish Provinces
Acute hospitals:
• 62 hospitals
• 104 campuses
Universitary hospitals:
• 4 hospitals
• 6 campuses
8
METHODOLOGY
Survey in 13 Flemish hospitals
- 2 to 3 hospitals per Flemish province
- both universitary and non-universitary
- interpretation of each PM step
9
RESULTS
*
PERFORMANCE
- multidimensional
- influenced by stakeholders
- context
10
RESULTS
*
*
SELECTION AND DEVELOPMENT OF PIs
- measuring all performance aspects:
unfeasible and irrelevant
- several performance aspects and indicators
- motivation
• statutory required !!!
• supportive networks
• self-developed indicators:
result of historical factors
added value for hospital
11
RESULTS
Statutary required financial indicators
Statutary required minimal hospital data
Clinical indicators Navigator
Patient safety indicators Navigator
Operational indicators Navigator
Patient satisfaction indicators
Employee satisfaction indicators
HR indicators
Financial indicators
Clinical indicators VZN Leuven
Clinical indicators
Patient safety indicators
Complaint services
Clinical indicators PATH-2
HR indicators PATH-2
Logistic indicators
12
RESULTS
BUT
*
*
- too many PI’s and unclear definitions
- “not reflecting the performance”
- “difficult to find reliable and valid indicators”
- no participation of employees in the selection
13
RESULTS
*
PERFORMANCE MEASUREMENT
- measurement on the departmental level
- several employees are involved
- mostly supported by ICT
14
RESULTS
*
BUT
- lack of motivation
- time consuming
- incorrect and incomplete data
- overlap
15
RESULTS
*
ANALYSIS AND INTERPRETATION
- focus on deviating measures
- supported by ICT: basic analysis
- put out the data analysis to supportive network
• Government
• Navigator
• Flemish Hospital Network Leuven
16
RESULTS
*
BUT
- feedback government and supportive networks
not satisfactory
- no integration of performance indicators
- lack of standards
17
RESULTS
*
USING PERFORMANCE INFORMATION
“what gets measured, gets done”
= ‘the most famous aphorism of performance measurement’ (Behn, 2003)
VERSUS
“to raise questions, not to provide answers”
(Likierman, 1993)
18
RESULTS
*
USING PERFORMANCE INFORMATION
Heemskerk & van
Zandwijk (2004)
(external) justification
(internal) direction
Kanji & Sá (2003)
communication to
employees
giving account
examining progress
driving improvement
supervision
resource allocation
decisions
FLEMISH HOSPITALS
communication and
reporting
giving account
strategic planning &
implementation
improving
performance
decision-making
19
RESULTS
*
BUT
- employees are not informed
- external use, but limited internal use
cf. ‘information is too general to use’
cf. ‘PM is not embedded in the decision
structure of the organization’
- PM does not achieve its goal
20
DISCUSSION
FOCUS ON USING PERFORMANCE INFORMATION
1. Comparison among hospitals
2. Having decisions made by external stakeholders
Cf. accreditation
Cf. pay-for-performance
3. Public disclosure of performance information
Cf. rankings
Improving the performance of hospitals
(Chandrima, 2005; Hamblin, 2007; Helm, Holladay & Tortella, 2007)
21
DISCUSSION
COMPARISON AMONG HOSPITALS

- incomplete and incorrect measures
- different ways of measuring indicators
- different indicators
- different definitions
- based on different strategies
- different input !
22
DISCUSSION
DECISION-MAKING BY EXTERNAL STAKEHOLDERS AND
PUBLIC DISCLOSURE
- reflecting performance?
cf. tunnel vision (Vakkuri & Melkin, 2006)
- is standardization possible?
cf. differences between hospitals
cf. do hospitals have a sector-specific strategy?
- getting a good score in stead of improving performance
cf. risk of misrepresentation (Vakkuri & Melkin, 2006)
• specializing in a few domains
• refusing patients who lower score
- risk of ossification (Vakkuri & Melkin, 2006)
23
DISCUSSION
CAUTION IS RECOMMENDED
revising previous steps before use
- reflection
- systematization
adjustment for patients’ characteristics
performance measures are “flags requiring cautious
interpretation in the light of local circumstances”
(Veillard et al., 2005; p. 492)
involvement of different stakeholders
24
Interesting references
Behn, R.D. (2003). Why measure performance? Different purposes require different
measures. Public Administration Review, 63( 5), 586-606.
Aidemark, L.-G. (2001). The meaning of balanced scorecards in the health care
organisation. Financial Accountability & Management, 17(1), 23-40.
Bauer, K. (2004). KPIs – the metrics that drive performance management. DM Review,
14(9), 63-64.
Becker, B., Formisano, A., & Roger, M.D. (2006). Strategic planning for departmental
divisions in an academic health care centre. The American Journal of Medicine, 199(4),
357-365.
Chandrima, B.C. (2005). Accreditation in hospitals. Expressed Healthcare Management.
Retrieved May 7, 2008, from
http://www.expresshealthcaremgmt.com/20050915/accreditation01.shtml.
Heemskerk, P., & Van Zandwijk, M.C. (2004). Verantwoordingsprocessen in de zorg op
basis van de balanced scorecard. Zoetermeer: het Expertise Centrum.
Helm, C., Holladay, C., & Tortorella, F.R. (2002). What’s in a name? Reporting data from
public institutions. CMAJ, 22(2), 193-194.
Kanji, G.K., & Sá, P.M. (2003). Sustaining healthcare excellence through performance
measurement. Total quality management, 14(3), 269-289.
25
Interesting references
Kollberg, B. Elg, M., Lindmark, J. (2005). Design and implementation of a performance
measurement system in Swedish health care services. Quality Management in Health Care,
14(2), 95-111.
Rooney, A.N., & van Ostenberg, P.R. (1999). Licensure, accreditation, and certification:
approaches to health services quality. Center for Human Services. Retrieved May 4, 2008,
from http://www.qaproject.org/pubs/PDFs/accredmon.pdf
Tarantino, D.P. (2003), Using the balanced scorecard as a performance management
tool. The Physician Executive, September-October, 69-72.
Ten Asbroek, A.H.A., Arah, O.A., Geelhoed, J., Custer, T., Delnoij, D.M., & Klazinga, N.S.
(2004). Developing a national performance indicator framework for the Dutch health system.
International Journal for Quality in Health Care 16(1), 165-171.
Vakkuri, J., & Meklin, P. (2006). Ambiguity in performance measurement: a theoretical
approach to organisational uses of performance measurement. Financial Accountability &
Management, 22(3), 235-250.
Veillard, J., Champagne, F., Klazinga, N., Kazandjian, V., Arah, O.A., & Guisset, A.I. (2005).
A performance assessment framework for hospitals: the WHO regional office for Europe
PATH project. International Journal for Quality in Health Care, 17(6),487-496.
Yap, C., Siu, E., Baker, G.R., Brown, A.D., & Lowi-Young, M.P. (2005). A comparison of
systemwide and hospital-specific performance measurement tools. Journal of Healthcare
Management, 50(4), 251-264.
26
Contact information
Vlerick Leuven Gent Management School
Reep 1
B-9000 Gent
Belgium
www.vlerick.com
Katrien Verleye
Scientific Staff Member
+ 32 9 210 98 30
katrien.verleye@vlerick.be
27