Strategies for health system
performance comparison: some
international experience
Peter C. Smith
Emeritus Professor of Health Policy
Imperial College London
Health system performance
comparison
•
•
•
•
•
Why comparison?
Some European efforts to date
What does HSPA look like?
Challenges and barriers
Conclusions
Tallinn Charter, WHO Europe, 2008
 “We, the member states, commit
ourselves to:
o Promote shared values of solidarity, equity
and participation ...
o Invest in health systems, and foster
investment across sectors that influence
health ...
o Promote transparency and be accountable ...
o Make health systems more responsive ...
o Engage stakeholders ...
o Foster cross-country learning and
cooperation ...
o Ensure that health systems are prepared and
able to respond to crises ...”
A European Union priority
• Letter from Jean-Claude Juncker, President of the
European Commission, to incoming
Commissioner for Health and Food Safety:
“… developing expertise on performance assessments of health
systems, drawing lessons from recent experience, and from EUfunded research projects to build up country-specific and crosscountry knowledge which can inform policies at national and
European level.”
Brussels, 1 November 2014
The universal role of performance
information
• … to enable actors throughout the system to
make better decisions
• … measuring and reporting performance
offers one of the most powerful instruments
for incentivizing and facilitating performance
improvement.
Three stages of HSPA
• What aspects of performance to include:
– Reporting framework
– Selection criteria
– Practical limitations
• How to report that performance:
–
–
–
–
At what level of organization
How much aggregation / detail
How to adjust for contextual differences
Media and dissemination tools
• What to do with the results:
– Who is intended to take action?
– What incentives do they have to take action?
– How is such action facilitated (eg benchmarking clubs)?
Conceptual Framework for Portuguese
HSPA Report
EuroHealth Consumer Index 2014
Good (3)
Subdiscipline weighted score
4.1 Equity of healthcare systems
F = Intermediary (2)
D = Not-so-good (1)
n.a. = data not available (1)
4.2 Cataract operations per 100 000 age 65+
4.3 Kidney transplants per million pop.
4.
Range and reach of
services provided
4.4 Dental care included in public healthcare?
4.5 Informal payments to doctors
4.6 Long term care for the elderly
4.7 % of dialysis done outside of clinic
n.ap. = not applicable (2)
4.8 Caesarean sections
Subdiscipline weighted score
M = abortion illegal/restricted
5.1 Infant 8-disease vaccination
5.2 Blood pressure
5.3 Smoking Prevention
5. Prevention
5.4 Alcohol
5.5 Physical activity
5.6 HPV vaccination
5.7 Traffic deaths
Subdiscipline weighted score
6.1 Rx subsidy
6.2 Layman-adapted pharmacopoeia?
6.3 Novel cancer drugs deployment rate
6.
Pharmaceuticals
6.4 Access to new drugs (time to subsi dy)
6.5 Arthritis drugs
6.6 Metformin use
For more info please visit:
www.healthpowerhouse.com
© Health Consumer Powerhouse 2015
6.7 Antibiotics/capita
Subdiscipline weighted score
Total score
Rank
C
D
D
D
D
n.a. F
D D
n.a. F
D
F
C
D
F
D
C
F
F
C
C
C
C
D
M
F
D
C
C
F
F
F
F
F
C
D
C
F
F
C
F
C
C
C
C
F
D
F
D
F
F
F
C
C
F
C
C
C
C
F
C
C
F
F
F
F
D
D
D
D
F
D
F
D
C
C
C
C
F
F
C
C
F
F
C
D
F
D
C
D
D
F
F
D
D
D
D
F
C
F
C
C
C
C
F
C
C
F
F
F
D
D
C
C
D
F
D
F
D
D
F
D
F
D
C
C
C
C
D
F
88
C
D
D
C
D
D
71 146 142
96 133
F C F D
D D D F
D C F D
n.a. F D D
D C F D
C C C F
C
D
D
C
F
C
F
C
C
C
C
C
F
C
50 119 138
C
D
D
C
D
C
D
D
n.a. C
D C
D F
65
83
F D
n.a. D
D D
n.a. F
n.a. D
n.a. D
D D
C D
C
F
C
C
D
D
D
F
C
C
D
F
D
F
F
F
D
D
C
69
56 100
83
60
71
F
D
D
F
n.a. D
n.a. C
D F
D
D
F
F
D
C
C
C
C
F
n.a. F
C D
D F
C
C
F
C
C
F
C
F
C
C
F
F
C
F
F
F
F
C
F
C
C
C
C
C
F
C
C
C
F
F
D
F
75 119 131 119 138 113
C
D
D
F
D
D
D
F
C
D
F
D
D
F
C
D
D
D
D
C
F
D
C
D
F
F
C
C
F
D
D
D
D
D
F
60
71
71
89
54
D F F F D F D F D D
D C C n.a. C C F C C C
n.a. C C n.a. D D F F F D
D C D n.a. n.a. n.a. C D F D
n.a. D C n.a. D D F D C D
n.a. D C n.a. F F F C F F
n.a. C D C F F D C C C
F
F
F
F
F
C
C
95
F
C
C
F
C
C
F
F
C
F
D
C
F
F
89
F
F
C
F
C
F
D
C
F
D
C
F
D
D
F
94
F
F
D
C
C
F
D
D
94
D
C
D
F
D
D
D
D
69
C
D
D
C
C
C
F
C
F
D
D
C
C
C
C
C
D
F
D
C
D
95
95
83
C C C
C F F
n.a. C D
n.a. C F
n.a. F D
D F F
F C D
D
C
D
C
D
F
F
D
C
F
F
D
C
C
C
C
D
D
C
F
C
F
F
F
88 125 100
C
D
D
D
C
C
F
F
F
C
F
C
D
F
D
D
F
F
D
D
F
F
D
F
F
D
F
C
D
F
D
88
D
F
F
D
D
D
C
F
81
D
C
C
C
F
C
C
F
F
C
D
D
C
C
F
C
D
C
F
C
F
F
D
D
F
D
C
F
83 107
89
95
71
D
C
D
D
D
F
C
D
D
D
D
83 113
C
D
D
D
D
F
C
C
D
C
F
F
C
F
F
F
F
F
F
D
D
F
C
D
96
F
F
D
D
D
F
C
D
D
F
F
D
C
D
D
F
C
C
D
D
F
D
D
F
C
D
C
C
C
C
C
F
F
D
D
F
F
D
n.a. M
C F
D
C
F
D
D
C
C
C
F
C
n.a. C
C C
F C
C
C
C
C
C
C
F
C
F
D
F
D
D
D
M
F
C
C
C
F
F
D
F
F
D C D D
C C C C
C C F D
n.a. F D n.a.
D C F D
n.a. F F D
F F D C
F
D
D
C
D
F
D
F
C
C
C
C
C
C
D
F
D
C
F
C
F
C
C
D
81 131 113
F
D
D
D
D
D
D
D C
n.a. C
D C
n.a. C
D C
n.a. C
D C
F C
C
C
C
C
C
F
C
C
56 150 144
F
C
D
F
n.a. D
n.a. C
F C
F
D
F
C
F
D
D
F
88
D
C
C
D
C
F
D
D
94
D
D
D
D
D
D
D
D
83
F
D
D
D
D
D
F
D
63
D
D
D
D
D
D
D
D
D
D
F
F
D
D
C
F
C
F
C
F
F
F
C
F
D
F
C
F
C
D
C
F
F
C
C
C
C
F
D
C
C
C
C
n.ap.
D
C
F
F
C
F
F
C
F
C
C
D
C
C
C
C
C
F
F
C
C
D
C
D
D
C
D
C
96 117 117 129 108
D
D
D
F
D
F
88 138 150 100 188 125 113 188 138 100 163 100 113 175 113 100
C
C
C
C
C
D
M
C
104 177 198 115 125 156 177 177 198 177 219 198 104 229 156 115 229 198 167 125 125 219 115 125 240 240 104 188
D F
n.a. C
D C
D C
D F
D F
D F
D F
83
D
D
F
D
D
F
n.ap.
F
C
D
F
C
C
F
D
C
C
C
D
UK England
3.8 Depression
C=
75 163 163 150 175 200 150 175 175 213 188 138 163 163
F
D
F
D
F
83 104 113 104 108
F
D
D
D
D
D
C
F
F
F
F
D
F
UK Scotland
3.7 Abortion rates
D
C
D
C
F
C
C
F
D
F
F
F
D
D
D
D
D
C
Sweden
3.6 MRSA infections
96 138
C
F
F
F
D
C
C
C
C
C
C
C
D
F
C
C
C
F
C
C
C
C
Switzerland
3.5 Preventable Years of Life Lost
58
D
C
F
D
F
F
n.ap.
C
C
F
C
C
F
F
D
D
D
D
D
Spain
3. Outcomes
F
C
C
C
F
F
C
C
C
C
C
C
F
Slovenia
3.4 Cancer survival
Pharmaceuticals :
FI, DE, IRL, NL, UK
C
F
C
C
C
C
n.a.
Serbia
3.3 Infant deaths
C
F
C
F
F
F
C
F
C
F
C
C
F
D
C
D C
C
C C
C C
D C
n.a. C
n.ap. C
n.ap. D C
D n.a. C
C D F
D D C
n.a.
Slovakia
3.2 Decrease of stroke deaths
F
C
C
F
F
F
D
F
n.a.
D
D
C
C
C
C
D
F
D
F
F
C
F
D
Romania
Prevention :
Iceland, Norway, Spain, Sweden
D
D
D
D
F
D
F
D
n.a.
D
F
F
C
F
F
C
F
D
C
C
C
F
C
D
F
F
D
Poland
3.1 Decrease of CVD deaths
200 200 225
C
F
C
C
F
C
C
F
F
F
F
C
C
F
D
D
C
D
Portugal
Subdiscipline weighted score
n.a.
C
F
F
D
C
C
C
C
C
D
C
C
C
F
D
D
C
D
Norway
2.6 A&E waiting times
83 100 142 121 133 117 138 121
C
D
D
C
D
C
C
F
F
F
F
C
C
D
F
F
F
D
Montenegro
2.5 CT scan < 7days
n.a.
C
F
D
C
D
C
F
F
F
D
F
F
D
n.ap. D
n.ap. D
C C
C D
C F
Netherlands
2.4 Cancer therapy < 21 days
n.a.
79 104
C
F
C
F
D
F
C
C
F
C
C
C
C
Malta
Range and reach of services
Netherlands, Sweden
2.3 Major elective surgery <90 days
54
n.a.
C
D
D
F
F
C
C
D
D
C
F
D
Lithuania
2. Accessibility
(waiting times for
treatment)
C
C
C
C
C
C
F
C
C
C
C
D
D
D
D
D
D
D
D
D
D
D
Luxembourg
Outcomes :
Netherlands, Norway
C
C
F
C
C
F
n.ap.
C
C
D
C
F
C
C
D
C
C
F
F
Italy
2.2 Direct access to specialist
C
C
C
F
F
C
n.a.
C
C
F
C
C
C
C
Latvia
2.1 Family doctor same day access
92 125 100
C
C
F
C
C
C
C
D
C
F
D
D
Ireland
Subdiscipline weighted score
n.a.
C
C
C
D
C
C
C
C
F
C
F
C
Iceland
1.12 e-prescriptions
Accessibility:
Belgium, Switzerland
n.a.
C
F
D
D
F
F
C
C
C
C
C
C
Greece
1.11 On-line booking of appointments?
n.a.
C
C
C
F
C
F
C
C
C
C
C
C
Hungary
1.10 EPR penetration
n.a.
n.ap.
C
F
D
C
C
C
D
D
F
F
F
D
Germany
Patient rights and information:
The Netherlands
D
D
D
D
n.a.
C
F
D
F
F
C
D
D
D
F
D
D
France
1.9 Provider catalogue with quality ranking
n.ap.
n.a.
C
F
D
C
F
F
D
D
D
C
C
C
FYR Macedonia
1.8 Cross-border care seeking freely allowed
n.a.
D
C
D
D
C
F
D
D
D
C
D
D
Finland
1.7 Web or 24/7 telephone HC info
n.a.
Estonia
1.6 Registry of bona fide doctors
n.a.
Denmark
1. Patient rights
and information
n.a.
Cyprus
1.5 Access to own medical record
C
F
D
C
C
C
F
D
D
F
F
D
Czech Republic
Sub-disciplines:
1.4 Right to second opinion
C
C
C
C
C
F
C
C
D
F
F
F
Croatia
Winner: Netherlands
Runner-up: Switzerland
Third place: Norway
C
C
D
F
C
C
D
Bulgaria
1.3 No-fault malpractice insurance
Bosnia
Herzegovina
1.2 Patient organisation involvement
Austria
Indicator
1.1 Healthcare law based on Patients' Rights
Belgium
Sub-discipline
Euro Health Consumer Index
at a glance:
Albania
For more info please visit:
www.healthpowerhouse.com
F
C
C
F
C
D
F
F
D
D
F
D
D
D
C
C
C
C
C
C
D
D
F
F
D
D
D
D
F
F
F
F
88 225 100 125
C
F
C
C
C
C
D
C
C
F
C
C
C
F
C
C
C
F
F
F
F
F
F
F
C
F
C
F
F
F
F
D
83 135 198 188 219 229 177 177
D
D
D
F
D
D
F
F
69
C
C
D
F
C
C
F
F
F
F
C
D
C
C
F
D
D
C
D
F
C
C
C
C
C
C
D
D
F
C
D
D
C
F
D
D
F
C
F
C
D
D
F
D
C
D
F
F
D
F
C
D
F
48 101
95
65
89 107
71
83
71
77
D C D D C F D F D D
C F C n.a. C C D C C C
D C n.a. n.a. F F D F D D
n.a. D n.a. n.a. F F C D D n.a.
D F n.a. n.a. F C D F D D
D F n.a. D C F F D D n.a.
C D F D C C F F C F
F
F
D
C
D
F
D
F
F
C
D
C
C
F
F
C
F
C
n.a. F
D F
C F
C
C
C
C
C
C
C
C
C
F
F
C
C
C
D
D
C
F
C
C
C
F
C
F
C
F
C
C
C
F
F
F
88 106 113 150 113 131 125
C
C
D
F
D
F
F
83
C
C
F
F
D
F
F
C
D
D
F
C
C
F
C
C
F
F
F
C
C
F
C
C
C
D
C
C
89 107 107
F
C
F
D
F
D
C
C
F
F
D
F
F
F
F
C
F
D
C
C
C
D
C
D
F
C
C
C
95
F
C
C
F
F
D
C
D
C
C
D
F
C
C
95
C
C
D
C
F
C
C
D
F
C
D
F
C
C
89
C
C
D
C
F
C
C
33
76
76
48
52
57
62
71
76
57
86
71
57
86
57
57
57
86
57
52
52
67
48
33
86
81
52
62
52
48
71
67
67
81
76
86
86
545 780 820 420 547 640 619 714 836 677 846 763 700 812 561 601 818 644 648 593 510 814 582 463 898 851 511 722 453 473 665 668 670 761 855 718 710
30
10
6
36
29
23
24
15
5
17
4
11
16
9
28
25
7
22
21
26
32
8
27
34
1
3
31
13
35
33
20
19
18
12
2
14
16
The various stated goals of HSPA
• Armenia: Enhance stewardship; Accountability; Transparency; Identify
policy priorities.
• Belgium: Transparency and accountability; Comparisons with other
countries; Performance monitoring over time.
• England: Performance management of public sector organizations.
• Estonia: Enhance accountability; Enhance stewardship; Provide a
monitoring scheme for the National Health Plan.
• Kyrgyzstan: Monitor progress and impact of health sector programmes;
Accountability to donors; Identify potential policy problem areas.
• Portugal: Accountability; Inform policy.
• Turkey: Provide a monitoring and evaluation scheme for the Health
Transformation Program; Transparency and accountability; Support the
development of evidence-based policy-making; Guide governmental
policy development; Identify policy priority areas.
World Health Organization, Case studies on health system performance assessment. A long-standing development
in Europe, 2012, Copenhagen: World Health Organization Regional Office for Europe.
Some differences in focus
•
•
•
•
Health (PT) vs health services (NL)
Between country (EE) vs within country (SE)
Aggregate outcomes (PT) vs distribution (BE)
Policymakers (planning) (GB) vs accountability
and transparency (NL)
• Trends over time (NL) or cross-sectional
international comparison (BE)
Belgian Health Care Knowledge Centre (KCE),
Performance of the Belgian health system report 2012.
Performance of the Belgian health system 2012:
continuity of care
Eight enemies of valid comparison
• ‘You cannot measure what we are trying to achieve.’ (eg mental
health)
• ‘Our objectives go beyond what you are trying to measure.’(eg
quality of life)
• ‘The data you are using are of poor quality and cannot be relied
on.’
• ‘There are external factors that influence our performance that you
have not taken account of.’ (eg low income population)
• ‘The risk adjustment methods you have used are inadequate.’
• ‘There is huge uncertainty in the reported measures.’
• ‘The data you are using are out of date.’
• ‘We are unique and cannot be compared with other institutions.’
EQ-5D: A Generic Quality of Life
Measure
Towards patient-reported outcome
measures (PROMS)
• Mandated in England since 2009
–
–
–
–
Hip replacement
Knee replacement
Hernia repair
Varicose veins
• Health status measured just before surgery and
3/6 months afterwards
• Independent data collection
• Also relevant for routine population health
monitoring
HSPA: some leadership responsibilities
• Development of a clear conceptual framework and a clear
vision of the purpose of performance measurement;
• Seizing information technology possibilities;
• Prompting and sustaining clinical involvement and leadership;
• Mandating data collection mechanisms;
• Information assurance and governance;
• Development of analytic devices and capacity to help
understand the data;
• Development of appropriate data presentational methods;
• Dissemination and securing attention;
• Stimulating action in response to performance measures;
• Proper evaluation of performance measurement instruments.
Health System Performance
Comparison
• Examines rationale for
international
comparison
• State of the art in six
performance domains
• Current practice and
resources
• Future prospects
http://www.euro.who.int/en/who-we-are/partners/observatory/studies/health-system-performance-comparison.-an-agenda-for-20
policy,-information-and-research