Canadian Institute for Health Information
Measuring Health System
Efficiency in Canada:
Introducing CIHI’s Program of Work
Web Conference Presentation
Sara Allin, Senior Researcher, CIHI
February 27, 2014
2
Why Is CIHI Measuring Health
System Efficiency?
• There is widespread recognition that the health
system needs to make better use of existing
resources and improve value for money
• Health system managers currently face tight
budget constraints
• Information about variations in efficiency could support
provincial health system performance improvement
• In this work, the health system includes all activities
under the jurisdiction of provincial ministries of health
3
Program of Work on Health System Efficiency
in Canada: Phase 1
• Developed a conceptual model for measuring health
system efficiency
• Summarized results of qualitative research, including a
stakeholder dialogue and interviews with senior health
system decision-makers
• Results showcased in CIHI’s technical report
Developing a Model for Measuring the Efficiency of
the Health System in Canada (released July 2012)
4
Phase 1: Qualitative Research to Develop a
Model for Measuring Health System Efficiency
Policy Scan
Elite Interviews
Stakeholder Dialogue
Goal
Identify the stated
objectives of the
health system
Identify provincial health
policy-makers’ views
on the inputs to and
outcomes of the
health system
Engage stakeholders in
discussion on health
system objectives,
boundaries and methods
Selection
Criteria
Publicly available
documents produced by
federal, provincial and
territorial governments
that address health
systems and policies
Current or former
senior health ministry
officials of provincial/
territorial governments
Current or former senior
decision-makers, health
system consultants and
senior executives from
health care organizations
17 interviewees from
9 provinces and
2 territories
16 participants from
6 provinces,1 territory and
the federal government
Sample Size
Note:
There was no overlap among interview and dialogue participants.
5
Program of Work on Health System Efficiency
in Canada: Phase 2
• Applied the conceptual model to spending and
health outcome data at the regional level
• Objective: To understand what the health system
is meant to achieve, i.e., the objective against which
we should be measuring efficiency
• Results showcased in the analytical report
Measuring the Level and Determinants of Health
System Efficiency in Canada (released April 2014)
6
Summary of the Proposed Conceptual Model
to Measure Efficiency (Developed in Phase 1)
Public spending on
•
•
•
•
•
Access to timely and
high-quality health care
Hospitals
Other institutions
Physicians
Community care
Prescription drugs
Inputs
• Potential years of
life lost (PYLL) from
treatable causes
of death
Health
region
Outputs
Environmental adjustors
Factors to explain inefficiency
• Environmental factors (e.g., socio-economic, demographic characteristics
of the regional population)
• Health system factors (e.g., clinical and operational factors)
7
Phase 2: Research Questions
• What is the average level of efficiency in Canada’s
regional health systems?
• What factors explain variations in efficiency across
the health regions?
• What are the key data gaps that CIHI could address
to improve future empirical analyses of health
system efficiency?
8
Next on the Agenda
• Michel Grignon, McMaster University, will present
the methods, data and results of CIHI’s analysis of
health system efficiency
• Martha Burd, B.C. Ministry of Health, will reflect on
the findings of the report from the perspective of a
decision-making organization
9
Measuring the Level and Determinants
of Health System Efficiency in Canada:
Summary of Methods and Findings
Web Conference Presentation
Michel Grignon, Director, Centre for Health Economics and Policy
Analysis, and Associate Professor, Department of Economics and
Department of Health, Aging and Society, McMaster University
February 27, 2014
10
Methods: Step 1—Data Envelopment Analysis
to Calculate Efficiency
• Calculate point estimates of efficiency using data
envelopment analysis (DEA), a descriptive approach
to measuring efficiency based on linear programming
• Apply a statistical outlier detection methodology
(Wilson 1993)
• Bootstrap point estimates to generate robust
efficiency estimates (Simar and Wilson 1998)
11
Methods: Step 2—Regression Analysis to
Explain Variations in Efficiency
• Factors affecting efficiency could fall into
3 broad categories
1. Clinical factors: Inappropriate or ineffective care
provided, and prevention opportunities that are missed
2. Operational factors: Overly expensive inputs are used
3. Characteristics of the environment
• Step-wise regression to identify the variables
significantly associated with efficiency estimates
12
Data: Summary of Input and Output Data in
Sample of 84 Regions Across 10 Provinces
Inputs
Source (Year)
Mean
Range
Hospitals, $ per capita
Canadian MIS Database (2007 to 2009)
1,719
951
3,826
Prescription drugs, $ per capita
IMS Brogan (2010)
546
289
884
Physicians, $ per capita
National Physician Database (2007 to 2009)
471
177
817
Residential care facilities (RCFs),
$ per capita
RCF Survey, Statistics Canada (2008)
336
74
902
Community nurses, $ per capita
Census, Statistics Canada (2006)
54
20
99
Education (% with high school
certificate or more)
Canadian Community Health Survey (CCHS),
Statistics Canada (2007–2008)
82
63
94
Recent immigrants (%)
CCHS, Statistics Canada (2007–2008)
3
0
17
Non-Aboriginal (%)
Census, Statistics Canada (2006)
93
50
99
1,666
1,067
2,453
Output
PYLL from treatable causes,
before age 80, age standardized
Vital statistics, Statistics Canada
(2007 to 2009)
13
Results: Robust Estimates of Efficiency
and Sensitivity Analyses
• Efficiency point estimates from DEA averaged
between 0.65 and 0.82 across 7 separate
model specifications
• This means that treatable PYLL could be reduced
by 18% to 35% if all regions operated efficiently
• Efficiency estimates were not sensitive to the age
cut-off for defining premature death (75, 80 or 85),
or the choice of PYLL versus the standardized
mortality rate from treatable causes
14
Results: Contribution of Each Category of
Factors Affecting Efficiency
Category
Variables With Statistically Significant
Associations With Efficiency (p<0.05)
Environmental and
Population Characteristics
• Average income of the population
• Inequity in the likelihood of visiting a physician
7%–14%
Clinical Factors
•
•
•
•
14%–26%
Operational Factors
• GPs (% of physicians)
• Alternate level of care length of stay (days)
Daily smoking (%)
Physical inactivity (%)
Multiple (3 or more) chronic conditions (%)
30-day overall readmission to hospital (rate per 100)
R2
12%–22%
These variables together explain nearly 50% of total variation,
leaving half of variation unexplained
15
Main Data Gaps
• More precise measures of patient flow
– Patient-level data for physicians, prescription drugs,
RCFs, nursing was unavailable
• Community care and public health spending data
– CMDB has this, but limited comparability across provinces
• Indicators of clinical and operational factors that may
affect efficiency
– Integration and coordination of care, and expanding
scopes of practice (e.g., for pharmacists and nurses)
16
Summary of Key Findings
• Years of life lost from treatable causes of death could be
reduced by up to 35% if systems were managed more
effectively and if their populations had lower health risks
and better health
• Clinical factors, namely the indicators of successful
prevention efforts such as the prevalence of smoking and
physical activity, were significant drivers of efficiency after
controlling for several key environmental characteristics
• Operational factors, such as investments in primary care
and the appropriate use of hospitals, were also important
• The unexplained variation in efficiency scores could be
driven in part by clinical practice variations and in part by
other unmeasured patient and population characteristics
17
Proposed Future Research
• Undertake case studies of a sample of
high-performing regions
– E.g., what are some of the decisions that health system
leaders have made that have led to good performance
in the indicators that are associated with health
system efficiency?
• Collect additional data on clinical and organizational
factors that could affect efficiency
18
Measuring Health System
Efficiency in Canada:
Reflections From B.C.
Web Conference Presentation
Martha Burd, B.C. Ministry of Health
February 27, 2014
19
Measuring Efficiency . . . Many Choices
Environment, society
Health system
Inputs
Health
Health
outcomes
Project choices
Outputs
20
High-Level Take-Away Learnings

CIHI project findings: Significant factors as predictors of efficiency







Average income
Physical activity
Daily smoking
Multiple chronic conditions
Relationship to a GP
Hospital readmission rates
Length of stay in alternate
level of care
Thoughts
How can the health
care system influence
these factors?
How does this fit with
B.C.’s strategic directions?
21
Physical Activity
How can the health care system
influence these factors?
We can promote . . .
But some things are
beyond our influence!
22
Smoking Rates
How can the health care system
influence these factors?
Government actions to
reduce smoking
Reduction of smoking rates has been
part of our strategic plan since 2000
 Increase price of tobacco
products through higher taxes
 Restrict sales of tobacco products
 Implement smoking bans
 2000: Vancouver was the first municipality in Canada to ban
smoking in all public places
 2002: B.C. banned smoking in public places, with allowance
for smoking rooms
 2008: B.C. banned smoking in public places, no exceptions
 Offer smoking cessation programs
23
Chronic Conditions


Identifying people with chronic conditions

Data sets may collect only single diagnosis per encounter

Chronic condition recorded when initially diagnosed

If stable, may not appear on later encounters

Analysis based on diagnoses recorded that visit or that year
may miss chronic conditions
B.C. has created virtual chronic condition registries
 Specific algorithms identify earliest diagnosis of specific chronic conditions
 Algorithm uses diagnoses from physician/hospital/prescription drugs databases
 If criteria are met, add person to registry
 Registries used to estimate prevalence rates, analyse services used by people
with chronic conditions

B.C.’s actions
 Developed best practice care pathways, integrated care programs
 Instituted physician incentive programs for best care, with measured compliance
 Promoted attachment to physician: GP for Me program
24
Multiple Chronic Conditions

2.01 million British Columbians (44%) have 1 or more of these 18 chronic conditions
(Chronic Disease Management Registries). Analysis of comorbidities is a challenge!
To understand multiple comorbidities, we use a population segmentation approach:

Divided people with these conditions into High, Medium and Low Complexity groups,
based on complexity of individual conditions or selected comorbidities
Chronic Conditions Population Segment
CC Sub-Group
Conditions
Dementia
Cystic Fibrosis
High Complex
Dialysis
Chronic Conditions
Transplant
CHF
Stroke
Acute Myocardial Infarction (AMI)
Pre-dialysis Chronic Kidney Disease (CKD_pre Dialysis)
Medium Complex Chronic Chronic Obstructive Pulmonary Disease (COPD)
Conditons
Intervention Cardiac Procedure (CABG, PCI)
Angina
Rheumatoid Arthritis (RA)
Osteoporosis (Ostpr)
Diabetes (DM)
Low Complex
Hypertension (HTN)
Chronic Conditions
Osteoarthritis (OA)
Depression (Depr)
Asthma
Combina tions
(Angina & COPD)
(AMI & CKD_pre Dialysis)
(RA & Ostpr)
(DM, HTN & OA)
(OSTPR & OA)
(OSTPR & HTN)
(DM & Depr)
(OA & HTN )
25
Understanding the Health of the Population and Their Use of Services
Evolution of Analysis
Use of Selected
Services, 2011/12
Single chronic condition
(Size of dot = per capita $)
Multiple chronic conditions
Use of services across
the health system
How can we improve quality of care,
access, efficiency, sustainability?
Are these the right services
in the right place at the right time?
Result: Strategic planning
and initiatives





Prevention
Early treatment
Better coordination of care
Integrated community care
Alternatives to hospital care
Percentage of Services
Overall health conditions
Percentage of B.C. Population
26
About the Canadian Institute for Health
Information (CIHI)
•
CIHI established in 1994 as independent, not-for-profit corporation
•
CIHI’s vision: Better data. Better decisions. Healthier Canadians.
•
CIHI’s mandate: To lead the development and maintenance of
comprehensive and integrated health information that enables sound
policy and effective health system management that improve health
and health care
•
CIHI’s data holdings: 27 databases of health information
•
Range of stakeholders in health system and beyond
– Government organizations (such as Health Canada and Statistics Canada),
ministries of health, regional health authorities, non-government
organizations, private-sector organizations, professional associations,
health facilities
27
About the Canadian Population Health
Initiative (CPHI)
• CPHI is a branch within CIHI
• CPHI’s mission: To support policy-makers and health
system managers in Canada in their efforts to improve
population health and reduce health inequalities
through research and analysis, evidence synthesis
and performance measurement
28
For More Information
• CIHI: www.cihi.ca
• CPHI: cphi@cihi.ca
29
Thank you!
30