The National Program for Quality
Indicators In Community
Healthcare: Methodological Issues
Orly Manor
7th meeting of the Eastern Mediterranean Region of the
International Biometric Society (EMR-IBS)
Tel-Aviv April 22-25, 2013
2
“Efforts to improve quality
require
efforts to measure it”
(Casalino, 2000:NEJM)
Healthcare in Israel
3
National Health Insurance law (1995)

Universal healthcare

Standard basket of medical services

Four health plans (kupot cholim)

“Justice, equity and solidarity…medical services will be
offered based on medical considerations, with
reasonable quality.”
Healthcare in Israel
4

Health Tax - progressive , paid to the National Insurance Institute
(funds are distributed to the health plans according to a capitation
formulae) + modest copayment directly to health plan for specific
services

Open enrollment, no option to reject applicant, annual option to
switch plans

Managed competition between plans (uniform benefits)

Competition is based on quality and nature of services
Israel National Program for Quality Indicators
in Community Healthcare (QICH)
5

Supervisory bodies were established “to follow and
assess the influence of the NHI law on health services in
Israel, their quality, effectiveness and cost"

QICH started as a research project (Porath & Rabinovitz,
2002) and later adopted as a national program.

Full cooperation and support by all four health plans
Mission
6
To provide the public and policy-makers information
on the quality of community healthcare provided in
Israel. This information covers various health
categories and is intended to promote and improve
the standard of healthcare in Israel.
Main Product
7
Annual report presenting national results
of quality indicators in community healthcare
Enables

the evaluation of developments and changes in healthcare over time

the early identification of risk factors in the Israeli population and in
sub-populations

the comparison of healthcare quality in Israel with other countries
8
"Not everything that counts
can be counted,
and not everything that can
be counted counts."
Einstein
Quality Indicators- Methods
9

Measures of clinical performance (structure, process, outcome)

Based on electronic health records from the four health
plans

All indicators are rates

Some indicators are conditional on others

Covariates: age, sex, SEP (proxy)
Quality indicators (2013)
10
Health
promotion
BMI
Smoking
cessation
Cancer
Children and
adolescents
Breast
cancer
screening
Anemia
screening
(infants)
Colon
cancer
screening
Weight and
height
documentation
Elderly
Influenza
vaccination
(seasonal)
Pneumoc.
vaccination
Asthma
Secondary
prevention
Appropriate
use of
control
medication
Influenza
vaccination
(seasonal)
Cardiovascular
health
Diabetes
Primary
prevention
Secondary
prevention
Cholesterol
assessment
Blood
pressure
documentation
Cholesterol
documentation
Blood pressure
documentation
BMI
Blood glucose
documentation
Eye exam
Kidney care
Influenza
vaccination
(seasonal)
Pneumococcal
vaccination
Secondary
prevention
LDL modifier
use
Effectiveness
of care
Cholesterol
control
Effectiveness
of care
Cholesterol
control
Blood glucose
control
Blood pressure
control
Example: Diabetes
11

Blood glucose levels of individuals with diabetes are
directly related to the development of complications:
cardiovascular disease, blindness, kidney failure

Monitoring blood glucose by periodic hemoglobin A1c
testing and achieving adequate glycemic control
Example: Diabetes
12
Prevalence measure:

Rate of individuals with diabetes mellitus from the entire population (overall
and by age and gender)
Process measure:

Rate of individuals with diabetes with documented levels of hemoglobin
A1c (HbA1c)
Outcome (intermediate) measure:

Rate of individuals with controlled levels of HbA1c from patients with
diabetes with documented levels of hemoglobin A1c (HbA1c)
Methodological Issues
13
1.
Criteria
2.
Population coverage
3.
Data quality-measurement
error
4.
Data sources
5.
Consistency of measures
6.
Reporting
Criteria
14
Importance
&
relevance
Evidence
Ability to
quantify
Availability
&
accessibility
of electronic
data
Population Coverage
15

Population-based, near-complete coverage (not
sample)

Transfers between health plans

Births/deaths

Other populations: e.g., soldiers
Data Quality- misclassification error
16

Estimating the prevalence of a medical condition in the absence of neither
a gold standard nor an additional classification.

We wish to estimate - prevalence, sensitivity and specificity , yet df=1.

We can use a Bayesian approach- simultaneous inferences of the
prevalence, sensitivity and specificity and positive and negative predictive
value (Joseph et al 1995)

Selecting priors- experts’ opinion, understanding sources of data and
errors (Greenland 2009)
Data Quality- Consistency of
Measures
17
Uniform definitions across health plans

Membership

Data collection period

Numerator and denominator
Data Sources
18
1. Medical records (e.g., documentation of BMI)
2. Nurse’s records (e.g., documentation of vaccination)
3. Pharmacy claim records (e.g., medication purchase)
4. Laboratory results (e.g., HbA1c levels)
5. Hospital procedure codes (e.g., CABG)
6. Other (e.g., mammography)
19
Data Quality – Sources of Data and
Sources of Error
1. Medical records (e.g., documentation of BMI)
Automated
manual data of
input
2. Nurse’s records
(e.g., vs
documentation
vaccination)
(variation between and within health plans)
Pop-up
options vs
typing
in vaccine
name,
3. Pharmacy
claim records
(e.g.,
medication
purchase)
historical data
ATC vsbetween
YARPA/LARGO
(variation
health plans)
4. Laboratory results
(e.g., HbA1c levels)
(variation between health plans)
Standardized
for calibration
5. Hospital procedure
codes values
(e.g., CABG)
(variation between laboratories)
MOH codes used for billing
6. Other (e.g., mammography)
(too broad)
Self reported, billing-based
Data Quality – Checks and Audit
20
1. Internal (health plans)
 Data checks (BI)
 Feedback loops/criterion validity
2. Quasi-external (directorate)
 Between and within health plan data checks (outliers)
 Comparison with existing national data
3. External (independent auditor)
 Process audit of infrastructure changes
 Process audit of indicator implementation
Reporting
21

Transparency + court
ruling of the
public reporting of indicators by health plan

Case mix: substantial differences between
health plans by SEP

Limited available data on SEP
Friedberg et al. (2011). Rand Corporation: Methodological
Issues in Public Reporting
Reporting - Adjusting for Case Mix
22

Israel is divided into statistical areas.

The Israel Central Bureau of Statistics calculates SEP
scores for each statistical area using recent census
information

Currently-using GIS each person’s address (unidentified
data) is linked to his statistical area and the respective
SEP score
Reporting -Benchmarking
23

Setting benchmarks: setting, testing, comparing??
Diabetes care
24
Diabetes care
25
Diabetes care includes routine monitoring and
proper control of:
• Blood glucose levels (93%)
• Cholesterol levels (90%)
• Blood pressure measurement (92%)
• BMI assessment (86%)
• Eye examination (65%)
• Influenza (55%) and pneumococcal vaccination (77%)
Diabetes care
26
International comparisons – Diabetes care
(2010)
27
*US data from the National Committee for Quality Assurance, HEDIS data set for 2010
Diabetes care – Israel and England (QOF) (2009)
Thank you
29
Directorate: Orly Manor, Arie Ben-Yehuda, Amir Shmueli, Ora Paltiel, Ronit Calderon and
Dena Jaffe
Directorate staff: Wiessam Abu Ahmad, Galit Shefer.
Maccabi
Clalit
Yair Birenboim
Chaim Bitterman
Einat Elran
Orit Ya’akobson
Nesya Gordon
Arnon Cohen
Rachel Marom
Margalit Goldsprecht
Guy Levy
Tamara Koren
External auditor: Aliza Lukach
Israel National Institute for Health Policy Research
Advisory boards
Meuhedet
Leumit
Liora Valinsky
Daniel Vardi
Yossi Zini
Eran Matz
Alon Yaffe
Doron Dushnitzky
Nirit Peretz
Importance and Relevance
30
Evidence – Moving Target
31
HbA1c control
<7%
Newer cases
<7%
18-74 years
<7%
<8%
75-84 years
<8%
Older cases
(10+ years)
“Intensive therapy was stopped after a mean of 3.5 years due to increased mortality”
The Action to Control Cardiovascular Risk in Diabetes Study Group N Engl J Med 2008;
358:2545-2559
.
Ability to Quantify – Defining Diabetes
32
Definition
2002-2010
• Purchase of
medications for
diabetes
Definition
2010-current
• Purchase of
medications for
diabetes
• Laboratory
values (high
glucose or high
HbA1c)
Uncontrolled HbA1C >9% (males)
Reporting
SES
0-4 yrs
5-17 yrs
18-24 yrs
25-34 yrs
35-44 yrs
45-54 yrs
55-64 yrs
65-74 yrs
75-84 yrs
85+ yrs
33
Low SES
0-4 yrs
5-17 yrs
18-24 yrs
25-34 yrs
35-44 yrs
45-54 yrs
55-64 yrs
65-74 yrs
75-84 yrs
85+ yrs
High SES
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
Quasi-external Audit
34
Comparison old v. new 2009
Three-year trend 2008-2010
Relative
difference
New Report
Measure name
Old
2009
New
2009
2010
20082009
20092010
Diff
Rel Diff
2008
2009
Influenza vaccination
0.605
0.605
0.00
0.00%
0.60
0.60
0.58
0.92%
-4.60%
Pneumococcal vacciation
0.500
0.796
0.81
0.79
0.78
-1.62% -1.75%
HbA1c documentation
0.931
0.931
0.00
0.00%
0.92
0.93
0.93
0.93%
-0.06%
Controlled HbA1c (0-74 yrs)
0.477
0.455
-0.02
-4.61%
0.44
0.46
0.44
3.01%
-2.50%
Uncontrolled HbA1c
0.096
0.096
0.00
0.00%
0.10
0.10
0.09
-6.62% -8.55%
Absolute value >3%
Absolute value 1-3%