Next Steps in Measuring
Clinical Quality
Joe V. Selby, MD
Division of Research
Kaiser Permanente Northern California
Differences in Clinical Quality –
Diabetes Care
Plan A
Plan B
Retinal Screening (P)
41.4
45.3
Hb A1c Testing (P)
72.7
75.1
Hb A1c Control (O)
61.9
55.2
Monitoring Nephropathy (P)
28.5
36.1
LDL-C Testing (P)
60.7
69.1
LDL-C Control (O)
29.1
36.7
*HEDIS
Health Plan Summary Data
The Chasm in Clinical Quality Assessment
What We Know
Quantitative effects of many
process measures and of
differences in outcomes on
survival and on non-fatal
complications in populations
– from clinical trials
What We Measure
Processes of care not
known to be related to
outcomes or effectiveness
Semi-quantitative
outcomes (Hb A1c >9.5%,
LDL-C < 100) that hide
more effectiveness
differences than they
reveal
Point #1
Population Rates for Simple Process
Measures do NOT Consistently Reflect
Clinical Benefit in those Populations
Translating Research Into Action for Diabetes
A multi-center cohort study of diabetes in managed care settings
Pacific Health
Research
Institute
Kaiser Permanente
No. California
UMDNJ
U. Michigan
Indiana U.
UCLA
PacifiCare
Texas
Centers for Disease Control Sponsor and Data Coordinating
Center
CDC
The TRIAD Sampling Scheme
10 health plans
(n=500 to 2000
per plan)
67 physician
groups with
> 50
members
in sampling
frame
(Sampling scheme: Aimed for equal numbers
from each physician group within health
plan, so from 50 - 1500 per physician group)
TRIAD Data (2000-2001)
Patient Surveys (telephone or mailed) –
11,928 respondents
Chart Reviews – 8,757 patients
Medical Director Surveys – health plan
and provider group directors
Four Measures of Disease Management
Intensity – from Health Plan and Provider
Group Director Surveys
Use of diabetes registries
Use of clinician reminders
Performance feedback to physicians
Diabetes care management:
Guideline use
Patient reminders,
Patient education
Use of care/case managers
Provider Group Performance Difference (%)
(80th – 20th Percentile of Dis Mgmt Intensity)
PROCESS MEASURES
Care
Management
Performance
Feedback
Diabetes
Registry
MD
Reminders
Hb A1c Test
11
0.001
9
0.0001
9
0.01
4
0.07
LDL-C Test
13
0.0001
8
0.001
11
0.01
2
0.59
Retinal Exam
7
0.01
8
0.001
4
0.13
7
0.001
Urine
Albumin
16
0.0001
11
0.0001
13
0.01
10
0.01
Foot Exam
8
0.01
6
0.01
3
0.45
5
0.05
Aspirin
Advised
0
0.99
1
0.74
3
0.30
3
0.38
adjusted for patient age, sex, race, education/income, diabetes treatment and
duration, comorbidities, SF-12 (PCS), health plan disease mgmt intensity
Provider Group Performance Differences
(80th – 20th Percentile of Dis Mgmt Intensity)
INTERMEDIATE OUTCOMES
Care
Management
Hb A1c (%)
0.1 0.71
Performance
Feedback
-0.1 0.74
Diabetes
Registry
MD
Reminders
-0.1
0.55
0
0.74
Syst. Blood
Pressure
(mmHg)
2
0.01
1
0.22
3
0.01
1
0.22
LDLcholesterol
(mg/dL)
2
0.06
2
0.70
2
0.46
0
0.70
adjusted for patient age, sex, race, education/income, diabetes treatment and
duration, comorbidities, health plan disease mgmt intensity
Moreover,
Provider Group intensity of disease management
also unrelated to the appropriateness* of treatment
for each condition
Provider Group Quality Scores based on process
measures were unrelated to provider group levels
of control of blood pressure, LDL-C or Hb A1c
*Proportion in control or on appropriately aggressive pharmacotherapy
Point #2
Even if we measure evidence-based
processes or outcomes, the potpourri of
indicators within and across diseases
don’t readily yield a measure of overall
clinical benefit
Differences in Clinical Quality
(hypothetical) based on evidence-based
processes/ outcomes
Plan A
Plan B
Patients Using Aspirin (%)
41
54
Mean Hb A1c (%)
8.1
7.5
Mean LDL-C (mg/dL)
106
131
Mean SBP (mmHg)
141
136
Flu Shot Past 12 mos (%)
67
54
How Do We Quantify the Net Benefit?
Each of these differences represents a predictable change
in expected survival and complications (i.e., each
measures a clinical benefit )
But practical questions remain:
Which is more important, the difference in Hb A1c levels
or the difference in BP control?
Should plans, providers work to improve multiple
measures modestly, or drive one indicator toward the
optimal for all patients?
We need a composite, quantitative measure of net
clinical benefit that can be compared across plans,
provider groups, systems.
Quality-adjusted life-year
The QALY
A common metric for measuring clinical
quality (both survival and quality of life)
Across interventions (using aspirin, BP lowering)
Across perspectives (patient, provider, purchaser)
Across diseases (diabetes, CHF, CAD, asthma)
Across activities (e.g., chronic disease care,
prevention)
Where Do QALY’s Come From?
Creating a Quantitative Metric for
Diabetes
Systolic
Blood
Pressure
Hemoglobin
A1C
Risk
Adjusters
Natural
History
Model
Expected
Survival &
Complicatons
LDLCholesterol
Aspirin
Use
Adjusted
Life-expectancy
Potential Advantages of Model
Expresses quality in familiar metric – life expectancy
Requires clinical trial evidence clearly evidence-based
Allows exploration to explain differences, which
emphasizes population importance of various indicators
Potential Disadvantages/Questions
Will require extensive explanation and transparency of the
model to gain acceptance
New evidence will have to be incorporated over time,
potentially altering metrics across years
Because it takes a population or public health perspective,
will not capture quality of care well for rare conditions
(because prevalence too small)
Questions of whether and how to adjust for case-mix
differences between population will have to be addressed