Does Variation in Population Subgroups Distort
Cross Sectional Evaluation of Glycemic Control?
Implications for National Diabetes Public Health
Health,
Performance Measurement and P
P--4-P Policy
Leonard Pogach MD, MBA, FACP
Director,
Director VA New Jersey Healthcare System
Center for Healthcare Knowledge
Management
Funded VHA HSRD DM
DM--QUERI and HSRD
IIR--06
IIR
06--091
Diabetes Epidemiology Cohort (DEPIC)
Leonard Pogach
g
& Donald Miller PIs
Bedford VAMC Center for Health Quality,
Outcomes,
Outcomes & Economic Research
Donald R Miller ScD
Cindy Christiansen PhD
B. Graeme Fincke MD
Cleveland VAMC Center for Implementation
Research
David Aron MD, MS
East Orange
g VAMC Center for Healthcare
Knowledge Management
Mangala Rajan MBA
g DrPH
Chin Lin Tseng
Miriam Maney MA
Background
A1c <7.0% considered “good” control based on DCCT and
UKPDS trials [ “younger” persons with “recent” onset].
Health benefit of A1c reduction to <7% varies byy duration
of disease and life expectancy; more difficult to achieve
with longer duration disease; hypoglycemia is a limitation
of insulin treatment
Nonetheless, Agency for Healthcare Research and Quality
reports the proportion of adults (40+ years) with A1c <7%
as quality measure
– Increased
I
d to
t 48.7%
48 7% iin 1999
1999--2004 ffrom 41
41.2%
2% iin 1988
1988-1994.
The new NCQA <7% measure evaluates pts. <65 years
old
ld age without
ith t CVD or advanced
d
d conditions.
diti
– No trend data yet.
What inferences about quality of care can be made
using
i aggregate
t cross sectional
ti
l data?
d t ?
Longitudinal Analysis
All available hemoglobin A1c values used in
calculating trends in mean A1c by month over 4
years in defined cohorts (panel or cross sectional)
Growth curve model: longitudinal linear regression
model with random effects (slopes and intercepts)
for individuals nested within facilityy and yyear.
Models included adjustment for age (18(18-75
years),
y
), sex,, race,, facility,
y, and seasonality.
y
– Thompson W et al. Health Serv Res. 2005; 40:181840:1818-35.
Longitudinal A1c Results
S i lC
Serial
Cross S
Sectional
ti
l vs. P
Panell
Highest Panel
Hi h t C
Highest
Cross S
Sectional
ti
l
Average Panel
Average Cross Sectional
Conclusion: Serial Cross Sections Overestimate Improvement:
Average
g HbA1c byy Month in VA - FY1999-2003
8.2
8.0
Prevalent panel surviving: -0.06% per year
7.8
7.6
7.4
7.2
Prevalent panel died:
died: -0.14% per year
Prevalent new to VA:
VA: -0.12%
0 12% per year
7.0
6.8
Incident:
I id t: -0.01%
Incident
0 01% per year
O
ct
D 99
ec
-9
Fe 9
b0
Ap 0
r-0
Ju 0
n0
Au 0
g0
O 0
ct
D 00
ec
-0
Fe 0
b0
Ap 1
r-0
Ju 1
n0
Au 1
g0
O 1
ct
D 01
ec
-0
Fe 1
b0
Ap 2
r-0
Ju 2
n0
Au 2
g0
O 2
ct
D 02
ec
-0
Fe 2
b0
Ap 3
r-0
Ju 3
n0
Au 3
g03
6.6
Best way to study trends in glycemic control is with multi-level
growth curve longitudinal modeling – beyond scope of
Does Variation in Population Subgroups
Distort Cross Sectional Trends in A1c in the
Veteran Population?
Objective:
– To assess whether serial cross sectional
changes
h
iin adherence
dh
tto < 7% A1
A1c vary b
by
Duration of disease
Serious Co
Co--morbid illness
In--migration biases
In
M
th d – S
l /M
Methods
Sample
Measures
Identified all VHA patients with diabetes
aged 1818-75 years in 2001, 2003, and 2005
Limited to those with VHA care in the prior
year
y
with A1c performed within VHA
Outcome measure was last A1c within FY
year
Methods - Stratification
New VHA patients – in VHA in 1999? y/n
Diabetes d
duration
ration
(limited to those with >1 of prior VHA use)
– incident if 1st diabetes indicator within 12 mos
mos.
1--3 years
– recent if within 1
– prevalent
l t if llonger
– new to VHA with diabetes (duration unknown)
Serious chronic co
co--morbidity y/n
e.g. end stage hepatic or renal disease, recent
cancer, COPD, stroke, CHF, major neurological
and mental health disorders
Veteran Diabetes VHA Patients
Aged 18
18--74 years,
years VHA use in prior year
year, A1c test in year
2001
2003
2005
Patient count
344,328
521,947
691,067
Mean age (yr)
58.9
60.6
61.8
% aged 65+
29.0%
35.7%
38.2%
% female
3 9%
3.9%
3 4%
3.4%
3 3%
3.3%
%Black/Hispanic
27.0%
24.3%
23.6%
Mean A1c
7 51+/
7.51
+/--1.82
1 82 7
7.39
39+/
+/--1.68
1 68 7
7.28+/
7.28+/28+/-1.63
1 63
%<7% A1c
43.9%
47.3%
51.0%
% 8% A1
%<8%
A1c
68.9%
68 9%
72.5%
72 5%
75.3%
75 3%
%>=9% A1c
16.8%
14.4%
12.8%
Proportion with A1c < 7% in 3 periods
by Diabetes Duration
2001
100%
2003
A1c<7%
2005
A1c<7%
A1c<7%
90%
80%
70%
70.5%
60%
39.4%
72.0%
42.9%
73.0%
45.6%
Prevalent
54.0%
15.4%
57.4%
16.6%
62.7%
Recent
55.7%
12.6%
60.3%
10.4%
70.3%
Incident
51.0%
All diabetes patients
50%
40%
30%
20%
10%
16.6%
12.9%
0%
43.9%
47.3%
Proportion with A1c < 7% in 3 periods
byy Diabetes Duration,, Time in VA Care,, & Serious CoCo-morbidityy
2001
2003
2005
100%
A1c<7%
90%
25.9%
34.7%
A1c<7%
25.5%
38.7%
A1c<7%
29.9%
80%
43.2%
Prevalent, on-going VA
No serious co-morbidity
70%
60%
25.0%
42.6%
26.6%
44.6%
29.9%
46.5%
With serious co-morbidity
50%
40%
11.6%
8.0%
30%
39.2%
10%
0%
7.7%
3.5%
3.3%
%
4.1%
2.0%
44.9%
48.8%
8.2%
44.8%
8.9%
20%
11.8%
49.7%
59.0%
56.8%
65 6%
65.6%
46.8%
55.6%
43.9%
8.0%
8.3%
7.1%
3.3%
3.1%
%
4.3%
1.9%
47.7%
54.4%
60.9%
63.7%
70 2%
70.2%
51.4%
57.8%
47.3%
Prevalent, new to VA
No serious co-morbidity
4 9%
4.9%
50 1%
50.1%
With serious co
co-morbidity
morbidity
9.3%
60.2%
65.8%
Recent, no co-morbid
Recent, with co-morbid
Incident, on-going
i VA ((+/-)
/)
Incident, new to VA (+/-)
7.3%
33.6%
6%
3.3%
2.4%
1.1%
72.2%
77.6%
59.3%
65.6%
51.0%
All diabetes patients
Conclusions
Overall population improvements in A1c
<7% represent increased proportions of
veterans with A1c <7% in all subgroups
Individuals with incident or recent onset
diabetes have the highest percent with
A1 <7%
A1c
Adherence rates higher among all
d rations if comorbid illness present
durations
Adherence rates differ by continuity of
care within system (self selection bias)
Do Facility-Level Rankings of Glycemic
Control Differ by Treatment Intensity?
Using the specifications of the current first year
NCQA <7% measure as the comparator, our
objectives were to evaluate VA facility rankings
– When complex glycemic regimens are used
– When additional coco-morbid conditions are
excluded
l d d
– Comparing a continuous and weighted
meas re [linear and proportional] o
measure
over
er the
range of 7.9% (<8%) to 6.9% (<7%) to a
threshold measure (<7%)
STUDY POPULATION
355,015 subjects <65 years from 133 facilities
in FY2004 who received VA care in 2003
100 258 subjects
100,258
bj t excluded
l d d using
i NCQA criteria
it i
for cardiovascular disease, advanced
complications and dementia.
The remaining 254,857 subjects were stratified
by complex glycemic regimens (CGR):
– Two
T o or three oral agents that incl
included
ded
thiazolidinediones or insulin
An additional 2,144
,
subjects
j
from 5 facilities
were dropped due to small CGR treatment
populations (< 100)
252 713 subjects with diabetes from 128
252,713
facilities in FY 2004.
Methods
Measures:
– Threshold: % of subjects with A1c<7%
– Continuous & Weighted: Using linear
interpolation, proportional credit awarded for
values between 7.9% ((=0.091)) and 6.9%
(=1.0)
Facilities ranked using NCQA methodology into 5
ordinal categories based upon both Z score
(<=10%; 1010-33%; 34
34--66%; 67
67--90%, and >90%)
and statistical significance
g
((p<0.05)) after
standardization. The interval was calculated as
follows:
– (Facility Rate – Overall mean rate) ± 1.96√
1 96√ (SE
Facility)² + (SE Overall)²
GUIDE TO TEXAS HMO QUALITY: 2007 http://www.dshs.state.tx.us/thcic/publications/HMOs/HMOReports.shtm
Table 1: Characteristics of Diabetic, NCQA Population, NCQA
Population with additional exclusion criteria and Complex Patient
Population
Attribute
Total N
% D enominator
Total Facilities
Mean N per
facility (SD )
Range
Mean Age
Gender
Fem ale
Male
N CQA
D enominator
254,857
Complex
Patients
N CQA
w ith
additional
Exclusions#
128
71,879
(28.2%)
128
176,708
(69.3%)
128
1,974 (1150)
582 – 6,353
430 (259 )
101 – 1,529
1,370 (802)
373 – 4,587
55.30
54.25
55.50
4.7%
95.3%
4.5%
95.6%
4.5%
95.5%
Table 1: Denominator Characteristics
NCQA
Q
Denominator
Total N
Complex
p
Patients
NCQA with
additional
Exclusions#
254,857
71,879
176,708
No medications
34.1%
NA
36.4%
OHA Onlyy
44.1%
23.4%
44.9%
- Three or more OHA's
4.4%
15.5%
4.8%
- Metformin + TZD
0.7%
2.4%
0.8%
- Sulphonyureas + TZD
1 6%
1.6%
5 5%
5.5%
1 6%
1.6%
13.3%
47.0%
11.8%
8.4%
29.6%
6.9%
Decreased Life Expectancy
5.1%
5.3%
NA
Advanced Complications
3.3%
6.1%
NA
Serious Medical Conditions
5.6%
9.1%
NA
Serious Neurological Conditions
3.1%
4.1%
NA
19.0%
21.0%
NA
Medication Profile
OHA and Insulin
Insulin only
Serious Mental Health Conditions
# Pogach et al. Am Journal of Man Care, 2007, 13:133, The analysis utilized SAS version 9.1.3, released by SAS Institute Ink., Cary,
Table 2A: Rankings Using Threshold
NCQA Star Ratings
NCQA Denominator
***** ****
49% 44% 39% 34% 24%
24%
Proportion <7% maximum
60% 49% 45% 40% 33%
60%
Mean Proportion
52% 46% 42% 37% 31%
42%
95
28
Total
Proportion <7% minimum
3
53
*
128
11
23
**
12
Complex Patients
12
***
8
11
128
Proportion <7% minimum
27% 25% 15% 16% 11%
11%
Proportion <7% maximum
36% 26% 28% 19% 16%
36%
Mean Proportion
31% 26% 22% 17% 14%
22%
Markedly lower rates of adherence with CGR; fewer best and worse outliers
Table 2B: Rankings using Continuous Measure
NCQA Star Ratings
NCQA Denominator measured with QALY's
***** ****
58% 53% 48% 43% 28%
28%
P
Proportion
i QALY'
QALY's achieved
hi
d - max
71% 57% 53% 48% 42%
71%
Mean Proportion
61% 55% 50% 46% 38%
50%
84
29
Total
Proportion QALY's achieved - min
8
46
*
128
11
29
**
12
Complex Patients measured with QALY's
12
***
13
12
128
Proportion QALY's achieved - min
37% 34% 24% 24% 18%
18%
Proportion QALY's achieved - max
45% 37% 38% 28% 24%
45%
Mean Proportion
41% 36% 31% 26% 22%
31%
Conclusions
Fewer facilities ranked as statistical outliers (2 or
4 star) when the subset of complex patients was
evaluated
– These findings
g may
y represent sample size
considerations and/or differential quality of
care
The use of a continuous and weighted measure
resulted in increased “adherence” to the
measure
– Increased n
number
mber of statistical o
outliers
tliers
The use of additional exclusion criteria did not
markedlyy impact
p
adherence or rankings
g ((data
not shown)
Policy Implications:
Longitudinal data using individual level data and panels
provides increased detail of trends for population
health, but is not practical for operations
Stratification, comprehensive exclusion criteria, and use
of continuous measures using
g cross sectional data may
y
– Minimize attribution error
error—
—e.g.
e.g.--higher
rankers=better care, which can then
– Prioritize quality improvement efforts for veterans at
highest risk and therefore
– Reward clinically significant improvement that does
not reach “threshold”
Evaluation of harms (ER visits, hospitalizations,
mortality) and costs (pharmaceutical agents) should be
evaluated
Aron & Pogach JCJQS 2007;33:6362007;33:636643