D I S S E R T A T I O N
R
The Potential of Claims
Data to Support the
Measurement of Health
Care Quality
Jennifer Hicks
RAND Graduate School
This document was prepared as a dissertation in March 2003 in
partial fulfillment of the requirements for the degree of Doctor of
Philosophy in Policy Analysis at the RAND Graduate School. The
faculty committee that supervised and approved the dissertation
consisted of Steven Garber (Chair), John Adams, Sandra Berry, and
Elizabeth McGlynn.
The RAND Graduate School dissertation series reproduces dissertations that have
been approved by the student’s dissertation committee.
RAND is a nonprofit institution that helps improve policy and decisionmaking
through research and analysis. RAND® is a registered trademark. RAND’s
publications do not necessarily reflect the opinions or policies of its research
sponsors.
© Copyright 2003 RAND
All rights reserved. No part of this book may be reproduced in any form by any
electronic or mechanical means (including photocopying, recording, or
information storage and retrieval) without permission in writing from RAND.
Published 2003 by RAND
1700 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138
1200 South Hayes Street, Arlington, VA 22202-5050
201 North Craig Street, Suite 202, Pittsburgh, PA 15213-1516
RAND URL: http://www.rand.org/
To order RAND documents or to obtain additional information,
contact Distribution Services: Telephone: (310) 451-7002;
Fax: (310) 451-6915; Email: order@rand.org
-iii -
PREFACE
While widespread quality measurement relying extensively on
medical records would provide more broad-based and accurate information,
the associated costs currently appear prohibitive to most organizations.
This economic reality provides the key motivation for this dissertation,
which explores the capacity of claims data, a less costly, and albeit
less informative data source, to expand the scope of quality
measurement.
This research examines (a) the dimensions of clinical quality that
can be measured with claims data and (b) the types of supplemental
information that would most increase our capacity to measure health care
quality with electronic data.
It is intended to be of interest to
researchers, policy makers, and practitioners who are currently engaged
in quality measurement activities or considering future investments in
administrative or clinical information systems.
This study was submitted as a dissertation to the RAND Graduate
School in March 2003 in partial fulfillment of the requirements for the
degree of Doctor of Philosophy in Policy Analysis.
The research was
funded by the Agency for Healthcare Research and Quality under
dissertation grant #1-R03-HS11721-01.
-v-
CONTENTS
PREFACE ............................................................. iii
CONTENTS .............................................................. v
LIST OF FIGURES ...................................................... ix
LIST OF TABLES ....................................................... xi
ABSTRACT ........................................................... xiii
ACKNOWLEDGEMENTS ..................................................... xv
1.
INTRODUCTION ......................................................
Data Sources and Quality Measurement ............................
Purpose of the Dissertation .....................................
Organization of the Dissertation ................................
1
2
4
5
2.
CONCEPTUALIZING AND MEASURING QUALITY OF CARE .................... 7
What is quality and how is it measured? ......................... 7
What is a quality indicator? .................................... 9
Data sources for quality measurement ........................... 11
Availability of claims data .................................... 13
Whose information is included in claims data?................ 13
What information is included in claims data?................. 14
Summary: Availability of Claims Data for Quality Measurement. 20
Accuracy of CLAIMS data ........................................ 20
Agreement between medical record and claims data............. 21
Sources of Inaccuracy in Claims Data......................... 23
Summary: Accuracy of Claims Data............................. 26
Applications of CLAIMS data to measure quality ................. 27
3.
ASSESSING THE FEASIBILITY OF MEASURING QUALITY WITH CLAIMS DATA . 29
Selected indicator Set: QA Tools ............................... 30
Constructing quality of care indicators ........................ 32
Conceptual model for characterizing quality measures ........... 33
Primary Means of Indicator Classification: Modality.......... 34
Secondary Means of Indicator Classification.................. 37
Summary...................................................... 43
Methods ........................................................ 43
Step 1: Identifying Data Elements............................ 43
Step 2: Classifying the Data Elements........................ 44
Step 3: Assessing the Availability of Data Elements in Claims
Data ...................................................... 47
Step 4: Feasibility of Constructing Indicators with Claims Data48
Example 1: An indicator that can be constructed with claims
data ...................................................... 48
Example 2: An indicator that cannot be constructed with claims
data ...................................................... 50
Summary of approach to feasibility analysis.................. 52
Findings: Data element analysis ................................ 53
Availability of information on laboratory tests.............. 55
Availability of information on procedures.................... 55
-vi-
Availability of information on medications...................
Availability of information on diagnoses.....................
Findings: indicator analysis ...................................
Summary of Feasibility Assessment............................
Increasing the capacity of CLAIMS data for quality measurement .
How Could Additional Information be Obtained?................
Methods......................................................
Findings and Discussion......................................
Description of effects of additional information on quality
measurement. ..............................................
Description of effects of additional information on quality
measurement. ..............................................
Summary......................................................
4.
56
57
59
64
65
65
69
70
75
76
77
THE ACCURACY OF QUALITY MEASUREMENT WITH CLAIMS DATA ............ 79
Description of the Data ........................................ 79
HMO Data..................................................... 80
Study approach ................................................. 81
Selected Indicators.......................................... 82
Constructing the QA Tools Indicators with Claims Data........ 84
Measuring Agreement.......................................... 88
Description of Agreement about Eligibility and Scoring....... 91
Theory of agreement ............................................ 95
The Effects of Errors on Quality Measurement................. 95
Factors Affecting Agreement Between Claims and Medical Records
Data ...................................................... 96
Summary of Hypotheses....................................... 102
Agreement Analysis ............................................ 103
Independent Variables....................................... 103
Dependent Variables......................................... 107
Distribution of Variables................................... 108
Bivariate Analysis of Agreement............................. 111
Multivariate Analysis....................................... 116
Scoring - Multivariate Analysis about Agreement............. 122
Summary of Multivariate Analysis of Agreement about Eligibility
and Scoring .............................................. 127
Agreement About Performance Rates........................... 131
Conclusions ................................................... 136
5. MOVING QUALITY MEASUREMENT FORWARD ...............................
Untapped Potential of Claims Data for Quality Measurement...
What can and cannot be Measured with Claims Data?...........
Building a Health Information Infrastructure................
What Additional Data Elements Contribute the Most to Quality
Measurement? .............................................
137
138
139
140
142
APPENDIX A – TABULAR RESULTS: INCREASING CAPACITY FOR QUALITY
MEASUREMENT .................................................... 145
APPENDIX B – INDICATORS USED TO ASSESS AGGREEMENT ................... 147
APPENDIX C – CLAIMS DATA SPECIFICATIONS .............................
ASTHMA ........................................................
CORONARY ARTERY DISEASE .......................................
DIABETES ......................................................
154
154
164
189
-vii-
HEART FAILURE ................................................. 196
PNEUMONIA ..................................................... 211
PREVENTIVE CARE ............................................... 228
APPENDIX D – KAPPA STATISTICS ....................................... 240
APPENDIX E – SEMI-STANDARDIZED REGRESSION COEFFICIENTS .............. 242
REFERENCES .......................................................... 245
-ix-
LIST OF FIGURES
Figure 3.1
Figure 3.2
Figure 3.3
Figure 3.4
Figure 3.5
Figure 3.6
Figure 3.7
Figure 3.8
Figure 3.9
Figure 3.10
Figure 3.11
Figure 4.1
Figure 4.2
Relationship of Function and Modality Among 553 QA
Tools Indicators
Distribution of QA Tools Indicators by Function and
Payment Status
Type and Modality of QA Tools Indicators
Distribution of QA Tools Indicators by Type and
Payment Status
Illustrating the 4 Steps of the Feasibility Analysis:
Example 1
Illustrating the 4 Steps of the Feasibility Analysis:
Example 2
Distribution of Indicators that can be Constructed
with Claims Data – by Payment Status
Potential Increase in Capacity for Quality
Measurement – All Indicators
Potential Increase in Capacity for Quality
Measurement – by Modality
Potential Increase in Capacity for Quality
Measurement – by Function
Potential Increase in Capacity for Quality
Measurement – by Type of Care
Schematic Overview of Analysis of Agreement between
Medical Record and Claims Data
Data for Agreement Analyses about Eligibility and
Scoring
38
39
41
42
50
52
62
72
73
74
75
89
93
-xi-
LIST OF TABLES
Table 2.1
Table 2.2
Table 2.3
Table 3.1
Table 3.2
Table 3.3
Table 3.4
Table 3.5
Table 3.6
Table 3.7
Table 3.8
Table 3.9
Table 3.10
Table 3.11
Table 4.1
Table 4.2
Table 4.3
Table 4.4
Table 4.5
Table 4.6
Table 4.7
Table 4.8
Table 4.9
List of Standard Data Elements: Hospital Claims Forms
List of Standard Data Elements: Outpatient Claims
Forms
List of Standard Data Elements: Prescription Claims
Forms
Adult Clinical Areas in the QA Tools System
Modalities of Care: Definitions, Distribution in QA
Tools, and Relationship to Billing
Categories Used to Classify Data Elements
Distribution of Data Element Types across Components
of Indicators
Number of Data Elements by Classification and
Proportion Available in Claims Data
Assessment of Data Elements in Claims Data:
Prescription Sub-Classifications
Assessment of Data Elements in Claims Data: Diagnosis
Sub-Classifications
Number and Proportion of Indicators that can be
Constructed with Claims Data – by Modality
Number and Proportion of Indicators that can be
Constructed with Claims Data – by Function of Care
Number and Proportion of Indicators that can be
Constructed with Claims Data – by Type of Care
Potential Increase in Capacity for Quality
Measurement – All Indicators
Descriptive Statistics of Study Sample
Description of Indicators Used in Agreement Analysis
Standardized Codes Used to Identify People Meeting
Eligibility and Scoring Criteria – Example
Claims and Medical Records Data Agreement about
Eligibility and Scoring
Independent Variables for Agreement Equations
Distribution of Covariates Across the Eligibility
Models
Distribution of Covariates Across the Scoring Models
Levels of Agreement about Eligibility – Bivariate
Comparisons
Level of Agreement about Scoring – Bivariate
Comparisons
15
16
17
31
36
45
47
54
57
58
60
63
64
71
81
84
87
94
105
109
111
113
115
- xii-
Table 4.10
Table 4.11
Table 4.12
Table 4.13
Table D.1
Table D.2
Table E.1
Table E.2
Odds Ratios from Logistic Regressions for Agreement
between Claims Data and Medical Records about
Eligibility
Odds Ratios for Agreement between Claims Data and
Medical Records about Scoring
Results of Hypothesis Testing
Comparing Performance Rates from Claims and Medical
Records Data
Overall Agreement about Eligibility: Kappa Statistics
Overall Agreement about Scoring: Kappa Statistics
Semi-Standardized Regression Coefficients for
Agreement Models about Eligibility
Semi-Standardized Regression Coefficients for
Agreement Models about Scoring
121
126
130
134
242
243
244
245
- xiii-
ABSTRACT
Annual health expenditures in the United States exceed $1.5
trillion.
Despite a growing number of efforts to measure and report
levels of health care quality resulting from these expenditures, useful
information is neither uniformly nor widely available.
However, the
information that is available suggests that millions of Americans fail
to receive adequate quality care.
Consequently, there are large
potential gains in welfare associated with improving the performance of
the health care system.
There would be enormous value in finding
reliable ways to measure the quality of care in a routine and affordable
manner.
The goals of this dissertation were to (1) examine whether the
capacity of claims data for quality measurement is being fully
recognized and (2) identify the types of information that, if available,
would increase the potential for quality measurement with electronic
data.
The strengths and limitations of quality measurement with claims
data were examined using two separate analyses. First, more than 550
quality of care indicators were analyzed for whether they could be
assessed with claims data.
This analysis was performed to evaluate the
dimensions of quality that can be evaluated with claims data and to
identify the types of information that would increase the capacity for
quality measurement with electronic data.
A separate analysis compared
agreement between claims and medical records data about quality
measurements to assess the accuracy of quality measurement with claims
data.
Of the 553 indicators reviewed, 186 (34%) could be assessed with
claims data.
This represents a significant increase in the number of
indicators commonly used to measure quality.
However, because the
primary purpose of claims data is to facilitate payment, they do not
include sufficiently detailed clinical information to assess many
quality of care indicators.
As investments in electronic health information increase, it would
be helpful to know, from the perspective of quality measurement, which
-xiv -
types of information would add the most value.
The results of this
study found that the addition of any one type of information would not
have a very large contribution to the number or type of indicators that
could be constructed.
While more detailed information about patients’
presenting signs and symptoms would have the largest contribution to
quality measurement, linking existing data sources, such as laboratory
test results, to claims data may be a more practical next step in
increasing the capacity for quality measurement with electronic data.
When performance rates for using claims data were compared to rates
constructed with medical records data, the medical records rates were
not consistently higher than the claims based rates.
This finding
contradicts the assumption that claims data will consistently
underestimate the rate of performance.
While widespread quality measurement relying extensively on medical
records would provide more broad-based and accurate information, the
associated costs currently appear prohibitive to most organizations.
There are four broad options for quality measurement over the next
several years: (1) the status quo; (2) expanded use of claims data,
possibly augmented with the kinds of information analyzed in chapter 3;
(3) expanded use of medical records; and (4) expanded use of both claims
data and medical records.
The benefits and costs of various versions of
the last three options relative to the status quo, and relative to each
other, are important, but open, questions.
-xv-
ACKNOWLEDGEMENTS
I am grateful for the support and guidance that many individuals
provided me during this study.
I extend special thanks to my committee
members – Steve Garber, Beth McGlynn, John Adams, and Sandy Berry – for
their contributions to my research as well as to my professional
training.
I have benefited greatly from working with these researchers
and am grateful for their patience and guidance.
Professor David
Nerenz, from Michigan State University, read an earlier draft of the
dissertation and offered constructive comments and insights from which I
have benefited.
I would also like to extend special thanks to Joan Keesey and David
Klein for their analytic support.
The programming expertise they
provided and their willingness to teach me the basics of SAS were
integral to this study.
I am also grateful to the staff at the HMO who
not only provided the data for this study but also accommodated
questions throughout my research.
I cannot begin to express my gratitude to my family and friends
for their continued support and encouragement in my academic and
professional choices.
This work was supported by a dissertation grant from the Agency for
Healthcare Research and Quality.