Secondary Data
Analysis:
Opportunities and Pitfalls
Who am I and why am I here?




Laurel A. Copeland, BS MPH PhD
– VA Health Services Research &
Development investigator; UTHSCSA
Department of Psychiatry Assistant Professor
- Research
copelandl@uthscsa.edu
http://czresearch.com/dropbox
What Can Secondary Data
Analysis Do for You?
•
•
•
Provide preliminary data for grant proposals
Facilitate publication while you apply for
grants
Explore new areas







































My CV
Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE; Butchart, Amy T (1993). Health Education Quarterly 20(3): 373-390.
Shope, Jean T; Copeland, Laurel A; Dielman, TE (1994). Alcoholism: Clinical and Experimental Research 18(3): 726-733.
Shope, Jean T; Copeland, Laurel A; Marcoux, Beth C; Kamp, Mary E (1996). Journal of Drug Education, 26(4): 323-337.
Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE (1996). Alcoholism: Clinical & Experimental Research, 20(5): 791-798.
Copeland, Laurel A; Shope, Jean T; Waller, Patricia F (1996). Journal of School Health, 66(7): 254-260.
Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A (1997). Journal of Family Practice, 45(2): 151-158.
Zimmerman, Marc A; Copeland, Laurel A; Shope, Jean T; Dielman, TE (1997). Journal of Youth and Adolescence, 26(2): 117-141.
Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A; Appel, Scott (1998). Family Medicine, 30(5):366-371.
Blow, Frederic C; Barry, Kristen Lawton; BootsMiller, Bonnie J; Copeland, Laurel A; McCormick, Richard; Visnic, Stephanie (1998). Journal of Psychiatric Research, 32: 311-319.
Valenstein, Marcia; Barry, Kristen Lawton; Blow, Frederic C; Copeland, Laurel A; Ullman, Esther (1998). Psychiatric Services, 49(8): 1043-1048.
Shope, Jean T; Copeland, Laurel A; Kamp, Mary E; Lang, Sylvia W (1998). Journal of Drug Education, 28(3): 185-197.
Kales, Helen C; Blow, Frederic C; Copeland, Laurel A; Bingham, Raymond C; Kammerer, Ericka E; Mellow, Alan F (1999). American Journal of Psychiatry, 156(4): 550-556.
Blow, Frederic C; Barry, Kristen Lawton; Copeland, Laurel A; McCormick, Richard; Lehmann, Laurent; Ullman, Esther (1999). Psychiatric Services, 50(3): 390-394.
Maio, Ronald F; Shope, Jean T; Blow, Frederic C; Copeland, Laurel A; Gregor, MaryAnn; Brockmann, Laurie M; Weber, Janet E; Metrou, Mary E (2000). Annals of Emergency
Medicine, 35(3):252-257.
Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Copeland, Laurel A; Mellow, Alan M (2000). Psychiatric Services, 51(6): 795-800.
Roberts, J Scott; Blow, Frederic C; Copeland, Laurel A; Barry, Kristen Lawton; Van Stone, William (2000). Journal of Geriatric Psychiatry and Neurology, 13 Summer: 78-86.
Blow, Frederic C; Walton, Maureen A; Barry, Kristen Lawton; Coyne, James C; Mudd, Sharon A; Copeland, Laurel A (2000). Journal of the American Geriatrics Society, 48(7): 769774.
Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Roberts, J Scott; Copeland, Laurel A; Mellow, Alan M (2000). American Journal of Geriatric Psychiatry, 8:301-309.
Blow, Frederic C; Ullman, Esther; Barry, Kristen Lawton; Bingham, C Raymond; Copeland, Laurel A; McCormick, Richard; Van Stone, William (2000). American Journal of
Orthopsychiatry, 70(3): 389-400.
Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Journal of Clinical Psychiatry 62(7): 545-551.
Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Psychiatric Services, 52(9): 1242-1244.
Duffy, Sonia A; Terrell, Jeff; Valenstein, Marcia; Ronis, David; Copeland, Laurel A; Connors, Mary (2002). General Hospital Psychiatry, 24: 140-147.
Valenstein, Marcia; Copeland, Laurel A; Blow, Frederic C; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2002). Medical Care 40(8):
630-639.
Hegedus, Andrea M; Copeland, Laurel A; Barry, Kristen Lawton; Blow, Frederic C. (2002). American Journal of Orthopsychiatry 72(3): 331-340.
Copeland, Laurel A; Blow, Frederic C; Barry, Kristen Lawton (2003). Health Education & Behavior 30(3):305-321.
Copeland, Laurel A; Zeber, John E; Valenstein, Marcia; Blow, Frederic C (2003). American Journal of Psychiatry, 160:1817-1822.
Valenstein, Marcia; Blow, Frederic C; Copeland, Laurel A; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2004). Schizophrenia Bulletin
30(2): 255-64.
Janz, Nancy K.; Wren, Patricia A.; Copeland, Laurel A.; Lowery, Julie C.; Goldfarb, Sherry L.; Wilkins, Edwin G. (2004). Journal of Clinical Oncology 22(15):3091-3098.
Lambert, Michael T.; Terrell, Jeffrey E.; Copeland, Laurel A.; Ronis, David L.; Duffy, Sonia A. (2005). Nicotine & Tobacco Research 7(2):233-241.
Copeland, Laurel A; Fletcher, Carol E; Patterson, Judith (2005). Military Medicine 170(7):602-606.
Copeland, Laurel A; Elshaikh, Mohamed A; Jackson, James; Penner, Lou A; Underwood III, Willie (2005). Cancer 104(7):1372-1380.
O'Neill, Jessica L; Flanagan, Petra S; Zaleon, Carolyn R; Copeland, Laurel A (2005). Pharmacotherapy 25(11):1560-1565.
Copeland, Laurel A; Zeber, John E; Rosenheck, Robert A; Miller, Alexander L (in press). Medical Care.
Pugh, Mary Jo V; Copeland, Laurel A; Zeber, John E; Cramer, Joyce A; Amuan, Megan T; Cavazos, Jose E; Kazis, Lewis E (in press). Epilepsia.
Silveira, Maria J; Copeland, Laurel A; Feudtner, Chris (2006). American Journal of Public Health v96:1243-1248.
Hopp, Faith P; Woodbridge, Peter; Subramanian, Usha; Copeland, Laurel A; Smith, David; Lowery, Julie C (2006). Telemedicine and eHealth.
Zeber, John E; Copeland, Laurel A; Grazier, Kyle L (2006). Military Medicine v171(7):619-626.
Alexander JA, Copeland LA, Metzger ME (in press). Clinical Toxicology.
Wilkins EG, Lowery JC, Copeland LA, Goldfarb SL, Wren PA, Janz NA (in press). Medical Decision Making.







































My CV without 2° Data
Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE; Butchart, Amy T (1993). Health Education Quarterly 20(3): 373-390.
Shope, Jean T; Copeland, Laurel A; Dielman, TE (1994). Alcoholism: Clinical and Experimental Research 18(3): 726-733.
Shope, Jean T; Copeland, Laurel A; Marcoux, Beth C; Kamp, Mary E (1996). Journal of Drug Education, 26(4): 323-337.
Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE (1996). Alcoholism: Clinical & Experimental Research, 20(5): 791-798.
Copeland, Laurel A; Shope, Jean T; Waller, Patricia F (1996). Journal of School Health, 66(7): 254-260.
Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A (1997). Journal of Family Practice, 45(2): 151-158.
Zimmerman, Marc A; Copeland, Laurel A; Shope, Jean T; Dielman, TE (1997). Journal of Youth and Adolescence, 26(2): 117-141.
Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A; Appel, Scott (1998). Family Medicine, 30(5):366-371.
Blow, Frederic C; Barry, Kristen Lawton; BootsMiller, Bonnie J; Copeland, Laurel A; McCormick, Richard; Visnic, Stephanie (1998). Journal of Psychiatric Research, 32: 311-319.
Valenstein, Marcia; Barry, Kristen Lawton; Blow, Frederic C; Copeland, Laurel A; Ullman, Esther (1998). Psychiatric Services, 49(8): 1043-1048.
Shope, Jean T; Copeland, Laurel A; Kamp, Mary E; Lang, Sylvia W (1998). Journal of Drug Education, 28(3): 185-197.
Kales, Helen C; Blow, Frederic C; Copeland, Laurel A; Bingham, Raymond C; Kammerer, Ericka E; Mellow, Alan F (1999). American Journal of Psychiatry, 156(4): 550-556.
Blow, Frederic C; Barry, Kristen Lawton; Copeland, Laurel A; McCormick, Richard; Lehmann, Laurent; Ullman, Esther (1999). Psychiatric Services, 50(3): 390-394.
Maio, Ronald F; Shope, Jean T; Blow, Frederic C; Copeland, Laurel A; Gregor, MaryAnn; Brockmann, Laurie M; Weber, Janet E; Metrou, Mary E (2000). Annals of Emergency
Medicine, 35(3):252-257.
Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Copeland, Laurel A; Mellow, Alan M (2000). Psychiatric Services, 51(6): 795-800.
Roberts, J Scott; Blow, Frederic C; Copeland, Laurel A; Barry, Kristen Lawton; Van Stone, William (2000). Journal of Geriatric Psychiatry and Neurology, 13 Summer: 78-86.
Blow, Frederic C; Walton, Maureen A; Barry, Kristen Lawton; Coyne, James C; Mudd, Sharon A; Copeland, Laurel A (2000). Journal of the American Geriatrics Society, 48(7): 769774.
Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Roberts, J Scott; Copeland, Laurel A; Mellow, Alan M (2000). American Journal of Geriatric Psychiatry, 8:301-309.
Blow, Frederic C; Ullman, Esther; Barry, Kristen Lawton; Bingham, C Raymond; Copeland, Laurel A; McCormick, Richard; Van Stone, William (2000). American Journal of
Orthopsychiatry, 70(3): 389-400.
Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Journal of Clinical Psychiatry 62(7): 545-551.
Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Psychiatric Services, 52(9): 1242-1244.
Duffy, Sonia A; Terrell, Jeff; Valenstein, Marcia; Ronis, David; Copeland, Laurel A; Connors, Mary (2002). General Hospital Psychiatry, 24: 140-147.
Valenstein, Marcia; Copeland, Laurel A; Blow, Frederic C; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2002). Medical Care 40(8):
630-639.
Hegedus, Andrea M; Copeland, Laurel A; Barry, Kristen Lawton; Blow, Frederic C. (2002). American Journal of Orthopsychiatry 72(3): 331-340.
Copeland, Laurel A; Blow, Frederic C; Barry, Kristen Lawton (2003). Health Education & Behavior 30(3):305-321.
Copeland, Laurel A; Zeber, John E; Valenstein, Marcia; Blow, Frederic C (2003). American Journal of Psychiatry, 160:1817-1822.
Valenstein, Marcia; Blow, Frederic C; Copeland, Laurel A; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2004). Schizophrenia Bulletin
30(2): 255-64.
Janz, Nancy K.; Wren, Patricia A.; Copeland, Laurel A.; Lowery, Julie C.; Goldfarb, Sherry L.; Wilkins, Edwin G. (2004). Journal of Clinical Oncology 22(15):3091-3098.
Lambert, Michael T.; Terrell, Jeffrey E.; Copeland, Laurel A.; Ronis, David L.; Duffy, Sonia A. (2005). Nicotine & Tobacco Research 7(2):233-241.
Copeland, Laurel A; Fletcher, Carol E; Patterson, Judith (2005). Military Medicine 170(7):602-606.
Copeland, Laurel A; Elshaikh, Mohamed A; Jackson, James; Penner, Lou A; Underwood III, Willie (2005). Cancer 104(7):1372-1380.
O'Neill, Jessica L; Flanagan, Petra S; Zaleon, Carolyn R; Copeland, Laurel A (2005). Pharmacotherapy 25(11):1560-1565.
Copeland, Laurel A; Zeber, John E; Rosenheck, Robert A; Miller, Alexander L (in press). Medical Care.
Pugh, Mary Jo V; Copeland, Laurel A; Zeber, John E; Cramer, Joyce A; Amuan, Megan T; Cavazos, Jose E; Kazis, Lewis E (in press). Epilepsia.
Silveira, Maria J; Copeland, Laurel A; Feudtner, Chris (in press). American Journal of Public Health.
Hopp, Faith P; Woodbridge, Peter; Subramanian, Usha; Copeland, Laurel A; Smith, David; Lowery, Julie C (in press). Telemedicine and eHealth.
Zeber, John E; Copeland, Laurel A; Grazier, Kyle L (in press). Military Medicine.
Alexander, Jeffrey A; Copeland, Laurel A; Metzger, Maureen E (in press). Clinical Toxicology.
Wilkins EG, Lowery JC, Copeland LA, Goldfarb SL, Wren PA, Janz NK (in press). Medical Decision Making.
Is it scientifically valid to use
data for purposes for which it
was not originally collected?



Yes, because….
No, because…
Maybe…please use caution
Today and next week





Uses of secondary data
Common biases encountered with secondary
data
Methods of adjustment
Sources of secondary data
Possible effects of HIPAA on this type of
research
Different Uses

Health Care Delivery






Natural history




Quality assessment
Geographic variation; under-/over-utilization
Adverse events
Outcomes of a particular treatment
Cost
incidence, prevalence
prognosis
Association not causation
Other

Regulatory
Some Examples
To show you the range of uses
To show you the work is publishable
Regulatory enforcement








TENET HEALTHCARE - 2002
Medicare Report Jan-Oct 2004 included these findings:
– At Alvarado Hospital, prosecutors accused Tenet of running covert
kickback arrangements
– Document requests were received from federal prosecutors in L.A.
– False claims were uncovered and the company agreed to pay
$22.5M to settle allegations
– Redding Medical Center was determined to be performing
unnecessary cardiac surgery
– Tenet sold some holdings to Hospital Partners of America to
satisfy part of the 2003 settlement with federal officials
– Whistleblowers were to receive over $8M
Whose data set is it
anyway? Sharing raw data
from randomized trials
Andrew J Vickers
Trials. 2006 May 16;7:15
http://www.trialsjournal.com/content/pdf/1745-6215-7-15.pdf
Risk of death in elderly users
of conventional vs. atypical
antipsychotic medications
Wang PS, Schneeweiss S, Avorn J,
Fischer MA, Mogun H, Solomon DH,
Brookhart MA. N Engl J Med. 2005
Dec 1;353(22):2335-41
http://content.nejm.org/cgi/reprint/353/22/2335.pdf
Cancer incidence in Kentucky,
Pennsylvania, and West
Virginia: disparities in
Appalachia
Lengerich EJ, Tucker TC, Powell RK,
Colsher P, Lehman E, Ward AJ,
Siedlecki JC, Wyatt SW (2005). J Rural
Health 21(1):39-47.
http://www.blackwell-synergy.com/toc/jrh/21/1
Does Medicare coverage of
colonoscopy reduce racial/ethnic
disparities in cancer screening
among the elderly?
Shih YC, Zhao L, Elting LS (2006).
Health Aff (Millwood). Jul-Aug;
25(4):1153-62.
http://content.healthaffairs.org/cgi/content/full/25/4/1153
Can you think of examples of
secondary data analysis that
you have read in the medical
literature?
 Any
that have changed your
practice or your research
direction?
Methods of Learning




Lecture
In class epidemiologic exercise
Journal article evaluation – readings are
online (http://czresearch.com/dropbox/)
Optional brief proposal for your own
secondary data analysis
Exercise #1
Policy Decision on
Funding
Of Regional Trauma
Center (handout)
Secondary Data Exercise #1
A Regional Trauma Center (which encourages the surrounding hospitals
to refer patients with serious injuries to it for expert care) is seeking
additional funds from next year’s health budget for more equipment
and staff. A local politician (who would rather spend such money on
a new hospital named after his father) criticizes this request for
additional resources by claming that Regional Trauma Centers do
not, in fact, save lives, and he submits the following data to back up
his claim.
You are asked for your opinion on these data. How should you respond?
Severity of trauma
surrounding hospitals
regional trauma center
# of cases
# of deaths
# of cases
# of deaths
Total
7266
458
3354
212
Case-fatality rate
458/7266
= 6.3%
212/3354
= 6.3%
Secondary Data Exercise #1
A Regional Trauma Center (which encourages the surrounding hospitals to refer patients
with serious injuries to it for expert care) is seeking additional funds from next year’s
health budget for more equipment and staff. A local politician (who would rather
spend such money on a new hospital named after his father) criticizes this request for
additional resources by claming that Regional Trauma Centers do not, in fact, save
lives, and he submits the following data to back up his claim.
By examining the data stratified by severity, the value of the regional trauma
center becomes clear.
Severity of trauma
surrounding hospitals
regional trauma center
# of cases
# of deaths
# of cases
# of deaths
Mild
3734
37
687
3
Moderate
1887
94
1238
37
Severe
1645
327
1429
172
Total
7266
458
3354
212
Case-fatality rate
458/7266
= 6.3%
212/3354
= 6.3%
Bias

Systematic error in measurement or a
systematic difference (other than the one of
interest) between groups

Selection



For cohorts, assembly, migration, contamination, and
referral bias
Measurement
Confounding
Bias: Anticipate and Control







Restriction (may lose generalizability)
Matching
Stratification
Standardization
Multivariate Adjustment
Assuming the worst (sensitivity analyses)
Discussing possible effects on your results
Example: Hospital Mortality
Report Cards



Originally unadjusted
Hospitals without trauma centers, doing
primarily elective surgery, etc., looked really
good
Made hospitals who took care of the sickest
of the sick look bad
Quality Assessment

Data Quality: Garbage in, garbage out

Risk Adjustment: To remove the confounding
effect of different institutions providing care to
patients with dissimilar severity of illness and
case complexity
Interpret data carefully
Data Quality
•
•
•
If no reliable (accurate and adequate) data are
available, questions about risk adjustment are
moot
Inconsistent practices in assigning standard
ICD9 codes & names to diseases exist—but this
has improved over time (Do you know who
assigns the codes?) http://www.eicd.com/
Lack of specificity of ICD9 system in some
diagnostic areas, especially with regard to
severity
Administrative Data
Limitations




Need patient-specific identifiers to link episodes
Need multi-year data when outcomes are infrequent
Limited generalizability if restricted by type of
institution or hospital, by type of payor, or by
location/region
May lack important clinical variables known to be
related to outcomes (especially clinical tests or
qualitative evaluations of severity)
Risk Adjustment




Controls for those patient characteristics that
are related to the outcomes of interest
Removes the confounding effect of different
institutions providing care to patients with
dissimilar severity of illness and case
complexity
Addresses regional variations
Inadequate case-mix adjustment can lead to
misclassification of outlier status
Risk Adjustment




Primary data collection vs. administrative
data
Disease-specific vs. generic
Commercial vs. developed for your study
Predictors vary by outcomes being predicted
Essential Elements of Risk
Adjustment




Outcome-specific
Contains specification of the principal
diagnosis
Contains demographics as proxies for
preexisting physiological reserve
Measure count of comorbidities and all the
most important comorbidities to assume their
own empirically derived coefficients
Classification of Disease
States



ICD-9: too many specific codes (n~10,000)
Clinical Classifications for Health Policy
Research (CCHPR): good for chronic illness
and longitudinal care
[http://www.ahrq.gov/data/hcup/his.htm]
Primary diagnosis: good for studies that focus
on a single episode of care
Risk Adjustment: Charlson

Advantages:


Commonly used case-mix classification system in
the health care industry
System with which most clinicians and reviewers
are familiar
Risk Adjustment: Charlson

Disadvantages




Principal diagnosis not differentiated
Original work did not specify ICD-9 codes that
went into the disease categories
Developed on inpatients predicting mortality; may
not be well suited to outpatients at low risk of
death
Not good for longitudinal care / chronic illness
Demographic Factors to
Consider in Risk Adjustment


Age (e.g., age-adjusted Charlson)
Proxies of Social Support





Marital status
Race
Gender
SES (occupation, employment status,
education)
Proxies of Socioeconomic Status


Health insurance status
Home address zip code average income
Race and Gender


Don’t adjust for automatically
Ideally adjust for variation in the patients’
physiological reserve and disease burden but
not for variation in care rendered to patients
Propensity Scores





Useful when dataset is small, to conserve power
Need a good proxy to develop a propensity score
Ask: propensity for what? (Tx)
Published schema may include predictors you want
to study separately
Best for non-randomized studies of treatment effect
where you want to adjust for the factors that may
have influenced the treatment choices
Your Assignment


Read other two articles
If you want extra feedback, e-mail me a 1page proposal (abstract) for a database
analysis related to your area of interest any
time in next few years
copelandl@uthscsa.edu