The good, the bad, and the ugly
of multiple data sources
Barbara J. McMorris, PhD
ONRS Seminar
May 19, 2011
UMN School of Nursing
Types of Triangulation
Methodological
Analysis
Interdisciplinary
Investigator
Data
Why is Data Triangulation
Important?
Better research study & results
Reviewers getting savvier;
asking for multiple sources of data
More “tools” for your
research tool box
Study 1: Administrative Claims
Data + Self Reports
Rhinitis Study Goals
Assess burden of allergic and non allergic
rhinitis
– What is the cost and utilization impact of
treatment for rhinitis?
Evaluate providers’ perceptions of rhinitis
burden on patients and severity of
symptoms
Rhinitis Survey Data Analysis
AR & NAR not
treated/managed
the same
59
40
Allergy Specialist
Patients' sympt
severity extrmly
impt
75
63
Rhinitis Tx
improves QOL
No Allergy
Specialist
79
63
0
50
100
% Agree/Strongly Agree
Rhinitis Claims Data Analysis
Count of and Costs of Health
Care Services Utilized
during 1 Yr Follow-up
Mean (SD)
Mild
Patients
(N = 5,670)
Moderate
Patients
(N = 22,216)
Number of Rhinitis-Related
ER Visits
0.03 (0.75)
0.05 (0.89)
0.04 (0.69)
0.05 (0.83)
Number of Rhinitis-Related
Office Visits
5.61 (10.1)
8.12( 11.69)
9.39 (12.85)
7.98 (11.75)
Number of Rhinitis-Related
Specialist Office Visits
3.08 (8.50)
5.65 (10.43)
7.22 (11.79)
5.56 (10.52)
Total Rhinitis-Related
Medical Cost
$358 (725)
$514 (919)
$556 (952)
$497 (900)
Total Rhinitis-Related
Pharmacy Costs
$364 (417)
$435 (474)
$440 (478)
$424 (467)
Severe
Patients
(N = 7,217)
Total
(N=35,103)
Reflection: “good-bad”
Known sampling frame
“Clean” processed data
Timing of data collection different
– Lag in health care claims being loaded
– Physical report of severity
Low response rate
Results used in marketing
http://www.gsk.com/media/pressreleases/2
007/2007_10_19_GSK1134.htm
Study 2: Self Reports +
Provider Care Flow Sheets
RA Infusion Study Goals
Describe patient-reported outcomes due to
bDMARD infusions:
– satisfaction
– lost productivity
– pain/discomfort
Quantify the amount of time providers spend for
initiating infusions (3 types)
Describe provider satisfaction with infusions
Data Sources
Health care claims database
Provider survey
Patient Survey
Case Report Form
Infusion Care Flow Sheet
RA Infusion Results
Patient and Provider time
associated with infusions
Means & %’s
Infusion 1
(N = 102)
Infusion 2
(N = 74)
Infusion 3
(N = 29)
Total patient time away from
work/home (min.)
172.6
267.9
446.9
% patients able to return to work
45%
17%
29%
% had friend/family member miss
work
31%
24%
43%
Total provider labor time (min.)
86.7
128.2
237.9
Reflection: “good”
Good example of triangulation
Comprehensive picture of “burden”
Small sample size
Results used in marketing
http://www.orencia.com/rheumatoidarthritis-medication/orencia-infusion.aspx
Study 3: Self Reports +
Process Data + Official Records
Broadway HS Study Goals
Provide a descriptive profile of pregnant
and parenting teen girls
Compare outcomes for Broadway and
comparison girls:
 progress toward graduation
 delaying a repeat birth
 keeping children fully immunized
Data Sources
Girls’ self reports on pre/post surveys
Case manager reports of time
Official Records
–Program
–School district
–County
Evaluation Results
%
County Data
Broadway
Students
(N = 158)
Comparison
Group
(N = 97)
Ave counted months of MFIP
9.7
10.76
Ave used months of MFIP
25.8
26.9
2 or more children
27%
27%
Used emergency assistance
47%
41%
Case closed due to sanction limit
9%
13%
MFIP and Other Assistance
Reflection: kinda “ugly”
Using survey created/revised by others
Linking datasets – what is the link?
Consent to access official records?
What were the data collected for?
– Who entered the data?
Budget enough time & money for both
data management & analysis?
Final Cautions about
Data Triangulation
Need well-defined goals (no “fishing”)
Costly both in time and $$$
May not solve bias/error issues
Replication is difficult
Know your “official” records sources and
their limitations
Do you have “permission” to link data
sets?
“triangulation state of mind”*
“If you self consciously set out to collect
and double-check findings using multiple
sources and modes of evidence, the
verification process will largely be built into
the data-gathering process, and little more
need be done than to report on one’s
procedures.”
*Miles & Huberman (1984: p.235)
Thank you!
Useful citations:
Begley, CM. (1996). Using triangulation in
nursing research. Journal of Advanced
Nursing 24:122-128.
Thurmond, VA. (2001). The Point of
Triangulation. Journal of Nursing
Scholarship 22(3): 253-258.