Why the Items versus Parcels
Controversy Needn’t Be One
Todd D. Little
University of Kansas
Director, Quantitative Training Program
Director, Center for Research Methods and Data Analysis
Director, Undergraduate Social and Behavioral Sciences Methodology Minor
Member, Developmental Psychology Training Program
crmda.KU.edu
Presidential Address presented 08-04-2011 @
American Psychological Association Meeting in Washington, DC
crmda.KU.edu
1
Why I Think the Items versus Parcels
Controversy Needn’t Be One
Todd D. Little
University of Kansas
Director, Quantitative Training Program
Director, Center for Research Methods and Data Analysis
Director, Undergraduate Social and Behavioral Sciences Methodology Minor
Member, Developmental Psychology Training Program
crmda.KU.edu
Presidential Address presented 08-04-2011 @
American Psychological Association Meeting in Washington, DC
crmda.KU.edu
2
Overview
• Learn what parcels are and how to make them
• Learn the reasons for and the conditions under
which parcels are beneficial
• Learn the conditions under which parcels can be
problematic
• Disclaimer: This talk reflects my view that
parcels per se aren’t controversial if done
thoughtfully.
crmda.KU.edu
3
What is Parceling?
• Parceling: Averaging (or summing) two or more
items to create more reliable indicators of a construct
• ≈ Packaging items, tying them together
• Data pre-processing strategy
crmda.KU.edu
4
A CFA of Items
-.29
1*
Positive
1
.83
Great
Cheer
Un
Happy Good Glad Super Sad Down
Blue
ful
happy
.31
.34
.84
.30
.77
.84
.41
.71
.30
.50
.82
.32
.82
1*
.76
.43
.81
Negative
2
.34
.81
.35
.69
.80
Bad
Terr
ible
.52
.35
Model Fit: χ2(53, n=759) = 181.2; RMSEA = .056(.048-.066); NNFI/TLI = .97; CFI = .98
crmda.KU.edu
5
CFA: Using Parcels
(6.2.Parcels)
y21
y11
1*
11
Great
& Glad
q11
Positive
1
21
Negative
2
31
42
52
1*
y22
62
Cheerful
& Good
Happy
& Super
Terrible
& Sad
Down
& Blue
Unhappy
& Bad
q22
q33
q44
q55
q66
Average 2 items to create 3 parcels per construct
crmda.KU.edu
6
CFA: Using Parcels
Similar solution
-.27
• Similar factor correlation
• Higher loadings,
Positive
more reliable info
1*
1
• Good model fit,
improved χ2
.89
Great
& Glad
.21
.89
Negative
2
.91
.87
.87
1*
.91
Cheerful
& Good
Happy
& Super
Terrible
& Sad
Down
& Blue
Unhappy
& Bad
.21
.17
.25
.25
.18
Model Fit: χ2(8, n=759) = 26.8; RMSEA = .056(.033-.079); NNFI/TLI = .99; CFI = .99
crmda.KU.edu
7
Philosophical Issues
To parcel, or not to parcel…?
crmda.KU.edu
8
Pragmatic View
“Given that measurement is a strict, rulebound system that is defined, followed, and
reported by the investigator, the level of
aggregation used to represent the
measurement process is a matter of choice and
justification on the part of the investigator”
Preferred terms: remove unwanted, clean,
reduce, minimize, strengthen, etc.
From Little et al., 2002
crmda.KU.edu
9
Empiricist / Conservative View
“Parceling is akin to cheating because
modeled data should be as close to the
response of the individual as possible in order
to avoid the potential imposition, or arbitrary
manufacturing of a false structure”
Preferred terms: mask, conceal, camouflage,
hide, disguise, cover-up, etc.
From Little et al., 2002
crmda.KU.edu
10
Hobbesian View
Parcels should be avoided because
researchers are ignorant (perhaps stupid) and
prone to mistakes. And, because the
unthoughtful or unaware application of
parcels by unwitting researchers can lead to
bias, they should be avoided.
Preferred terms: most (all) researchers are
un___ as in … unaware, unable, unwitting,
uninformed, unscrupulous, etc.
crmda.KU.edu
11
Other Issues I
• Classical school vs. Modeling School
– Objectivity versus Transparency
– Items vs. Indicators
– Factors vs. Constructs
• Self-correcting nature of science
• Suboptimal simulations
– Don’t include population misfit
– Emphasize the ‘straw conditions’ and
proofing the obvious; sometimes over
generalize
crmda.KU.edu
12
Other Issues II
• Focus of inquiry
– Question about the items/scale
development?
• Avoid parcels
– Question about the constructs?
• Parcels are warranted but must be done
thoughtfully!
– Question about factorial invariance?
• Parcels are OK if done thoughtfully.
crmda.KU.edu
13
Measurement
“Whatever exists at all exists in some amount. To know it
thoroughly involves knowing its quantity as well as its quality”
- E. L. Thorndike (1918)
• Measurement starts with Operationalization
Defining a concept with specific observable characteristics
[Hitting and kicking ~ operational definition of Overt Aggression]
• Process of linking constructs to their manifest indicants
(object/event that can be seen, touched, or otherwise recorded; cf. items vs. indicators)
• Rule-bound assignment of numbers to the indicants of that
which exists [e,g., Never=1, Seldom=2, Often=3, Always=4]
• … although convention often ‘rules’, the rules should
be chosen and defined by the investigator
crmda.KU.edu
14
“Indicators are our worldly window into the
latent space”
- John R. Nesselroade
crmda.KU.edu
15
Classical Variants
a) Xi = Ti + Si + ei
b) X = T + S + e
c) X1 = T1 + S1 + e1
Xi : a person’s observed score on an item
Ti : 'true' score (i.e., what we hope to measure)
Si : item-specific, yet reliable, component
ei : random error or noise.
Assume:
• Si and ei are normally distributed (with mean of zero) and uncorrelated
with each other
• Across all items in a domain, the Sis are uncorrelated with each other, as
are the eis
crmda.KU.edu
16
Latent Variable Variants
a) X1 = T + S1 + e1
b) X2 = T + S2 + e2
c) X3 = T + S3 + e3
X1-X3 : are multiple indicators of the same construct
T : common 'true' score across indicators
S1-S3 : item-specific, yet reliable, component
e1-e3 : random error or noise.
Assume:
• Ss and es are normally distributed (with mean of zero) and uncorrelated
with each other
• Across all items in a domain, the Ss are uncorrelated with each other, as
are the es
crmda.KU.edu
17
Empirical Pros
Psychometric Characteristics of Parcels
(vs. Items)
• Higher reliability, communality, & ratio of common•
•
to-unique factor variance
Lower likelihood of distributional violations
More, Smaller, and more-equal intervals
Never
Seldom
Often
Always
Happy
Glad
1
1
2
2
3
3
4
4
Mean
Sum
1
2
1.5
3
2
4
2.5
5
crmda.KU.edu
3
6
3.5
7
4
8
18
More Empirical Pros
Model Estimation and Fit with Parcels
(vs. Items)
• Fewer parameter estimates
• Lower indicator-to-subject ratio
• Reduces sources of parsimony error
•
•
(population misfit of a model)
 Lower likelihood of correlated residuals &
dual factor loading
Reduces sources of sampling error
Makes large models tractable/estimable
crmda.KU.edu
19
More Empirical Pros
Variance of (𝑿𝟏 + 𝑿𝟐 ) / 2
𝟏
= 𝐯𝐚𝐫[ 𝑻 + 𝑺𝟏 + 𝑬𝟏 + 𝑻 + 𝑺𝟐 + 𝑬𝟐 ]
𝟐
=
=
=
𝟏
𝐜𝐨𝐯[𝟐𝑻 + 𝑺𝟏 + 𝑺𝟐 + 𝑬𝟏 +
𝟒
𝟏
[𝟒𝐯𝐚𝐫 𝑻 + 𝐯𝐚𝐫 𝑺𝟏 + 𝐯𝐚𝐫
𝟒
𝟏
𝟏
𝐯𝐚𝐫 𝑻 + 𝐯𝐚𝐫 𝑺𝟏 + 𝐯𝐚𝐫
𝟒
𝟒
Construct
T
Specific
S1
Error
E1
𝑬𝟐 , 𝟐𝑻 + 𝑺𝟏 + 𝑺𝟐 + 𝑬𝟏 + 𝑬𝟐 ]
𝑺𝟐 + 𝐯𝐚𝐫(𝑬𝟏 ) + 𝐯𝐚𝐫 𝑬𝟐 ]
𝑺𝟐 +
T
+
2
𝟏
𝐯𝐚𝐫
𝟒
𝑬𝟏 +
𝟏
𝐯𝐚𝐫(𝑬𝟐 )
𝟒
T
S2
E2
crmda.KU.edu
¼ of their
original size!
20
Construct = Common Variance of Indicators
crmda.KU.edu
21
Construct = Common Variance of Indicators
crmda.KU.edu
22
Empirical Cautions
Construct
Specific
Error
T
T
M
M
S1
E1
+
T
S2
E2
¼ of their
original size!
2
crmda.KU.edu
23
Construct = Common Variance of Indicators
crmda.KU.edu
24
Construct = Common Variance of Indicators
crmda.KU.edu
25
Three is Ideal
y21
y11
1*
11
Great
& Glad
q11
Positive
1
21
Negative
2
31
42
52
1*
y22
62
Cheerful
& Good
Happy
& Super
Terrible
& Sad
Down
& Blue
Unhappy
& Bad
q22
q33
q44
q55
q66
crmda.KU.edu
26
Three is Ideal
Matrix Algebra Formula: Σ = Λ Ψ Λ´ + Θ
Great
& Glad
P1
P2
P3
N1
N2
N3
Cheerful
& Good
Happy
& Super
Terrible
& Sad
Down
& Blue
Unhappy
& Bad
11y11 11 +q11
11y1121
21y11  21+q22
11y1131
21y1131
31y11  31+q33
11y2142
21y2142
31y2142
42y22 42 +q44
11y2152
21y2152
31y2152
42y2252
52y22 52 +q55
11y2162
21y2162
31y2162
42y2262
52y2262
62y22 62+q66
• Cross-construct item associations (in box) estimated only via Ψ21
•
– the latent constructs’ correlation.
Degrees of freedom only arise from between construct relations
crmda.KU.edu
27
Empirical Cons
•
Multidimensionality
• Constructs and relationships can be hard to
interpret if done improperly
•
Model misspecification
•
•
•
Can get improved model fit, regardless of
whether model is correctly specified
Increased Type II error rate if question is about
the items
Parcel-allocation variability
•
Solutions depend on the parcel allocation
combination (Sterba & McCallum, 2010; Sterba, in press)
crmda.KU.edu
28
Psychometric Issues
•
Principles of Aggregation (e.g., Rushton et al.)
• Any one item is less representative than the
average of many items (selection rationale)
• Aggregating items yields greater precision
•
Law of Large Numbers
•
•
More is better, yielding more precise estimates of
parameters (and a person’s true score)
Normalizing tendency
crmda.KU.edu
29
Construct Space with Centroid
crmda.KU.edu
30
Potential Indicators of the
Construct
crmda.KU.edu
31
Selecting Six (Three Pairs)
crmda.KU.edu
32
… take the mean
crmda.KU.edu
33
… and find the centroid
crmda.KU.edu
34
Selecting Six (Three Pairs)
crmda.KU.edu
35
… take the mean
crmda.KU.edu
36
… find the centroid
crmda.KU.edu
37
How about 3 sets of 3?
crmda.KU.edu
38
… taking the means
crmda.KU.edu
39
… yields more reliable & accurate
indicators
crmda.KU.edu
40
Building Parcels
• Theory – Know thy S and the nature of your items.
• Random assignment of items to parcels (e.g., fMRI)
• Use Sterba’s calculator to find an allocation
method that is in the middle of the distribution.
• Balancing technique
• Combine items with higher loadings with items
having smaller loadings [Reverse serpentine
pattern]
• Using a priori designs (e.g., CAMI)
• Develop new tests or measures with parcels as the
goal for use in research
crmda.KU.edu
41
Techniques: Multidimensional Case
Example: ‘Intelligence’ ~ Spatial, Verbal, Numerical
• Domain Representative Parcels
• Has mixed item content from various dimensions
• Parcel consists of: 1 Spatial item, 1 Verbal item,
and 1 Numerical item
• Facet Representative Parcels
• Internally consistent, each parcel is a ‘facet’ or
singular dimension of the construct
• Parcel consists of: 3 Spatial items
• Recommended method
crmda.KU.edu
42
Domain Representative Parcels
+
V
S
S
S
Spatial
+
N
=
Parcel #1
+
V
+
N
=
Parcel #2
+
V
+
N
=
Parcel #3
Verbal
Numerical
crmda.KU.edu
43
Domain Representative
crmda.KU.edu
44
Domain Representative
Intellective Ability,
Spatial Ability,
Verbal Ability,
Numerical Ability
But which facet is driving the correlation among constructs?
crmda.KU.edu
45
Facet Representative Parcels
+
S
V
N
S
+
S
=
Parcel:
Spatial
+
V
+
V
=
Parcel:
Verbal
+
N
+
N
=
Parcel:
Numerical
crmda.KU.edu
46
Facet Representative
crmda.KU.edu
47
Facet Representative
Intellective Ability
Diagram depicts smaller communalities (amount of shared variance)
crmda.KU.edu
48
Facet Representative Parcels
+
+
=
+
+
=
+
+
=
A more realistic case with higher communalities
crmda.KU.edu
49
Facet Representative
crmda.KU.edu
50
Facet Representative
Intellective Ability
Parcels have more reliable information
crmda.KU.edu
51
2nd Order Representation
Capture multiple
sources of variance?
crmda.KU.edu
52
2nd Order Representation
Variance can be
partitioned even
further
crmda.KU.edu
53
2nd Order Representation
Lower-order
constructs retain
facet-specific
variance
crmda.KU.edu
54
Functionally Equivalent Models
Explicit Higher-Order
Structure
Implicit Higher-Order
Structure
crmda.KU.edu
55
Thank You!
Todd D. Little
University of Kansas
Director, Quantitative Training Program
Director, Center for Research Methods and Data Analysis
Director, Undergraduate Social and Behavioral Sciences Methodology Minor
Member, Developmental Psychology Training Program
crmda.KU.edu
Presidential Address presented 08-04-2011 @
American Psychological Association Meeting in Washington, DC
crmda.KU.edu
56