Construct Validity
And its Threats
“Boy, a few more like that and I’ll be ready for Gamblers Anonymous”
Jill Hoxmeier
H615: Advanced Research Design
October 10, 2013
Twin Problems of Construct
Validity
Construct Validity: making inferences from the sampling
particulars of a study to the higher-order constructs they
represent
• Understanding them
• Assessing them
“Thinking without the positing of categories and concepts in
general would be as impossible as breathing in a vacuum”
(Einstein, 1949)
Meehl & Cronbach
Construct Validity in
Psychological Tests
• Three steps to evaluate construct validity:
• Articulation of a set of theoretical constructs and their
interrelations
• Developing ways to measure the hypothetical constructs
proposed by a theory
• Empirically testing the hypothesized relations
• Evolution of “Attitude”
• From a loose variable comprised of beliefs, norms, intentions…
• … to Fishbein-ian perspective that attitude = person’s evaluation
of behavioral consequence x strength of belief that behavior will
lead to those consequences
Cronbach & Meehl, 1955
Why do we need valid constructs?
• Research cannot be done without constructs
• Constructs are the central means we have for connecting the
operations used in an experiment to theory
• Construct labels carry social, political, and economic
implications
• The creation and defense of basic constructs is a fundamental
task of all science
Construct Validity
• Fostered by:
• Starting with a clear explication of the person, setting, treatment,
and outcome constructs of interest
• Carefully selecting instances that match those constructs
• Assessing the match between instances and constructs for
“slippage”
• Revising construct descriptions accordingly
Understanding Psychological
Domains
“…there are often no natural units of measurement…”
(Shonkoff & Philllips, 2000, p.83-83)
• Pattern-matching logic: does the given
instance sufficiently match the prototypical
features to warrant using the category label?
How do you measure your
domain?
• Think about your own research interest…
• What are the domains or constructs that are measured?
• Have you discovered from the literature any challenges in
measuring those domains?
• What are ways investigators in your field have revised constructs
to overcome those challenges?
Program of Research
• A theory of constructs must emphasize
• 1) operationalizing each construct several ways within
and across studies
• 2) probing the pattern match between the multivariate
characteristics of instances and characteristics of the
target construct
• 3) acknowledge legitimate debate about the quality of
the match given the socially constructed nature of
both operations and constructs
Threats to Construct Validity
•
•
•
•
•
•
•
•
•
•
•
•
•
Inadequate explication of constructs
Construct confounding
Mono-operation bias
Mono-method bias
Confounding constructs with levels of constructs
Treatment sensitive factorial structure
Reactive self-report changes
Reactivity to the experimental situation
Experimenter expectancies
Novelty and disruptive effects
Compensatory rivalry
Resentful demoralization
Treatment diffusion
What (potential) threats are most relevant to your research?
And how do you plan to guard against such threats?
Questions for You!
• What role does qualitative research play in contributing to
construct validity?
• How do we balance the theory construct testing of
quantitative research and the “naturalistic generalizations”
that may come from readers of “thick descriptions” that
qualitative researchers provide?