Construct validity
http://www.socialresearchmethod
s.net/kb/consthre.htm
What is construct validity
• Construct validity refers to the degree to
which inferences can legitimately be made
from the operationalizations in your study
to the theoretical constructs on which
those operationalizations were based.
• Construct validity involves generalizing
from your program or measures to the
concept of your program or measures.
Threats to construct validity (1)
• Inadequate Preoperational Explication of
Constructs
– preoperational means before translating
constructs into measures or treatments, and
explication means explanation -- in other
words, you didn't do a good enough job of
defining (operationally) what you mean by the
construct
Threats to construct validity (2)
• Mono-Operation Bias
– Mono-operation bias pertains to the independent
variable, cause, program or treatment in your study -it does not pertain to measures or outcomes
• Mono-Method Bias
– Mono-method bias refers to your measures or
observations, not to your programs or causes
• Interaction of Different Treatments
– The result might be the combination of the separate
programs they participated in not only yours.
Threats to construct validity (3)
• Interaction of Testing and Treatment
– Does testing or measurement itself make the groups
more sensitive or receptive to the treatment? If it
does, then the testing is in effect a part of the
treatment, it's inseparable from the effect of the
treatment.
• Restricted Generalizability Across Constructs
– unintended consequences
• Confounding Constructs and Levels of Constructs
– Slight increases or decreases of the dosage may
radically change the results.
Threats to construct validity (4)
• The "Social" Threats to Construct Validity
– Hypothesis Guessing
• Participants are likely to base their behavior on what they guess
about the study, not just on your treatment.
– Evaluation Apprehension
• Many people are anxious about being evaluated. ( performed good
and poorly)
– Experimenter Expectancies
• Sometimes the researcher can communicate what the desired
outcome for a study might be (and participant desire to "look good"
leads them to react that way). For instance, the researcher might
look pleased when participants give a desired answer. If this is what
causes the response, it would be wrong to label the response as a
treatment effect.
External validity
• External validity refers to the approximate
truth of conclusions the involve
generalizations. Put in more pedestrian
terms, external validity is the degree to
which the conclusions in your study would
hold for other persons in other places and
at other times.
Threats to External Validity
• A threat to external
•
validity is an
explanation of how
you might be wrong
in making a
generalization
Time, place, people,
setting.
Improving External Validity
• based on the sampling model, suggests
that you do a good job of drawing a
sample from a population,
– such as random selection.
• use the theory of proximal similarity more
effectively.
– do your study in a variety of places, with
different people and at different times.
Conceptualizing and controlling for
threats to validity
• Selection bias  expand or purify the
experiment group.
• Generality of the findings  viewing
simple designs as portions of potentially
larger design.
• Once again, construct validity and external
validity  heteromethod replication.