Measurement in Psychology:
Validity
Lawrence R. Gordon
Psychology Research Methods I
You have a reliable measure…
 It gives you a consistent measure of
something every time that you measure it
 SO WHAT???
 Is that all a measure needs to do?
Validity
 NOTE: a scale CAN be reliable but not valid; it
CANNOT be unreliable scale, but be valid!
 In short, a measure has to measure what it claims
to measure (or what you claim it measures)
 How can you show others that you are measuring
what you say you are measuring?
 Example
SAT’s and College Potential
 How can we show that SAT’s are actually
measuring college potential?
Types of Validity
 Content/Face validity

does it look like it is measuring what it should?
• Examples (Polygraph measures lying through GSR,
EEG, heart rate? Reaction time measured with a
stopwatch?)

Does the content make sense in measuring the
construct?
 Content/Sampling Validity

Where did the content come from?
• Examples (Why do we need to know insane
vocabulary words and geometry on the SAT’s? Who
wrote those questions?)
Types of Validity
 Criterion validity


validity related to an outcome
concurrent validity (Does it relate to other
variables the way the theory says it should?)
• Example (Does the SAT relate to Achievement
Tests? IQ?)

predictive validity (Does it predict something
related to the construct?)
• Examples (Does the SAT predict first-year college
GPA? Does the SAT predict GRE scores? Does it
“postdict” HS class rank?)
Types of Validity
 Discriminant validity

Does the measure fail to relate to measures of
other constructs?
• Examples (Does the SAT relate to measures of SES
or self-esteem?)
Types of Validity
 Construct validity



Does the measure really measure the construct
it says it does?
Is the construct (and its operational definition)
appropriate for the research?
Examples (Does the SAT measure college
potential? Is this an appropriate measure for
college potential?)
Old-fashioned Racism
 Old-fashioned racism is the overt, blatant
expression of racist attitudes
 It’s easy to measure - ask questions like



“are you racist?”
“do you like African-Americans?”
“would you want African-Americans to attend
the same schools as your children?”
 scale has what type of validity?
Old-fashioned Racism (cont.)
 Problems with “old-fashioned racism”,


didn’t relate to subsequent voting behavior
(what type of validity lacking?)
people were not reporting that they were racist
anymore
• 80% of White Americans were reporting that they
were not racist at all (Dovidio & Gaertner, 1991)
 Where do we go from here?
Validation of a Measure
 Subtle measure of racism
 Use of political attitudes rather than direct
inquiry (not quite the same thing!)
 Modern Racism Scale (McConahay,
Hardee, & Batts, 1981)

How much do you agree that:
• “over the past few years, Blacks have gotten more
economically than they deserve”?
• “discrimination against Blacks is no longer a
problem in the United States”?
Validation of a Measure (cont.)
 MRS needed to show that it was reliable
and valid
 How could it show that it was reliable?
 How could it show that it was valid?
Modern
2002
ModernRacism
RacismScale,
Scale,Fall
F 2001
70
80
60
Frequency
Frequency
60
50
40
40
30
20
20
Std.
Std. Dev
Dev ==4.09
3.85
Mean
Mean ==4.1
3.5
10
N
N== 183.00
190.00
0
0
0.0
0.0
2.5
2.5
5.0
5.0
7.5
7.5
10.0
10.0
12.5
12.5 15.0
15.0 17.5
17.5 20.0
20.0
Modern
Modern Racism
Racism Scale
Scale (0-28)
(0-28)
Modern Racism Scale
INTERNAL CONSISTENCY RELIABILITY (Homogeneity)
RELIABILITY ANALYSIS - SCALE (ALPHA)
N of Cases =
183.0
Reliability Coefficients N of Items =
Alpha =
.7834
7
SPLIT-HALF RELIABILITY (Consistency)
RELIABILITY ANALYSIS - SCALE (SPLIT)
N of Cases =
183.0
Reliability Coefficients
N of Items =
Correlation between forms =
.5993
Unequal-length Spearman-Brown =
.7525
4 Items in part 1 / 3 Items in part 2
7
Validation of a Measure (cont.)
 Validity of MRS

Face validity?
• Survey says...
MRS FACE VALIDITY
Not Racism
Related
5%
Missing
24%
Racismrelated
71%
Validation of a Measure (cont.)
 Validity of MRS

Face validity?
• Survey says...

Criterion validity?
• Concurrent validity?
• Predictive validity?
– McConahay (1983) - tell me about it...
Validation of a Measure (cont.)
 Validity of MRS

Face validity?
• Survey says...

Criterion validity?
• Concurrent validity?
• Predictive validity?
– McConahay (1983) - tell me about it...

Discriminant validity?
DISCRIMINANT VALIDITY OF THE MRS
20
10
0
38
40
42
44
46
48
50
52
Social Desirability (Marlowe-Crown)
54
56
Validation of a Measure (cont.)
 Validity of MRS

Construct validity?
 Since its creation, the MRS has been
probably the most widely used measure of
racism in psychology
 What do you think about its continued
validity?
McConahay -- Major Result
Correlations Relating MRS to Willingness to Hire
by Race of candidate and Decision Context
0.8
0.6
0.4
0.2
r
0
-0.2
-0.4
-0.6
Negative
Positive
White
Black
Construct Validity -- Your Data
and Civil Unions
 How does the Attitudes to Civil Union scale do?





No reliability info, but similar scales work well
Face validity?
Concurrent validity? Correlation with MRS?
Discriminant validity? Correlation with Soc Desirability
CONSTRUCT
•
•
•
•
which sex more “tolerant”?
which political party?
which type of hometown?
Related to weight? Height? GPA?