Chapter 2
Personality Assessment,
Measurement, &
Research Design
Measurement
■ Personality psychology is historically
closely linked to psychometrics
• Correlation Coefficient
• Factor analysis
• Construct Validity
• Test construction
(1900)
(1930)
(1950)
(1960s)
Primary Measurement Issues
1. Where we get our information about
personality, i.e., our data sources.
2. How we evaluate our data quality.
3. How we use personality
measurements in data analysis
procedures to…
4. Address different research questions
with different research designs.
Data Sources: LOTS of data
L
O
T
S
Life outcome data
Observer reports
Test data
Self-reports
Sources of Personality Data
Self-Report
Info that a person reveals about
her/himself using, e.g.,….
Questionnaires
Interviews
Self-Reports
•
Unstructured “I am ____________”
EXAMPLE: “20 StatementsTest”
I am___________
I am___________
I am___________
Sentence Completion Blanks
Washington University Sentence Completion test of
Ego-Maturity (WUSCTED)
Jane Loevinger
1) At times she worried about ____________
2) The thing I like about myself is___________
3) I am_________________
•
Content coding procedure (requires training)
Structured tests
I am devious.
T F
To be famous.
123456789
Various response formats:
Check list
True-False
Likert
“Likert Scale”
I am kindhearted.
To be famous.
(Likert, 1932)
1 2 3 4 5
123456789
Self-Reports
Experience Sampling:
-
Beepers, cellphones, daily diary
McClelland (1980)
•
•
•
•
Affiliation study
Compared TAT vs Questionnaire
Outcome variable = Freq of affiliation
Result: Both TAT & Ques. predicted
Self-Reports
Strength: Individuals have access to a
rich amount of info. on themselves
Weakness: RESPONSE BIAS.
Impression management
Self-deception
Del Paulhus (UBC)
Socially
Desirable
Responding
SDR
Paulhus (1984; 2008)
■ SDR is not one thing, but two:
■ 2 dimensions of social value:
agentic, communal
Agentic bias
• Look competent, powerful, superior
Communal (“Moralistic”) bias
• Look kind, trustworthy, loyal
Agency and Communion
Paulhus (2008) model of social
desirability response bias
SDR
IMPRESSION
MANAGEMENT
Agentic
image
Communal
image
SELF DECEPTION
Asset
Exaggeration
Deviance
Of Denial
Acquiescence
 Yea-saying and Naysaying
Agreeing regardless of content
Disagreeing regardless of content
 Solution: Balance wording direction
 Controversial because reverse-keyed items
sometimes have weaker validity than affirmative
worded items.
Sources of Personality Data
■ Observer Report Data
• Unique access to some data
• Data aggregation across observers
Cancel idiosyncratic biases
Improve reliability & validity
Reiman et al. (1997)
Effect of Aggregation Across Observers
on Heritability Estimates
Reiman et al. (1997)
Effect of Aggregation Across Observers
on Heritability Estimates
Sources of Personality Data
■ Observer Report Data
• Types of observers:
• Professional (eg IPSR)
+trained to assess reliably and
validly
• Intimate observers (eg roommate)
+access to natural behavior
+multiple vantage points (spouse,
parents)
Sources of Personality Data
Observer Report Data
Naturalistic vs. Artificial Observation
 Immediate vs. Retrospective Observation
 Molar vs. Molecular Units of Observation.

Sources of Personality Data
Test Data

Participants are placed in a
standardized testing situation
Procedures are designed to elicit
behaviour difficult to observe in
everyday life .

Example: Megargee (1969)
• Dominance, gender & leadership
• Pre-tested M and F on dominance
• Groups=MM, FF, MF, FM (H/L dom)
• Contrived leadership task
"fix the box " task
"assign your own leader"...result:
• MM: Leader was Dom (75% of time)
• MF: Leader was M (80% of time)
MF: Dom F chose leader! ( chose M)
Sources of Personality Data
Test Data
Mechanical Recording Devices
Buss et al. (1980) “Actometer” study
1) Do measures converge? (ratings, actometer data)
2) Is activity-level stable over time?
3) Does actometer predict psychological functioning?
Sources of Personality Data
Test Data
Physiological Data…
Functional MRI
Positron Emission Tomography
Neuroscience of extraversion
■ Johnson et al. (1999)
• PET
• 9 low (I), 9 high (E)
• resting state only
■ RESULTS
•
Thalamus
• Insula
• Broca’s Area
I: ↑ Anterior, E: ↑ posterior
I: ↑ Anterior, E: ↑ posterior
I > E ( talking to yourself !!)
Neuroscience of spiritual feelings
Borg et al. (1999)
Neuroscience of spiritual feelings
Borg et al. (1999)
• Measured “self-transcendence”
• Measured serotonin receptor density
Conclusions
• Weak serotonin binding
• Weak gating of sensory stimuli [?]
Sources of Personality Data
Test Data
Mechanical Recording Devices
Physiological Data
Projective Tests
Rorshchach
Rorschach
Thematic Apperception Test (TAT)
ap-perception (away from + perception)
"assimilation of perception by prior
knowledge"
• Created 1930s by Christiana Morgan
and Henry Murray
• "Tell a story"
• Analyzed for motivational themes
(n=need): nAchieve, nPower,
nAffiliation, n-Intimacy
2000
Rorschach? -No.
TAT? -Yes.
Sources of Personality Data
Life-Outcome Data
Information that can be gleaned
from the events, activities, and
outcomes in a person’s life .
Motor Vehicle Accident Death
rate per 10,000
Bad-tempered Boys Outcomes
■ Caspi, Elder, Bem (1989)
Shy Boys' Outcomes
■ Caspi, Elder, Bem (1989)
Evaluating Measure Quality
Reliability
How trustworthy is the score?
Test-Retest Reliability
Split Half Reliability
Alternate Forms
Internal Consistency
Coefficient Alpha
Why so many similar statements?
■ EPQ Extraversion Scale
• 30 statements!
■ Answer: improve reliability
Aggregation
Reduce error variance
Increase Signal/Noise ratio
Effect of aggregation on Reliability
Evaluating Data Quality
Reliability = trustworthiness
• Say I use Shoe Size to measure IQ…
• Is it a reliable test?
• Yes. Measurements of Shoe Size are
very stable across time and situations.
• Is it a VALID test? … Nope.
Validity = meaning of scores.
• Do the scores truly measure the thing
they are supposed to be measuring?
Types of Validity
Face Validity
Face validity = Whether the item content in the measure appears to be
logically relevant to the underlying concept being measured. This is not a
critical form of validity for a measure, but in some circumstances can be
important.
Content Validity
Content validity = whether the content is fully representative of all aspects of the construct
being measure. The paranormal beliefs scale above has items to cover all of the primary
content distinctions that had been proposed for the concept of “paranormal belief”.
■ Convergent Validity
• Proof that the scores ARE associated with
things they SHOULD be associated with.
■ Discriminant Validity
• Proof that the scores are NOT associated with
things they should NOT be associated with.
Above are correlations between paranormal beliefs and various other traits.
■ Criterion Validity
• Does the measure predict something that can be considered a
theoretical criterion for the concept--something it should
predict if the scores were measuring what they are supposed to
be measuring?
This example here used
“known groups” as criteria:
If the paranormal beliefs
measure is valid, the groups
here should differ in an
expected pattern: Relg.
fundamentalists should be
lowest and “spiritualists”
should be highest in paranormal beliefs.
■ Factorial Validity
• Do the items all hang together--do they
intercorrelate with each other in the expected
manner? In other words, do the items show
good “structure”?
• Example:
The example on the next page shows results from a factor
analysis. The numbers are correlations with two “underlying factors”. All
the paranormal items correlate with the same factor, (except P07). All the
fundamentalism items correlate with the other factor. These are good
results. They suggest the paranormal scale items all hang together as they
should, and the fundamentalism items all hang together as they should.
Factorial Validity of a Paranormal Beliefs Scale
Fac1 Fac2
Summary of Types of Validity
Research Designs
3 Primary Types
1) Case Study
2) Correlational
3) Experimental
Research Designs
Case Studies
•
SINGLE PERSON
+ More detail, more depth
+ Excellent for developing theory
- Bad for testing theory
Influential Case Studies
• Phineus Gage (1860)
• Anna O (Freud & Breuer, 1895)
• HM (Henry Molaison, 1953)
• Mask of Sanity (Cleckley, 1941)
Dodge Morgan
●
Morgan sailed alone
around the world.
●
Team of trait
researchers did case
study of Morgan.
●
Numerous personality
assessments done.
●
Different measurement
approaches were pitted
against each other.
●
Whole issue of Journal
of Personality (1997)
devoted to Morgan.
Dodge Morgan Study by Nasby & Reid (1997)
1) Interpersonal circumplex (Wiggins)
2) Five factor Model (Costa & McCrae)
3) Life Story narratives (D. McAdams)
4) MMPI, Clinical Assessment (J.Butcher)
Each perspective provided something
unique in understanding Dodge Morgan.
Research Designs
■ Correlational: Quantify
covariation between things.
Correlation coefficient:
r
– Magnitude & direction of assoc.
– (Usually) cannot infer causation
from correlation.
The Correlation Coefficient:
• Numerical index of covariation
• Ranges from: -1.0 ..to.. +1.0
• “.00” = no association.
Q: Which correlation below is the
strongest association in the list?
A) -.59
B) +.50
C) -.09
D) +.34
Guess the correlation !
r
Height and Weight
Extrav & Happiness
Consc and GPA
Consc and Creativity
Aspirin and Heart attack risk
Neuroticism and Happiness
+ .70
+ .50
+ .30
- .20
- .08
- .50
3 Common Correlational Relationships:
Additive, Mediator, Moderator
Additive
r =.45
Self-Control
Well-being
Mindfulness
r =.30
Mindfulness
Mediation
r =.60
Self-Control
r =.45
r = .00
Well-being
Moderation (interaction)
Maltreatment
in childhood
r =.42
MAO
Antisocial
Fundamentalist
Background
Spiritual
Belief
?
Paranormal
Belief
Religious Fundamentalism
Low
High
Spirituality
r = .00
r = .50
Paranormal Belief
Multiple Correlation & Regression
Personality of Video Gamers
Causal Modeling
Factor Analysis
“Seeing the forests instead of the trees”
• Finding broad patterns of covariation
among many variables
• Partitioning covariation among many
measures into independent sets
• Shrinking a large number of smaller
variables into a small number of larger
variables.
Factor a number:
15 = 3 x 5
Factor a matrix of numbers:
Intercorrelation matrix
Intercorrelation matrix
2 product matrices
Intercorrelation matrix
e.g., We factor analyze a correlation matrix.
o Yields 2 product matrices
o Eigenvalues, and eigenvectors.
o Eigenvectors = “factor loadings”
Intercorrelation matrix
Intercorrelation matrix
2 product matrices
Loadings
Body Mental
Mass Ability
Factor I Factor II
Factor Analysis
■ Many uses
1. Discovering trait dimensions
Perceptual sensitivity? Curiosity?
Fantasy-proneness? Liberal Values?
OPENNESS TO EXPERIENCE
2. Constructing taxonomies
(e.g., Big Five)
3. Data reduction
e.g., Aggregating predictors for regression analyses
4. Test construction
e.g., Selecting questionnaire items
Factor loading
Factor Analysis
5. Evaluating the
factor structure
of a test
“Factorial
validity”
Experimental Methods
■ Typical experiment
• Measure a trait
• Select Hi vs. Lo on the trait
• Put subjects in controlled situation
• Manipulate some variable(s)
• Measure some outcome(s)
■ Example…
Experimental Methods: Example
Howarth & Eysenck (1988)
• How does arousal influence memory?
• Depends if STerm or LTerm memory:
“Action Decrement theory”
Example: strong emotion
• Short-term memory: worsens
• Long-term memory: improves
Howarth & Eysenck (1988)
• High extraversion = LOW brain arousal
• Low extraversion = HIGH brain arousal
If true…
E >I
I >E
short-term recall
long-term recall
# Words recalled
Howarth & Eysenck (1988)
Delay Interval 
“Significance”
Need to distinguish:
• Statistical significance
What is the statistical probability that
the result was due to random error?
• Practical significance
Does a result of that magnitude (r =
.09) have any real world importance?
Personality of Video Gamers
Effect Size
■ Measures of effect size
1. Magnitude of experimental effect
Cohen’s d = (Group1 – Group2) / SD
2. Correlations
Correlations are effect sizes.
Can convert d to correlation.
Effect Size
■ Useful benchmarks:
Large
40+
Medium 25 - 40
Small
15 - 25
■ Example:
• Milgram (1962) Obedience Study…
r = .40
Effect Size
■ Milgram (1962)
• effect size was equivalent to r = .40
• This is one of the most well-known
studies in social psychology.
• Trait measures often predict important
life outcomes > .40
■ Traits are indeed real and
important.
Some Effect Sizes in Medicine:
Effect sizes in personality
psychology are not abysmal.
A growing concern is how replicable
important research findings are.
Now great interest in systematic
replication research in psychology.
SUMMARY
• Data sources:
LOTS
• Data quality:
Reliability and Validity
• Research designs:
Case study
Correlational
Experiment