Chapter 2
Personality Assessment,
Measurement, &
Research Design
Psychometrics & Personality
■ Individual differences psychology is
closely linked to psychometrics
• Correlation Coefficient
• Factor analysis
• Construct Validity
• Test construction
(1900)
(1930)
(1950)
(1960s)
Primary Measurement Issues
1. Where we get our information about
personality, i.e., our data sources.
2. How we evaluate our data quality.
3. How we use personality
measurements in data analysis
procedures to…
4. Address different research questions
with different research designs.
Data Sources: LOTS of data
L
O
T
S
Life outcome data
Observer reports
Test data
Self-reports
Sources of Personality Data
Self-Report
The information a person reveals about
her/himself in….
Questionnaires
Interviews
Sources of Personality Data
Self-Report
•
Unstructured “I am ____________”
EXAMPLE: “20 StatementsTest”
I am___________
I am___________
I am___________
Sentence Completion Blanks
Washington University Sentence Completion test of
Ego-Maturity (WUSCTED)
Jane Loevinger
1) At times she worried about ____________
2) The thing I like about myself is___________
3) I am_________________
•
Content coding procedure (requires training)
Structured tests
I am devious.
T F
To be famous.
123456789
Response formats:
Check list
True-False
Likert
“Likert Scale”
I am kindhearted.
To be famous.
(Likert, 1932)
1 2 3 4 5
123456789
Sources of Personality Data
Self-Report
• Diary
data
e.g., Experience Sampling
Sources of Personality Data
Self-Report

Strength: Individuals have access to a
wealth of information about themselves
 Weakness: Respondents must
 Impression management
 Self-deception
be honest
Del Paulhus (UBC)
Socially
Desirable
Responding
SDR
Paulhus (2008)
■ SDR comes in 2 flavors
■ They correspond to 2 kinds of
social value: agentic, communal
Agentic bias
• Look competent, powerful, superior
Communal bias
• Look kind, trustworthy, loyal
Agency and Communion
SDR
IMPRESSION
MANAGEMENT
Agentic
image
Communal
image
SELF DECEPTION
Asset
Exaggeration
Deviance
Of Denial
Sources of Personality Data
■ Observer Report Data
• Unique access to some data
• Data aggregation across observers
Cancel idiosyncratic biases
Improve reliability & validity
Reiman et al. (1997)
Effect of Aggregation Across Observers
on Heritability Estimates
Reiman et al. (1997)
Effect of Aggregation Across Observers
on Heritability Estimates
Sources of Personality Data
■ Observer Report Data
• Types of observers:
• Professional (eg IPSR)
+trained to assess reliably and
validly
• Intimate observers (eg roommate)
+access to natural behavior
+multiple vantage points (spouse,
parents)
Sources of Personality Data
Observer Report Data
Naturalistic vs. Artificial Observation
 Immediate vs. Retrospective Observation
 Molar vs. Molecular Units of Observation.

Sources of Personality Data
Test Data

Participants are placed in a
standardized testing situation
Procedures are designed to elicit
behaviour difficult to observe in
everyday life .

Example: Megargee (1969)
• Dominance, gender & leadership
• Pre-tested M and F on dominance
• Groups=MM, FF, MF, FM (H/L dom)
• Contrived leadership task
"fix the box " task
"assign your own leader"...result:
• MM/FF: 70-75% dom led
• MF/FM: 80-90% males led
Sources of Personality Data
Test Data
Mechanical Recording Devices
 Physiological Data

Brain Imaging
Functional MRI
Positron Emission Tomography
Neuroscience of extraversion
■ Johnson et al. (1999)
• PET
• 9 low (I), 9 high (E)
• resting state only
■ RESULTS
•
Thalamus
• Insula
• Broca’s Area
I: ↑ Anterior, E: ↑ posterior
I: ↑ Anterior, E: ↑ posterior
I > E ( talking to yourself !!)
Neuroscience of spiritual feelings
Borg et al. (1999)
Neuroscience of spiritual feelings
Borg et al. (1999)
• Measured “self-transcendence”
• Measured serotonin receptor density
Conclusions
• Weak serotonin binding
• Weak gating of sensory stimuli
Neuroscience of spiritual feelings
■ Newberg (1989): virtuoso medit.
• Deep meditation? ↘ parietal lobe
• Caused dissolved self boundaries
• Feelings of “oneness” with universe
Source of mystical/spiritual feelings?
Sources of Personality Data
Test Data
Mechanical Recording Devices
 Physiological Data
 Projective Tests

Rorshchach
Rorschach
Thematic Apperception Test (TAT)
ap-perception (away from+ perception)
"assimilation of perception by prior
knowledge"
• Created 1930s by Christiana Morgan
and Henry Murray
• "Tell a story"
• Analyzed for motivational themes
Need for Power, nAchiev,
nIntimacy
2000
Rorschach? -No.
TAT? -Yes.
Sources of Personality Data
Life-Outcome Data
Information that can be gleaned
from the events, activities, and
outcomes in a person’s life .
Motor Vehicle Accident Death
rate per 10,000
Bad-tempered Boys Outcomes
■ Caspi, Elder, Bem (1989)
Shy Boys' Outcomes
■ Caspi, Elder, Bem (1989)
Sources of Personality Data
Issues in Personality Assessment

Links among different data sources

Fallibility of personality measurement .
Evaluating Measure Quality
■ Reliability
■ Validity
■ Generalizability
Evaluating Measure Quality
Reliability
How trustworthy is the score?
Test-Retest Reliability
Split Half Reliability
Alternate Forms
Internal Consistency
Coefficient Alpha
Why so many similar statements?
■ EPQ Extraversion Scale
• 30 statements!
■ Answer: improve reliability
Aggregation
Reduce error variance
Increase Signal/Noise ratio
Effect of aggregation on Reliability
Evaluating Data Quality
■ Reliability
= stability of scores
• Say I use Shoe Size to measure IQ…
• Is it a reliable test?
• Yes. Measurements of Shoe Size are
very stable across time and situations.
• Is it a VALID test? … Nope.
■ Validity = meaning of scores.
• Do the scores truly measure the thing
they are supposed to be measuring?
Face Validity
Face validity = Whether the item content in the measure appears to be
logically relevant to the underlying concept being measured. This is not a
critical form of validity for a measure, but in some circumstances can be
important.
Content Validity
Content validity = whether the content is fully representative of all aspects of the construct
being measure. The paranormal beliefs scale above has items to cover all of the primary
content distinctions that had been proposed for the concept of “paranormal belief”.
■ Convergent Validity
• Proof that the scores ARE associated with
things they SHOULD be associated with.
■ Discriminant Validity
• Proof that the scores are NOT associated with
things they should NOT be associated with.
Above are correlations between paranormal beliefs and various other traits.
■ Criterion (or Predictive) Validity
• Does the measure predict something that can be considered a
theoretical criterion for the concept--something it should
predict if the scores were measuring what they are supposed to
be measuring?
This example used
“known groups” as criteria:
If the paranormal beliefs
measure is valid, the groups
here should differ in certain
ways on the scale: Relg.
fundamentalists should be
lowest and “spiritualists”
should be highest in paranormal beliefs.
■ Factorial Validity
• Do the items all hang together--do they
intercorrelate with each other in the expected
manner? In other words, do the items show
good “structure”?
• Example:
The example on the next page shows results from a factor
analysis. The numbers are correlations with two “underlying factors”. All
the paranormal items correlate with the same factor, (except P07). All the
fundamentalism items correlate with the other factor. These are good
results. They suggest the paranormal scale items all hang together as they
should, and the fundamentalism items all hang together as they should.
Factorial Validity of the Paranormal Beliefs Scale
Fac1 Fac2
Summary of Types of Validity
1954
“ Technical Recommendations
for Psychological Tests and
Diagnostic Techniques. “
Cronbach (1954)
Evaluating Measure Quality
■ Generalizability
• Does the measure retain its validity
across different contexts?
Populations
Conditions
Research Designs
3 Primary Types
1) Case Study
2) Correlational
3) Experimental
Research Designs
Case Studies
•
SINGLE PERSON
+ More detail, more depth
+ Excellent for developing theory
- Bad for testing theory
Influential Case Studies
• Anna O (Freud & Breuer, 1895)
• Phineus Gage (1860 mining accident)
• HM (Henry Molaison, 1953 surgery)
• Mask of Sanity (Cleckley, 1941)
More recently….
Dodge Morgan
Sports Illustrated Interview
"The feeling out there of being alone is
immense. If you want to know how
insignificant each of us is in the whole realm of
things, sail around the world. It made me ask
myself, 'In what realm are we important?'
We're important to the people who mean the
most to us—family and friends. We can make
a difference by being honest with them and
demanding of them. Other than that...hey, out
there on the sea I'm just one little hummer."
Dodge Morgan Study by Nasby & Reid (1997)
 IAS-R, Interpersonal circumplex
assessment (Wiggins)
 MMPI, Clinical Assessment
(Butcher)
 NEOPI, Five factor Model (Costa
& McCrae)
 Narrative Life Story (McAdams)
Research Designs
■ Correlational Studies
Identify covariation between 2
things
Correlation coefficient
– Direction and magnitude
– Cannot infer causation from
correlation .
Correlation Coefficient
■ Correlation ( r )
• Numerical index of association
• Ranges from -1.0 …0 …+1.0
Height, weight:
r = .60
Exam 1, Exam2
r = .45
Divorce, Neuroticism
r = .20
Divorce, Agreeableness r = -.20
Smoking, Extraversion
r = .08
Scatterplot
Scatterplot 1
1
r = .60
r = .60
Strong Positive Correlation
Strong Negative Correlation
3 Common Correlational Relationships:
Additive, Mediator, Moderator
Additive
r =.45
Self-Control
Well-being
Mindfulness
r =.30
Mediation
Mindfulness
r =.60
Self-Control
r =.45
r = .00
Well-being
Moderation
Maltreatment
in childhood
r =.42
MAO
The strength of the correlation
between two variables depends
on another variable.
Antisocial
Fundamentalist
Background
Spiritual
Belief
?
Paranormal
Belief
Religious Fundamentalism
Low
High
Spirituality
r = .00
r = .50
Paranormal Belief
Multiple Correlation & Regression
Personality of Video Gamers
Multiple Regression in “Causal Modeling”
Structural Equation Modeling
Persistence
Time
External Experiences
Factor Analysis
“Seeing the forests instead of the trees”
• Finding broad patterns of covariation
among many variables
• Partitioning covariation among many
measures into independent sets
• Shrinking a large number of smaller
variables into a small number of larger
variables.
Factor a number:
15 = 3 x 5
Factor a matrix of numbers:
Correlations among six variables
Height, weight
Height, shoesize
Weight, shoesize
Height, IQ
Weight, IQ
Shoesize, IQ
Educ, IQ
Job Status, Educ
.64
.52
.72
.03
.01
.08
.55
.63
Intercorrelation matrix
Intercorrelation matrix
2 product matrices
o Factor analyze the correlation matrix
(“singular value decomposition of the matrix”)
o We examine the 2 product matrices
o One is called the loading matrix
o It contains the factor loadings
Loadings
Body Mental
Mass Ability
Factor I Factor II
Factor Analysis
■ Many uses
1. Discovering trait dimensions
Perceptual sensitivity? Curiosity?
Fantasy-proneness? Liberal Values?
OPENNESS TO EXPERIENCE
2. Constructing taxonomies
(e.g., Big Five)
3. Data reduction
e.g., Aggregating predictors for regression analyses
4. Test construction
e.g., Selecting questionnaire items
Factor loading
Factor Analysis
5. Evaluating the
factor structure
of a test
“Factorial
validity”
Experimental Methods
■ Typical experiment
• Measure a trait
• Select Hi vs. Lo on the trait
• Put subjects in controlled situation
• Manipulate some variable(s)
• Measure some outcome(s)
■ 2 Examples…
Experimental Methods
■ Howarth & Eysenck (1988)
• Effect of arousal on recall
• Early vs. later recall
Theory of “Action Decrement”
– Effect of arousal on memory
– Depends on time interval
– High arousal?
Early better, later worse
Interaction between TRAIT (E) x SITUATION (Interval)
Significance & Effect Size
■ Need to distinguish…
• Statistical significance
Is r = .09 a statistical fluke (random)
or is it trustworthy?
• Practical significance
Does a correlation that size (r = .09)
have any real world implications or
practical importance?
Personality of Video Gamers
Effect Size
■ Measures of effect size
1. Magnitude of experimental effect
Cohen’s d = (Group1 – Group2) / SD
2. Correlations
Correlations are effect sizes.
Can convert d to correlation.
Effect Size
■ Useful benchmarks:
Large
40+
Medium 25 - 40
Small
15 - 25
■ Example:
• Milgram (1962) Obedience Study…
r = .40
Effect Size
■ Milgram (1962)
• effect size was equivalent to r = .40
• That is the most well-known study in
social psychology.
• Trait measures often predict important
life outcomes > .40
■ Traits are indeed real and
important.
Some Effect Sizes in Medicine:
So, effect sizes in psychology are not
abysmal.
A more grave issue is how replicable
psychology findings are.
Growing interest in systematic
replication research in psychology.
SUMMARY
• Data sources:
LOTS
• Data quality:
Reliability and Validity
• Research designs:
Case study
Correlational
Experiment