Research Approaches
• Internal Validity
• External Validity
“A study that is fetchingly realistic might bring us no closer to the
truth than one that seems painfully contrived” (Myers &
Hansen, 2006, p. 63).
1
Dimensions of Research
• Antecedent Manipulation
–Treatments
–Independent variable (IV)
• Imposition of Units
–Behavioral measures
–Dependent variable (DV)
2
Imposition of Units
Dimensions of Research
High
Medium
Low
High/
Low
Medium/
Low
High/
Medium
Medium/
Medium
High/
High
Medium/
High
Low/
Low
Low/
Medium
Low/
High
Low
Medium
High
Antecedent Manipulation
3
Imposition of Units
True Experiments
High/
High
High
Medium
Low
Low
Medium
High
Antecedent Manipulation
4
Nonexperimental Approaches
• Phenomenology
• Case studies
• Field studies
– attending to and describing one’s own experience
– outside observer records an individual’s experiences & behaviors
– research method conducted in the field using a variety of techniques
– Field experiments
– a type of field study?
• Archival studies
• Qualitative studies
– reexamine existing data for a new reason
– data are verbal descriptions rather than numbers
5
Nonexperimental Approaches
• Phenomenology Description of one’s own immediate
experience
Examples: pain in my neck (C5 vertebrae)
the Purkinje effect
6
Imposition of Units
Phenomenology
High
Medium
Low
Low/
Low
Low
Medium
High
Antecedent Manipulation
7
Nonexperimental Approaches
• Case studies Descriptive records of another
individual’s experiences or behavior.
Evaluative case studies – case compared to hypothetical
“normal” psychological diagnosis – DSM-IV? Now DSM-5
http://www.psychiatry.org/psychiatrists/practice/dsm/dsm-5/onlineassessment-measures
Deviant case analysis – deviant case compared to “normal”
for significant differences.
e.g. Mednick, 1969 – ANS of schizophrenic children functions
different compared to normal controls.
8
Imposition of Units
Case studies
High
Medium
Low
High/
Low
Medium/
Low
Low/
Low
Low
Medium
High
Antecedent Manipulation
10
Nonexperimental Approaches
• Field studies Studies done in situ, in real-life settings as
opposed to the laboratory.
Compare to - A field Experiment in
Chicago (p. 86).
11
Imposition of Units
Field studies
High
Medium
Low
High/
Low
Medium/
Low
Low/
Low
Low
Medium
High
Antecedent Manipulation
12
Nonexperimental Approaches
• Naturalistic observation a technique of observing behaviors as
they occur spontaneously in the natural
setting.
e.g. dominance hierarchies in social
groups.
13
Imposition of Units
Naturalistic Observation
High
Medium
Low
Low/
Low
Low
Medium
High
Antecedent Manipulation
14
Nonexperimental Approaches
• Systematic observation a technique of using specific rules in a prearranged way to objectively record
observations.
Female sexual receptivity (rodents only)
Lordosis- 1. darting, 2. ear wiggling 3.
inverted back and 4. tail diversion
15
Nonexperimental Approaches
• Participant-observer studies the researcher becomes part of the group
being studied.
Undercover roid guy… just what baseball
needed!
16
Nonexperimental Approaches
• Archival study already existing records are reexamined for a
new purpose. E.g. data on crime, death rates,
education levels, salaries housing patterns and
disease rates are accessible to researchers.
Bioinformatics
Gene database
17
Nonexperimental Approaches
Self-reports personal narratives
expression of ideas, memories,
feelings and thoughts
• Qualitative research
relies on words rather than numbers
Is there a paradigm shift occurring?
18
Phenomenology is used as part of
qualitative research
Contemporary or Empirical Phenomenology
1. Researcher self-reflects on experiences
related to the phenomenon
2. Others provide verbal or written descriptions
of experiences
3. Accounts of the phenomenon are gathered
from literature, art, television, the internet
and other sources
19
Correlational and QuasiExperimental Designs
Chapter 5
20
Correlational Designs
Determine the degree of relationship
between two traits, behaviors or
events; predict one set from another.
• Antecedents are preexisting
• Degree of imposition of units - high
• Tend to be higher in external validity
21
Correlational Designs
Imposition of Units
High
Low/
High
Medium
Low
Low
Medium
Antecedent Manipulation
High
22
Quasi-experimental Designs
Can seem like an experiment, but subjects
are not randomly assigned to treatment
conditions.
• Antecedent control varies
• Degree of imposition of units - high
• Tend to be higher in external validity
23
Quasiexperimental Designs
Imposition of Units
High
Low/
High
meduim/
High
Low
Medium
Medium
Low
Antecedent Manipulation
High
24
Example of a Quasiexperiment
Lighting condition – fluorescent vs
incandescent.
Subjects – from company A (fluorescent
lights) or B (incandescent).
Performance measure – productivity.
Can cause-effect be established with
confidence?
25
Pearson Product-Moment Correlation
Coefficient (r )
Most common procedure for calculating
simple correlations – relationship
between pairs of scores for each
subject. Three outcomes are possible:
• Positive relationship
• Negative relationship
• No relationship
26
Scatterplots
Visual representations of the scores belonging to
each subject in a study. Each dot = two
scores (x,y) from one subject.
• One score places the dot along the
horizontal axis (x) and the other score places
it along the vertical (y) axis.
• Regression lines (of best fit) represent the
mathematical equation that best represents
the relationship between the two measured
scores.
27
Hypothetical Relationships
B.
Positive r = +.69
Variable Y
Negative r = -.72
Variable Y
A.
Variable X
No correlation r = -.02
Variable Y
C.
Variable X
Variable X
28
Four possible causal
directions of a correlation
•
1)
2)
3)
4)
Given a strong positive relationship between
childhood aggressiveness and watching
violent TV (r = +.70).
Watching violent TV  aggressiveness
Aggressiveness  watching violent TV
Aggressiveness  watching violent TV
Both are caused by a third variable
(unknown or not measured, e.g., parental
supervision)
29
Coefficient of determination
•
•
•
•
Estimates the amount of variability in scores
on one variable that can be explained by the
other variable.
E.g., if r = .56, then r 2 = .31.
31% of the variability in scores on variable X
can be accounted for by variable Y.
An r 2 ≥ .25 can be considered a strong
association.
30
Variable Y: calculate mean and S
Regression equation
Positive r = +.56
Y
Y intercept
slope
X
Variable X: calculate mean and S
31
Regression Equation
•
Given the score on one variable you can
predict the score on the other if you know:
– The value of r
– Average scores of X and Y (the means)
– Standard deviation (S) of X and Y
Y = Y + r [Sy / Sx] (X – X)
32
Multiple Regression
•
•
•
Used to predict the score on one behavior
from the scores on others included in the
analysis.
The regression equation provides beta
weights for each predictor (indicating their
importance)
Beta weights can simply be reported or used
in an advanced correlational analysis to
construct causal sequences for the behaviors.
33
Multiple Correlation
•
•
•
Intercorrelations among 3 or more behaviors
(R)
Can not explain why the 3 measures are
related but it may suggest that a “third
variable” is important.
Influence of one variable is held constant
while measuring the correlation between
the other two – partial correlation
34
Causal Modeling
•
•
Advanced correlational techniques
provide information about the direction of
the cause and effect sequences among
variables. Two techniques:
1. Path analysis
2. Cross-lagged panel designs
35
Path Analysis
•
•
•
Creates models of possible causal sequences
when several related behaviors are measured
Beta weights from multiple regression
analysis are used to evaluate the direction of
cause and effect from correlated variables.
Internal validity is low (correlational data),
consequently causal statements can not be
made.
36
Path Analysis
Perceived
Risk
.25**
.20*
.30**
Monitoring
* p < .05, ** p < .01
Intrusive
Thoughts
.37**
Psychological
Distress
Internal validity?
Third variables?
From Schwartz, Lerman, Miller, Daly, and Masny (1995)
37
Cross-Lagged Panel Design
•
•
•
•
Uses relationships measured over time to
suggest causal models.
The same pair of related behaviors or
characteristics are measured at two separate
time points for each subject.
Can only suggest the direction of causal
relationships (not conclusive).
Bidirectional causation and the third variable
problem cannot be ruled out.
38
Cross-Lagged Panel Design
Age 3
Age 8
r = .14
Time watching
TV
r = .20
Time watching
TV
r = .05
r = .07
Size of
Vocabulary
r = .41
Size of
Vocabulary
Hypothetical Cross-Lagged Panel design
39
Quasiexperimental Designs
•
•
•
•
Subjects cannot be randomly assigned to
different treatments
Quasi-treatments are formed based on a
particular event, characteristic or behavior of
interest.
E.g., gender differences in sleep patterns.
Low internal validity.
40
Quasiexperimental Designs
•
•
Subjects may be exposed to different
treatments, but without random assignment
(e.g. the lighting-productivity study)
There is a lack of control over other potential
confounds (i.e., an inability to hold all else
constant except for the treatment condition).
41
Ex Post Facto Studies
•
•
•
•
Ex Post Facto – systematic examination of the
effects of subject variables (characteristics)
without manipulation.
Low Antecedent Manipulation
High Imposition of Units
Greater external validity
42
Nonequivalent Groups
•
•
•
A manipulation is carried out but subjects are
not randomly assigned to groups
E.g. the lighting experiment yet again
Internal validity can be increased by
controlling extraneous variables after careful
consideration of potential confounds.
43
Longitudinal Designs
•
•
•
•
Measure the behavior of the same group of
subjects across time.
A form of within-subject design
Important for studying growth and
development and aging
Retaining subjects may be difficult
44
Cross-sectional Studies
•
•
•
Investigates changes across time by
comparing groups of subjects already at
different stages at a single point in time.
Typically requires more subjects than the
longitudinal study.
Subjects may differ in ways other than those
being studied (similar to Ex post facto).
45
Pretest/Posttest Design
•
Investigates the effects of a treatment by
comparing behavior before and after the
treatment.
• Practice effects (pretest sensitization)
• Outside influences cannot be ruled out
• Low internal validity
e.g., exposure to cocoa on cognitive
performance
46