Using Natural Settings
Observational Studies
Field Experiments
~ Chapter 10
Observational Research
Designs
Useful as a starting point
- What should the researcher look for?
Generalization of lab findings
- Do lab results apply to the “real” world?
Sometimes is the only way ...
T. Matsuzawa
- studying the phenomenon where (and as) it happens
Observational Research
Designs
Example: Chimpanzees’ nutcracking behavior
in the wild
- more clips (1), (2), (3)
Two (main) types
Naturalistic observation
- researcher doesn’t intervene
Participant observation
- researcher is part of the group of participants
Naturalistic Observation
Researchers unobstrusively observe the
phenomenon (e.g., behavior) in its natural
setting
When focused on specific behaviors/settings:
- Systematic Naturalistic Observation
When observations are made overtly:
- Desensitization: researcher gradually moves closer to
participants until s/he can sit close to them
- Habituation: researcher appears in setting many times
until presence no longer affects behavior
Participant Observation
Researcher is an active participant in the
situation
- Undisguised participants studies
- Disguised participants studies
Researcher must be aware of personal biases
- (Very hard!!)
- Defining relevant terms in advance helps
Field Experiments
Controlled studies that occur in a natural
setting
Researcher manipulates an IV and measures its
effect on a DV (looking for causality)
Great ecological validity
Internal validity
- same issues as in Lab experiments
T. Matsuzawa
Problems with Observational
Studies
Usually they don’t allow establishing causal
relationships (exception: field experiments)
Two main sources of confounds
- influence of the observer in participants
- biased observations
Influence of the Observer in
Participants
In Social Sciences: Hawthorne effect
- participants alter their behavior as a result of being
part of a study
Reactivity (reactive measure)
- change in participants’ behavior due to their
awareness of being observed
- brings problems with internal validity
- can be minimized via desensitization and habituation
Observer Bias
Expectancy effects
- (un)conscious preferences affecting observations
- it will become evident if replications fail
- can be reduced by using naïve observers (coders)
Can be reduced by using recording devices
- which bring other advantages (re-viewing, slow
motion, etc.) and disadvantages (e.g., reactivity)
Choosing the Type of
Observational Design
Ecological validity
- Naturalistic Obs > Field Exp > Lab Exp
Control
- Lab Exp > Field Exp > Naturalistic Obs
Practical/ethical considerations
- Time, resources, logistics, ethical issues, etc.
Data Collection
What phenomenon will be observed and how?
- Dependent measure must be reliable and valid
Reliability
- Measurement procedure yields consistent results
- Interobserver reliability (different formulas)
- based on % agreement / disagreement
- ordered data
- interval / ratio data
Techniques for Data
Collection
Research Question determines what will be
observed and how it will be observed
Narrative records
Checklists
Tech-mediated advanced methods
Narrative records
Running records of behavior in a given situation
Can be complete or skecthy
Obtained via naked-eye obs, or via audio/visual
recordings
Provide lots of info, but sometimes it is hard to
organize it
- Data reduction (e.g., transcription and coding)
- Reliability criteria apply
Checklists
Used to guide observations
- especially when there is a focus on limited number
of specific behaviors
- operational definitions
Static checklist
- record observations that won’t change during obs
- e.g., demographics
Action checklist
- record status of specific features over time
- e.g., child’s type of play behavior over time
Narrative Records vs.
Checklists?
Narrative records
no restrictions of pre-
no need of data reduction
(e.g., by coding)
data reduction (e.g., by
coding) is needed
focus on a relatively small
subset of behaviors
(+) defined types of behaviors
(-)
Checklists
Types of Data Collected
Frequency (how often?)
Duration
Timing
Ratings
Sampling Techniques
Natural situations cannot be observed all
over the place all of the time
- Need of samples
Behavior sampling
- Time sampling
- Event sampling
Situation sampling