PSY 620P
Messinger

Design
 Developmental Designs
 Internal and External Validity of a Study

Measurement
 Reliability and Validity of Measures
 Instrument Construction Stages
 Dealing with missing data

Ethics in Developmental Studies
 Children as vulnerable population
 Assent

Analysis
 Visualizing your data
 Hypothesis Testing
 Approaches to Analyzing Change over Time




Replicability
Access to samples
Replicable (objective?) measurement
Addressing the crisis…
Strange Situation
examples
Ubisense examples
What about SEM?
What is causality?
Mattson, et al.,PLOS One, 2013

Between subject
 A treatment (independent variable) is
assigned randomly
 creating treatment and control groups
Within-subject
 All infants get treatment and control
 Examples
▪ Rating study, Face-to-face still-face
Estimated Marginal Means

Estimated Marginal Means of MEASURE_1
.16
.14
.12
.10
.08
.06
EXP2
.04
Comparison
.02
Exposed
1
SMILES
Messinger
2
3

Quasi-experimental
 differences in naturally occurring groups

Observational  Differences in naturally occurring conditions

Complementary, not exclusive

Is age (development) studied experimentally
or observationally?
Messinger
T indicates
children who
have just
transition from
junior high
school
Alfieri et al.,
1996
Belfort et al., 2013

Quasi-experiment
 Between subject exploration
of differences in naturally
occurring groups

Observational
 Differences in naturally
occurring conditions
▪ Gazing at mother versus gazing
away
Messinger
0.8
Proportion of Open Mouth Smiling Only
▪ Drug exposure, breast-feeding,
and attachment groups
Figure 2b.
Solo Open Mouth Smiling by Gazing at Mother
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0



Experimental and all observational
approaches measures variables
Variable - a measurable component of
behavior or physiological functioning that can
take on different values
Not all aspects of behavior or physiology
 specific features of interest
Messinger

Intensive description in regular language
 Not measuring variables
▪ E.g., baby biography, one infant described over time

Pro: Insight into individual and developmental
process
 Emerged with romantic emphasis on individual

Con: Not generalizable

Complementary (counting themes), not exclusive,
NIH
Messinger
Longitudinal vs. cross-sectional designs
Same infants over
time
 Pro: Answers ‘How do
individuals change in
time?’
 Con: Takes a long
time
Typical Trajectory:
Cognitive Scores Decline

110
Bayley Cognitive Score
 Attrition
120
100
90
80
70
60
50
1
Year
M = 93.5
n = 200
Messinger
2
Years
M = 79.1
n = 190
3
Years
M = 82.1
n = 132
Rosenquist et al. PNAS | January 13, 2015 | vol. 112 | no. 2 | 357
Different infants at
different times
 Pro: Efficient, large
numbers of subjects
 Con: Differences do
not necessarily reflect
individual’s
development

 e.g. cohort
Messinger
95
90
85
80
75
70
65
60
55
50
One
Year
Two
Year
Three
Year
Different
Individuals

Development is relatively
stable on large time scales
 Motor, physical, emotional,
communicative


But choppy on smaller scales
Only longitudinal research
can show individual
development
Messinger
Messinger
6 mos.. 17”
Birth 13.75”
12 mos. 18”
Messinger
24 mos. 19”

Developmental trajectories take many
forms

Accurate depiction of trajectory depends
on sampling rate of observations
 Overly large sampling intervals can distort
shape of change
 Inaccurate picture of developmental
trajectory

“Microgenetic method” – small time
intervals to observe developmental
process
Jutagir

How small is small enough? How large
is too large?

Previously, measurement intervals
chosen by:
 Intuition, convenience, and tradition
Jutagir

Call for study design based on formal
theoretical models about the shape of
the underlying function
 Dictates number of data points and
optimal spacing

Difficult to apply in psychology
 “Chicken and egg” conundrum: Cannot
mathematically calculate the shape of the
function until we know the shape of the
trajectory
Jutagir
Hoffman

Study collected daily measurements of motor
development

Next, systematically manipulated data to
alter time between assessments

Observed how trajectories were affected
Jutagir

Results
 Many kids are
variable in achieving
motor skills
 Increased duration
of time (as small as
2-3 days more)
between assessment
periods masks that
variability
Jutagir

Continuity(=absolute change)
 Behavior level is continuous
(discontinuous) across ages
 How does a behavior change in form and/or
function over the course of development?
Stability
 Rank of individual in group is stable
Typical Trajectory:
Cognitive Scores Decline
 How does a behavior change differently among
individuals in the same group? (=relative change)
120
110
Bayley Cognitive Score

100
90
80
70
60
50
1
Messinger
Year
M = 93.5
n = 200
2
Years
M = 79.1
n = 190
3
Years
M = 82.1
n = 132
Validity of Developmental Studies

External validity =

Internal validity =
 Methodological soundness of study allowing changes
in DV to be attributed to the IV
 Threats to internal validity = uncontrolled confounds
▪ Need to control for various methodological confounds
through adequate sampling, random assignment (when
possible), inclusion of control group etc.
Threats of particular concern in
Developmental Studies (cont)

History:

Maturation:

Testing:

Instrumentation:

Regression:
▪ Example of Regression * Selection effect
Threats of particular concern in
Developmental Studies: Regression

High anxious freshmen selected for
intervention in first week of school; by midyear show significant decrease in anxiety
Pretest
90
Intervention
Posttest
70

The process of quantifying abstract concepts
such as:
▪ Intelligence
▪ Sociability
▪ Emotion Regulation

Developmental assessments often rely on
indirect measures
 i.e., habituation as processing index

Are we measuring what we think we’re
measuring,
 Do the variables measured the constructs
mentioned in the research questions?
 There is no final answer
▪
▪
▪
▪
Reunion behavior = Attachment?
Smiling = Joy?
Looking = Preference?
Heart rate = Arousal?
Messinger

Requires
 Detailed operational definitions
 Creation of sensitive instruments
 Rules for scoring instrument to create summary
scores

Validity
 Does measure provide intended information for
intended population?
▪ Can vary with age and subgroup (e.g., ethnicity or SES)

Reliability
 How consistent is children’s behavior?
▪ Tends to increases with age and diversity of sample


Observed on-line or video-recorded
Measured with
 Trait rating - global judgement
 Time sampling
 Event sampling (frequency)
 Event sampling (duration)
Messinger
Messinger

Heart rate & respiration (video)
 avoidant infants, infants on visual cliff

Electroencephalogram
 Relative lateral activation during crying

Actigraphy
 Index of ADD?

Increasingly important supplement to
behavioral measures
Messinger

Missing Data
 Most common reason for low power in studies of
change over time

Options
 Deletion
 Substitution
 Imputation

The methodological literature favors
maximum likelihood and multiple imputation
 a strong theoretical foundation, less restrictive
assumptions, and the potential for bias reduction
and greater power.

Benefits are especially important for
developmental research where attrition is a
pervasive problem
Enders, Craig K.
Child Development Perspectives, Vol 7(1), Mar 2013, 27-31.

Reliability
 Consistency of measurement
▪ Inter-rater reliability of observations

Bias
 Systematic impact of unmeasured variables
▪ Blinding in drug studies
▪ Keeping observations independent
Messinger

Analysis
 Approaches to Analyzing Change over Time
▪ Describing group level patterns of change over time
▪ Describing individual differences in patterns of change
▪ Processes underlying/modifying patterns of change
▪ Mediating and moderating variables