Basic Experimental Design
• Common Problems
• Assigning Participants to Groups
• Single variable experiments
– bivalent
– multivalent
– baseline
• Multivariate
– factorial
– converging series
© 2001 Dr. Laura Snodgrass, Ph.D.
1
Common Problems
•
•
•
•
Confounds
Lack of control group(s)
Nonequivalent control groups
Why control groups
– history
– maturation
– testing
– instrument decay
– statistical regression
© 2001 Dr. Laura Snodgrass, Ph.D.
2
Assigning Participants to Groups
• Independent or Random Groups Design
– between groups
• Repeated Measures
– within groups
© 2001 Dr. Laura Snodgrass, Ph.D.
3
Between Groups
• Advantages
– generalizable
– collect more data at a given level
– shorter time for each participant
• Disadvantages
– may not be random
– unequal N
– potential confounds
– requires more participants
© 2001 Dr. Laura Snodgrass, Ph.D.
4
Between Groups
• Matching to equate groups and decrease error variance
• How
– correlated variables
– pairs
– yoked controls
– performance criterion
© 2001 Dr. Laura Snodgrass, Ph.D.
5
Matching
• Advantages
– equates groups
– increase power of experiment
– decrease number of participants needed
• Disadvantages
– extra work
– extra testing
– lose individual differences - less generalizable
© 2001 Dr. Laura Snodgrass, Ph.D.
6
Repeated Measures
• Advantages
– fewer participants needed
– impt for special groups
– statistically more powerful
• Disadvantages
– not naïve after first trials
– order effects
• practice and fatigue
• non-symmetric or differential transfer
© 2001 Dr. Laura Snodgrass, Ph.D.
7
Counterbalancing
• Vary order of treatment to distribute or measure order effects
• Complete counterbalancing
– within participants ABBA
– between AB for some, BA for others
• Latin Squares
– each cond at each ordinal position
– precedes and follows each other once
© 2001 Dr. Laura Snodgrass, Ph.D.
8
Counterbalancing
• Randomized blocks
• Time interval between trials
– mortality
© 2001 Dr. Laura Snodgrass, Ph.D.
9
Single Variable Experiments
• Bivalent
– one independent variable with two levels
• Multivalent (functional)
– one independent variable with three or more levels
• Baseline
© 2001 Dr. Laura Snodgrass, Ph.D.
10
Bivalent
• Two levels of the independent variable
– experimental and control groups
– two different levels of the variable
• Post-test only vs. pre-test/post-test
• Advantages
– easy to interpret and analyze
– decide if IV is worth studying
© 2001 Dr. Laura Snodgrass, Ph.D.
11
Bivalent
• Disadvantages
– limited theoretical value
– conclusions may be based on arbitrary choice of
levels
– negative findings are not conclusive
– does not describe shape of relationship therefore you
may over generalize for non-linear relationships
• interpolation and extrapolation
• plateau or asymptote
© 2001 Dr. Laura Snodgrass, Ph.D.
12
Multivalent (functional)
• Gives more info about the shape of the relationship
• Advantages
– better estimate true relationship
– individual choice of levels becomes less critical
• Disadvantages
– more: time, effort, cost, subjects
– more complex statistics and interpretation
© 2001 Dr. Laura Snodgrass, Ph.D.
13
Baseline
• Only works with certain types of variables
– will not work with variables that cause permanent
change
• Procedure:
– establish baseline or steady-state response level
– introduce IV until stable transition
– allow subject to return to baseline
© 2001 Dr. Laura Snodgrass, Ph.D.
14
Baseline
• Advantages
– rules out most confounds
– easy to interpret (often no statistics)
– flexible and replicable
– investigate behavior of an individual
• Disadvantages
– does not show small changes
– may not generalize
© 2001 Dr. Laura Snodgrass, Ph.D.
15
Multivariate Experiments
• Factorial Designs
– two or more independent variables, each with two or
more levels
– variables can be all between, all within, or mixed in
many combinations
• Converging series
– series of small experiments in which a variable
manipulated in an earlier experiment becomes a
control variable in a later experiment
© 2001 Dr. Laura Snodgrass, Ph.D.
16
Factorial
• Design matrix
– produces a family of functions
– study main effects and interactions
• Advantages
– study interactions
– increases precision and generalizability
– decrease statistical error and increase power
– theoretical value
© 2001 Dr. Laura Snodgrass, Ph.D.
17
Factorial
• Disadvantages
– increases time, money and number of subjects
increases dramatically as number of cells increases
– assumptions of ANOVA may not be met
– N-way interactions are very difficult to interpret
© 2001 Dr. Laura Snodgrass, Ph.D.
18
Converging series for applied problems
• Optimal designs
– e.g. car, medical treatment, office
• Find an optimal level of a variable and turn it into a control
variable
– lose higher order interactions
© 2001 Dr. Laura Snodgrass, Ph.D.
19
Converging Operations
• Converge on a single hypothesis
– start with several possible hypotheses or
explanations
– each experiment eliminates one or more until only
one remains (hopefully)
• For example:
– perceptual defense against vulgar words
– isolation tank
© 2001 Dr. Laura Snodgrass, Ph.D.
20
Converging Series
• Advantages
– flexible, many choice points
– efficient, leave out factors that have no effect
– built in replications
• Disadvantages
– interactions are lost
– almost always between subjects
– analyze and interpret prior before next experiment
so can take a long time
© 2001 Dr. Laura Snodgrass, Ph.D.
21