Conducting Experiments
• Choosing methods
• Sampling and sample size
• Independent variables
• Dependent variables
• Controls
• Debugging
© 2001 Dr. Laura Snodgrass, Ph.D.
1
Choosing Methods
• Laboratory experiments can be artificial
– too much control
• Field experiment
– more natural setting
– lose some control
• Ethical and practical concerns
• Participant variables
– quasi-experimental designs
© 2001 Dr. Laura Snodgrass, Ph.D.
2
Choosing
• Description and prediction can be done without casual concerns
• Human complexity
– number of interacting causal variables
• Neglect of individual differences
– averaged across groups
• Social responsibility
– objectivity
– values
– Gergen’s paradigm II
© 2001 Dr. Laura Snodgrass, Ph.D.
3
Sampling
• Generalization requires adequate sampling
• Populations
– you define population “of interest”
• Why sample
– cost-benefit analysis - law of diminishing returns
– destruction of items tested
– infinite populations
– may increase accuracy
© 2001 Dr. Laura Snodgrass, Ph.D.
4
Participant Sampling
• Psychology as the study of white rats and college sophomores
• Generalization from
– different species
– different groups
• students have different pressures and performance
anxiety
• volunteers differ from non-volunteers
© 2001 Dr. Laura Snodgrass, Ph.D.
5
Sampling Techniques
• Systematic random sampling
• Stratified random sampling
• Cluster sampling
• Haphazard or convenience sampling
• Quota sampling
© 2001 Dr. Laura Snodgrass, Ph.D.
6
Other Types of Samples
• Experimenters as samples
– gender, age, ethnicity, behavior, dress
• Stimulus sampling
– representative of pop of stimuli
– random or controlled
• Condition sampling
• Response sampling
– number of dependent measures
– number of trials
© 2001 Dr. Laura Snodgrass, Ph.D.
7
Sample Size
• Tradition
– look in journals
• Expected variability in results
– consistency within and between participants
• Planned statistical analysis
– parametric versus nonparametric
– significance level
– size of difference between means expected
– do a power analysis
© 2001 Dr. Laura Snodgrass, Ph.D.
8
Independent Variables
• Setting the stage
– informed consent
– brief explanation of what is expected
• Types of manipulations
– straightforward
– staged
• to create a psychological state
• to simulate a real world situation
• use confederates
© 2001 Dr. Laura Snodgrass, Ph.D.
9
Independent Variables
• Strength of manipulation - choosing levels
– number of levels
– range
– how close together are the levels
• Combining variables
– incomplete or unbalanced designs
(leaving out some cells)
• redundant or illogical
• data from literature
• too many cells to fill
© 2001 Dr. Laura Snodgrass, Ph.D.
10
Independent Variables
• Confounding
– environmental confounds
• the “Hawthorne Effect”
– participant confounds
• equality of groups
• Cost of manipulation
© 2001 Dr. Laura Snodgrass, Ph.D.
11
Dependent Measures
• Types of measures
– self-report
• rating scales
– behavioral
• reaction time
• error rate
– physiological
• GSR, heart rate
© 2001 Dr. Laura Snodgrass, Ph.D.
12
Dependent measures
• Sensitivity
– ceiling effect - too easy
– floor effect - too hard
– no effect
• Multiple Measures
– e.g. time perception
• Ethics of measures (e.g. privacy)
• Cost
© 2001 Dr. Laura Snodgrass, Ph.D.
13
Other Controls
• Participant Effects
– loss of subjects
– volunteers
– social desirability
– demand characteristics
• deception
• filler items
• placebo groups
© 2001 Dr. Laura Snodgrass, Ph.D.
14
Controls
• Experimenter effects
– experimenter bias or expectancy effects
• subtle coaching
• recording errors
• teacher expectancy
• Solutions
– training
– run conditions simultaneously
– single-blind
– double-blind
© 2001 Dr. Laura Snodgrass, Ph.D.
15
Debugging
• Research proposals
– getting feedback from others
• Pilot studies
• Manipulations checks
– especially in pilot study
– can explain non-significant results
© 2001 Dr. Laura Snodgrass, Ph.D.
16
Data Defects
• Missing data
– some statistics allow for missing daat
– can replace with averaging techniques
– SPSS have several missing data options
• Extreme score or outliers
– techniques for discarding
– replace with averaging
• Appropriate Statistics!!!!!!!!!!!!!!!!!
© 2001 Dr. Laura Snodgrass, Ph.D.
17