Revised Grading Scheme
Examinations
Exam #1
Exam #2
Exam #3
Exam #4
Assignments
= 35 pts.
= 35 pts.
= 35 pts.
= 35 pts.
Drop lowest test score
_______
Total Exam pts
= 140 105
Total points for the course = 280 235
Introduction and References
Methods section
Results section
Final paper
Lab activities Assignments
Participation (Research proj)
Summary presentation
Total assignment pts
= 25 pts. 30 pts
= 25 pts. 20 pts
= 20 pts.
= 40 pts.
= 10 pts.
= 10 pts. vMWM
= 10 pts.
_______
= 140 130
1
Chapters 12 & 15
And so much more
2
Large and Small N designs
Small N
• one or a few subjects
Large N
• Greater than a few subjects (often multiple
groups)
• most common technique used in research
design
3
Large N Designs
• Gained in popularity after Sir Ronald
Fisher invented the analysis of variance in
the 1930s
• Easier to generalize with more than one
subject (greater external validity)
4
Why even use small N?
• Precision – pooling or combining data can
obscure the results of individual subjects
• You may miss effects by pooling data
across individuals.
Subject 1
Subject 2
Combined
8
8
8
7
7
6
6
6
5
5
5
4
4
4
3
3
2
2
1
1
0
0
7
3
2
1
1
2
3
4
1
2
3
4
0
1
2
3
54
Why even use small N?
• Another example where pooling data led to
a misinterpretation of what subjects had or
had not learned?
• Hint: a series of water maze studies on the
effects of partial reinforcement (PR)
– How many subjects in the PR group?
– What data was pooled?
– What was discovered by de-aggregating the
data?
– What’s the big picture lesson?
6
The BIG PICTURE lesson
• Large N’s aggregate over subjects.
• Smaller N studies sometimes aggregate over
time.
• Both have the potential to loose fidelity
Mirriam-Webster Online
a: the quality or state of being faithful b: accuracy in details :
exactness 2: the degree to which an electronic device (as a record
player, radio, or television) accurately reproduces its effect (as sound
or picture)
From Wikipedia, the free encyclopedia
High fidelity (disambiguation)
High fidelity or hi-fi is most commonly a term for the high-quality
reproduction of sound or images
7
Small N Designs
• Also used for practical reasons
– Only a few patients in clinical research for a
rare disease, plenty with common ones
– Animals may be expensive (especially those fancy rats)
Just the crowd I
want to hang
around and get
advice from
So, it’s ideal for poor
researchers with restricted or
limited access to human
patients and/or those that may
lack motivation to collect
acceptable amounts of data in
order to do a real study
deemed credible by other
8
scientific peers!
Small N Designs
Popular in:
• Clinical and animal research
• Laboratory and field studies
• Psychophysics
• Studies of learning
• Used most extensively in operant
conditioning research
9
10
ABA Design
• The return to baseline in the ABA design
tests whether B had an effect or whether
another extraneous variable confounded
the study.
• Thus, the effect of B, the experimental
treatment, must be reversible
• it is also called a reversal design
11
Variations of the ABA Design
• ABABA – two treatments and two returns to
baseline – can detect cumulative effects of the
treatment
• ABACADA – multiple experimental conditions B, C and D represent different treatments
• AB design – sacrifice the return to baseline if it
would harm the subject (e.g., behavior
modification worked in reducing self-injurious
behavior)
12
Variations of the ABA Design
A Swedish
design that
only made
sense in the
drug-induced
haze of the
70s disco era.
13
Variations of the ABA Design
• Multiple baseline design – a series of
baselines and treatments are compared,
but once a treatment is established it is not
withdrawn (e.g. AAABBB no more As)
• Discrete trials design – does not rely on
baselines at all, but compares
performance across treatment conditions
(e.g. BCDE) a BC design would be
analogous to what large N design?
14
Variations of the ABA Design
AC/DC – a.k.a, the “Indiscrete trials design”
• After “A”, never
return to baseline
• skip all the boring B
condition stuff and
go right for the CDC
conditions that put
you on a fast track to
the land downunder…
• Apply thunderbolt
between C and D.
15
B. F. Skinner
• Studied changes in the
rate of behavior (e.g., a rat
lever pressing for food)
• by careful, continuous
measurement of a single
subject over time.
The control and experimental conditions are given to the same subject at different times
A
B
A
Baseline
Experimental
Baseline
16
Evaluating the Experiment
• Internal validity – was the experiment free of
confounding?
• Manipulation check – assesses how
successfully the experimenter manipulated the
situation she or he intended to produce.
• Pact of ignorance – subjects who have
guessed the hypothesis might try to hide the fact
because they know that their data might be
discarded.
17
Statistical problems
• Statistical conclusion validity – are
conclusions about the statistical results
valid?
• Did you use an appropriate test?
• Too many a priori tests – increases the
chance of making a Type 1 error.
• Small effect size – the results can be
significant but not very meaningful if the
effect size is small.
18
External validity
• Two requirements:
– The experiment is internally valid
– And can be replicated
What form of validity is a prerequisite for
another form of validity?
19
Research significance
• Are the results consistent with prior
studies?
• Do the results extend our knowledge of
the problem?
• Are there any implications for broader
theoretical issues?
20
Multivariate Designs
• Involve multiple variables studied
concurrently
– MANOVA (multiple DVs)
– Multiple correlation
– Factor analysis
21
Unobtrusive measures
• Specific procedures for measuring a
subjects behavior without them
knowing that their behavior is being
measured
– Greater external validity because the
behavioral data is similar to behavior
occurring outside the experiment
– E.g., a field experiment
• Manipulate antecedent conditions
• Observe outcomes in natural setting
22
Nonsignificant results
You should reconsider:
• The experimental hypothesis
• The procedures used in the study
• The possibility that a Type 2 error
occurred
23