Experimental Design
• Experiment: A type of research study that tests
the idea that one variable causes an effect on
another variable.
Anatomy of an Experiment
Example
Memory Cues
N1 = 10
M1 = 16.2
S1 = 2.49
No Memory Cues
N2 = 10
M2 = 9.9
S2 = 2.33
Independent variable = Memory Training Group
Dependent variable = Memory for personal history
Anatomy of an Experiment
Example
Experimental
Group
Memory Cues
N1 = 10
M1 = 16.2
S1 = 2.49
Control
Group
No Memory Cues
N2 = 10
M2 = 9.9
S2 = 2.33
Internal Validity
• The study allows the researcher to determine
that on variable causes an effect on another
variable.
Conditions to establish internal validity
1. Time-Order relationship
Cause  Effect
I.V.  D.V.
Conditions to establish internal validity
2. No alternative explanations
• The difference between the means is due
only to the independent variable.
• Anything else represents a threat to the
internal validity of the study
Threats to internal validity
• Non-equivalent control group
– Confound: A way in which the groups differ from
each other, other than the independent variable.
– Controlling for confounds
• 1. Random assignment to groups
• 2. Matching
Threats to internal validity
• Floor or Ceiling effects
– The independent variable has made the groups
different from each other, but the dependent
variable is unable to detect it.
– Floor effect: The test is so difficult that everyone
gets a very low score.
– Ceiling effect: The test is so easy that everyone
gets a high score.
– They make the means closer together than they
should be.
Threats to internal validity
• Experimenter effect
– The experimenter gives an indication of what
they want or expect the subject to do in a
particular condition.
• Participant effect
– The participant changes their behavior to fit
what they think the researcher is studying.
Ways to address experimenter and participant
effects
– Single-blind study: The participant doesn’t know
which condition they’re in.
• Example: a placebo-controlled condition.
– Double-blind design: Neither the participants or
the researcher knows which condition the
subject is in.
External Validity
• The results of the study are generalizable
1. Generalization to different samples
– Get the same results if repeat the same study
with a different sample (from the same
population)
– Replication
External Validity
2. Generalization to different populations
– Get the same results if repeat the same study
with a sample from a different population
3. Generalization to different settings
– Get the same results under different conditions
– The effect is observed in more than one setting
– Example: The effect is observed in real life, not
just in the laboratory
Independent Samples T-Test
• Tests the difference between two sample means
Memory Cues
N1 = 10
M1 = 16.2
S1 = 2.49
No Memory Cues
N2 = 10
M2 = 9.9
S2 = 2.33
Prediction of the researcher: The mean of the Memory
Cues Group will be significantly higher than the mean of
the No Memory Cues Group.
Independent Samples T-Test
Prediction of the researcher: The mean of the Memory
Cues Group will be significantly higher than the mean of
the No Memory Cues Group.
– Example of a one-tailed test
– One-tailed test: One mean is predicted to be higher
or lower than the other one.
– Two-tailed test: One mean is predicted to be different
from the other one.
Independent Samples T-Test
Prediction of the researcher: The mean of the Memory
Cues Group will be significantly higher than the mean of
the No Memory Cues Group.
– Example of a one-tailed test
– Alternative hypothesis: The mean of the Memory
Cues Group is significantly higher than the mean of
the No Memory Cues Group.
– Null hypothesis: The mean of the Memory Cues
Group is not significantly higher than the mean of the
No Memory Cues Group.
Independent Samples T-Test
– No way to know for sure which hypothesis is true.
– We can know the odds that the null hypothesis is
true.
– We can decide how unlikely the null hypothesis
would have to be before we can’t believe it anymore.
That’s the Alpha Level of the test.
–
“α = .05” means “Reject the null hypothesis if the
odds are less than 5% that it’s true”
Independent Samples T-Test
An independent samples t-test tells you if the odds are
less than 5% that the null hypothesis is true.
1. Find the number we’re making our decision about
• It’s the difference between the two group means
• M1 – M2 = 16.2 – 9.9 = +6.3
• We’re comparing this number to a difference of zero.
2. Convert that number to a standard score
– In SPSS, t = +5.85
– The difference between the two sample means is 5.85
standard deviations above a difference of zero.
Independent Samples T-Test
3. Find how far from zero that number needs to be to
be significant  Critical Value for t
• We predicted that this difference would be in the positive
direction, so it’s a one-tailed test.
• α = .05
• Degrees of freedom = N1 + N2 – 2  10 + 10 – 2 = 18
• Critical value = +1.73
• Decision rule: If t ≥ +1.73, reject the null hypothesis.
Independent Samples T-Test
Conclusion: The mean of the Memory Cues Group is
significantly higher than the mean of the No
Memory Cues Group, t (18) = 5.85, p < .05.