Housekeeping
• Assignments:
– #3 and #4 and the remarking of #1 will be returned on
Monday
– #5 due on Wednesday, Nov 9th (5 pm)
• Mid-term will be returned on Monday
• Draft #1 of research paper is now due on
Monday, November 14th (5pm)
– http://web.cs.dal.ca/~hawkey/RMS/RMS_research_paper.pdf
– Also, the research proposal talk and today’s talk from the Writing Center
– When in doubt, refer to good research papers that you’ve read for ways to
present the ideas
Factorial Designs
Research Methods & Statistics
Fall 2011
Kirstie Hawkey
Example drawn from Ch. 12 of McBurney’s research
methods textbook
Good tutorials
• Between subjects factorial design:
– http://web.mst.edu/~psyworld/between_subjects.htm#
– http://www.experiment-resources.com/factorial-design.html
– http://www.experiment-resources.com/factorial-anova.html
• Mixed factorial design:
– http://web.mst.edu/~psyworld/mixed_designs.htm
• Course notes (Ch 12 of McBurney’s research methods):
– http://axon.psyc.memphis.edu/~charlesblaha/3000/Lec%208%20True%2
0Exp/Lec%208%20True%20Exp%20-%20Part%202-STUDENTS.pdf
Factorial Designs
• When you are manipulating more than one
independent variable
– Need to examine the impact of multiple IVs on the
DVs
– More efficient than running 2+ experiments
– Can also examine interactions between the IVs
– Must examine all possible cominations of the
values of the variables
Terminology
• #levels of 1st IV x # of levels of 2nd IV
– 2 IVs each with 2 levels: 2 x 2
– 3 IVs each with 2 levels: 2 x 2 x 2
– 2 IVs each with 3 levels: 3 x 3
• Usually identify the levels when writing about it:
– A 2 (interface: A, B) x 2 (screen size: small/large)
factorial design
– A 2 x 2 factorial design was used. Between subject
factors were interface (A or B) and screen size (small
or large)
– Follow the conventions in papers in your domain
Simple 2 x 2 design
• Examining characteristics of a person that influence
judgment of guilt
– Facial expression (smiling person less guilty)
– Attractiveness (attractive person less guilty)
• 2 IV’s each with 2 levels
• Need to have a condition for each combination of
factors
– Between subjects experiment
– 4 groups of participants judging different sets of faces:
unattractive neutral faces; unattractive smiling faces;
attractive neutral faces; attractive smiling faces
– Participants judge whether guilty or not
Simple 2 x 2 design
Factor A1
(neutral face)
Factor A2
(smiling face)
Row Means
(Effect of B)
Factor B1
(unattractive)
A1B1
88
A2B1
24
56
Factor B2
(attractive
A1B2
16
A2B2
52
24
Column
means (Effect
of A)
52
28
Main effect: The effect of a variable averaged over all values of
another variable (or variables)
Is there a main effect of Facial expression on the judgment of guilt?
Is there a main effect of Attractiveness on the judgment of guilt?
Simple 2 x 2 design
Factor A1
(neutral face)
Factor A2
(smiling face)
Row Means
(Effect of B)
Factor B1
(unattractive)
A1B1
88
A2B1
24
56
Factor B2
(attractive
A1B2
16
A2B2
52
24
Column
means (Effect
of A)
52
28
Interaction effect: When the effect of one IV depends on the level of
another IV
Is there an interaction effect of facial expression and attractiveness
on the judgment of guilt?
Graph it!
Factor A1
(neutral face)
Factor A2
(smiling face)
Row Means
(Effect of B)
Factor B1
(unattractive)
A1B1
88
A2B1
24
56
Factor B2
(attractive
A1B2
16
A2B2
52
24
Column
means (Effect
of A)
52
28
Interaction effect: When the effect of one IV depends on the level of
another IV
Is there an interaction effect of facial expression and attractiveness
on the judgment of guilt?
Easier to see if graph it: guilt on y axis, facial expression on x axis
What if you have an interaction?
• Report it
• Consider the impact on the main effects that you
are observing
• What kind of interaction is it?
– Antagonistic (the IV’s reverse each other’s effects)
• Lines cross, main effect can be flat
– Synergistic (the IV’s reinforce each other’s effects)
• Steeper slopes
– Ceiling-effect (one variable has a smaller effect when
paired with higher levels of a second variable)
• Converging lines
Within Subjects Factorial Design
• 2 x 2 (each subject sees each of the 4 possible
conditions)
– E.g., Investigate whether the size of a handbag
impacts our perception of its weight
– Weight: 2 levels (heavy/light)
– Size (volume): 2 levels (small/large)
– Measure the apparent heaviness
– Want to have a measure from each participant for
each possible combination
Mixed Designs
• Some factors are between, some are within
• “A 2-factor mixed design was used: adaptive
accuracy (Low or High) was a between-subjects
factor and menu type (Control, Short-Onset or
Long-Onset) was a within factor. Order of
presentation was fully counterbalanced and
participants were randomly assigned to
condititions”
– Findlater et al., “Ephemeral Adaptation: The Use of
Gradual Onset to Improve Menu Selection
Performance” CHI 2009