Review
Bigger sample size (more data) will
A. Increase reliability
B. Decrease reliability
Review
Greater variability among individuals will
A. Increase reliability
B. Decrease reliability
Review
A bigger effect size in your data will
A. Increase reliability
B. Decrease reliability
Research Design
8/28
Overview
•
•
•
•
•
•
Components of scientific studies
Types of scientific studies
Inferring causation
Independent and dependent variables
Confounds, random assignment
Quasi-independent variables
Components of Scientific Studies
• Scientific study: Basic unit of empirical research
• Variables
– Anything that can take on multiple values
• Height, IQ, reaction time, extraversion, favorite color
– Measured in scientific studies
• Hypothesis
– Conjecture about how the world works
– Prediction about how variables relate
• Taller people are smarter
• This drug improves memory
• Blue is more popular than red
• Data (singular: datum)
– Results of measurements
– Values of variables
• IQ of Subject 4
• Reaction time of Subject 12 on Trial 23
Types of Scientific Studies
• Experiment
– Involves some sort of intervention or manipulation
– Researcher sets some variable(s) and assesses
effect on other variable(s)
• Vary number of items in a memory list
• Different drugs to different rats
– Allows inference of causation
• List length affects memory
• Drugs differentially affect lever pressing
• Non-experimental study
– Purely observational
– Measure naturally occurring variables and examine
relationships
• Row of classroom, exam grade
– Can't be sure about causation
Non-experimental Studies
• Measure variables without influencing
–
–
–
–
Row of classroom, exam grade
Time spent outside, depression
Bicycles currently owned, lifetime head injuries (6, 4)
Apples per week, colds per year
• Correlation
– Relationship between variables, in terms of what values co-occur
• More apples, fewer colds
• Smarter people tend to like the color red
– All that can be inferred from non-experimental studies
– Does not say what causes what
• Problems with inferring causation
– Reverse causation
– Third variable problem
– Self-selection
Reverse Causation
• Researcher expects X causes Y, but actually Y causes X
• Depression and time outdoors
– Might predict outdoors alleviates depression
– Might find such a correlation
– But, depression might reduce desire for activity
• XY or YX both mean X and Y co-occur
– If you only measure co-occurrence, can’t tell difference
• Solution: Intervention
– Manipulate X
– Any resulting effect on Y must be caused by X, not vice versa
Experiment Group
Time Outdoors
Depression
Depr(Outdoor Group) < Depr(Indoor Group)
Third-variable Problem
• X and Y might co-vary because they’re both caused by Z
• Apples and colds
– Overall health-consciousness could increase apples and reduce
colds
– People who eat more apples would also tend to get fewer colds
– But, no direct causal relationship
• Solution: Intervention (again)
– Manipulate X
– Shouldn’t affect Z
– Any effect on Y must be direct
Attitude
Apples
Experiment Group
Colds(Apple Group) < Colds(No-apple Group)
Colds
Self-selection
• Differences between groups of people can be due to who chooses
to be in which group
– Not necessarily consequence of group membership
• Math GREs by major
– Physics majors might do better than Psych
– Does physics make you better at math?
– Kids good at math more likely to choose Physics
• Height by sport
– Playing basketball makes you taller?
• Effects of alternative medicine
• Can view as reverse causation
– Being tall makes you better at basketball
• Can view as 3rd-variable problem
– Math aptitude affects both major choice and GRE
Experiments
• Independent variable (IV)
– Manipulated by researcher
– Drug/placebo, training time, priming
• Dependent variable (DV)
– Measured by researcher
– Pain tolerance, proficiency, reaction time
• Intervention assures causality
Attitude
X
Apples
Colds
Confounds and Control
• Importance of experimental control
– Only manipulate the IV
– Hold everything else constant
• Confound
– Variable that accidentally covaries with IV
– Subject expectations about drug effects
– Familiarity with experimental context
• Control means not having confounds
– Necessary for knowing effect is due to IV
Random Assignment
• Values of IV must be chosen at random for each subject
• Only way to assure causal relationship
• 3rd variable again
– Outright cheating
– Time of semester
Ability
Experiment
Group
Performance
Quasi-independent Variables
• Some variables can’t be manipulated, but can
be used to create groups
– Sex, age, birthplace
• Sometimes causal direction is obvious
– Height, men vs. women
– Hockey enjoyment, Canadians vs. Americans
• Allows non-experimental study to be treated like
an experiment
– Grouping variable is quasi-independent
– Can treat other variables like DVs
Review
Test rats in a maze, half in morning, half at night.
Measure how long each rat takes to learn.
Time to learn is a(n)
A.
B.
C.
D.
Variable
Hypothesis
Experiment
Datum
Review
Test rats in a maze, half in morning, half at night.
Measure how long each rat takes to learn.
Rats are smarter in the morning is a(n)
A.
B.
C.
D.
Variable
Hypothesis
Experiment
Datum
Review
Test rats in a maze, half in morning, half at night.
Measure how long each rat takes to learn.
The 3rd rat takes 5:30. This is a(n)
A.
B.
C.
D.
Variable
Hypothesis
Experiment
Datum
Review
Test rats in a maze, half in morning, half at night.
Measure how long each rat takes to learn.
You randomly decide which group each rat is in.
This is a(n)
A. Experiment
B. Non-experimental study
Review
Do the same with people, and let them decide which
time to sign up for.
This is a(n)
A. Experiment
B. Non-experimental study