AS 3.12, Externally assessed, 4 credits
Experiments and Observational Studies
Do drugs have an
adverse effect on
teenage brains?
What information
would you need to
answer these
questions?
Does listening to
music help students
study?
What acne
treatments are the
most effective?
Article Structure
1. Introduction – Broad summary of story or main
finding. Slightly more detail than provided in
the heading.
2. Main angle developed – further details
provided
3. More on main angle
4. Strongest Quote
5. More details such as results of other studies or
quotes from other experts or witnesses
6. Background information
Who, What, Where, Why and How ? Identify in an article
What is an Experiment?
• Participants randomly
allocated to
treatments
• Variable of interest is
measured and results
compared between
treatment groups
Identify which are EXPERIMENTS ?
• Study which compares pulse rates of students with and without
exercise
• Study that compares performance in an IQ test with and without
caffeinated drinks
• Study of teenage binge-drinkers’ brain activity
• Impact of smoking during pregnancy on infant performance ability
• Effect of birth weight on adult academic performance
• Effect of acupuncture or physiotherapy on back pain
• Effect of exercises on age babies start walking
• Study of breakfast habits and study results
• Study of massage as a cure for migraines
• Are food colours linked to hyperactivity in children?
• Are defensive drivers better drivers?
• Effect of sleep deprivation on driving ( in a simulator)
KEY TERMINOLOGY
• Response Variables – the variable measured as the outcome of an
experiment. Measure blood pressure to assess effect of new drug.
• Explanatory Variables – the variable that attempts to explain
differences in the response variable
• Treatment- one or a combination of explanatory variables assigned by
the researcher
• Confounding Variables – variables connected to the explanatory
variable that may be the actual cause of the differences in the
response variable.
CONFOUNDING VARIABLES
“ Imported lemons reduces highway fatalities – study reveals”
Identify Explanatory, Response Variables and
Treatment in following studies
• Study which compares pulse rates of students with and without
exercise
• Study that compares performance in an IQ test with and without
caffeinated drinks
• Study of teenage binge-drinkers’ brain activity
• Impact of smoking during pregnancy on infant performance ability
• Effect of birth weight on adult academic performance
• Effect of acupuncture or physiotherapy on back pain
• Effect of exercises on age babies start walking
• Study of breakfast habits and study results
• Study of massage as a cure for migraines
• Are food colours linked to hyperactivity in children?
• Are defensive drivers better drivers?
• Effect of sleep deprivation on driving ( in a simulator)
How to design a good experiment
• Random Allocation – participants randomly allocated to
treatments and random order of treatment
• Use of a Control Group – use to establish base line effect
• Use of a Placebo – dummy, existing or no treatment
• Use of Blinding – single and double blinding
What is an Observational Study?
A study which observes differences in the explanatory
variable and then assesses whether these differences
are related to differences in the response variable
Famous NZ Observational Study is the Dunedin
Multidisciplinary Health and Development Study (DMHDS)
.A study of 1037 children who were born in Dunedin, in
1972-73. Where they now live shown below.
Potential Problems with Observational
Studies
Confounding Variables
“Happy People
Live Longer”
Possible confounding
variables?
PROBLEM – Causal Claim
cannot be made by
Observational Study
Why is this an observational
study?
Potential Problems with Observational
Studies
Extending Results inappropriately
“WOMEN HIT HARDER BY HANGOVERS”
The study observed only
undergraduate students at one
University so results cannot be
extended to cover ALL WOMEN
Potential Problems with Observational
Studies
Using the past as a source of data
“LEFT HANDED PEOPLE DIE
YOUNGER”
In early part of twentieth
century children forced to
write with right hands, hence
greater numbers of righthanders.
Confounding variables can
change over time.
Recalled information often
unreliable.
COMPARISON OF EXPERIMENTS AND
OBSERVATIONAL STUDIES
Experiments
Observational Studies
Explanatory Variable specified
Explanatory variable specified
Response variable specified
Response variable specified
Participants randomly allocated to treatments
Participants NOT randomly allocated to treatments,
sometimes because it is unethical to do so
Treatment is the only factor that is varied.
Changes in response variable measured
Changes in response variable observed
CAUSAL CLAIM CAN BE MADE
CAUSAL CLAIM CANNOT BE MADE
Claim can be made about study participants or similar
group ONLY
Good experiments include random allocation to
treatments, control groups, placebos and use
blinding
Good observational studies acknowledge & account
for all potential problems
EVALUATION OF STATISTICALLY BASED
REPORTS
Step One – Is report based on
sound research?
Step Two – Identify type of study –
experiment or observational?
Step Three – Who funded research?
Did they have vested interest in
outcome?
Step Four – Where reliable data
collection methods used?
Step Five – Who or what was
studied? How were they selected?
EVALUATION OF STATISTICALLY BASED
REPORTS
Step Six – Are measurements
provided?
Step Seven – Familiar or unfamiliar
settings?
Step Eight – Size of observed effect
quantified or just described?
Step Nine – Confounding variables?
Step Ten – Results extended
appropriately?
Step Eleven – Any missing data?
Step Twelve – Do findings make
sense?
EVALUATION OF STATISTICALLY BASED REPORTS
Does your evaluation persuade
you or someone you know to
modify their behaviour? What
is your overall conclusion?