Why should we trust scientists
about food and health?
The value of observational
studies
Professor Kay-Tee Khaw FMedSci
Identifying the environmental causes of
disease: how should we decide what to
believe and when to take action?
A report of an Academy of Medical Sciences working group
Main outputs
1.Five key recommendations
2. Detailed guidelines for:
• Researchers
• Editors
• Science or medical writers
• Clinicians and healthcare practitioners
• Policy managers
• Funders
Cartoon that illustrates the confusion that sometimes
accompanies research into the causes of disease.
Why are we interested in causes?
1.Explanation
Why did this happen?
2.Prediction
Can we predict what is likely to happen in future?
3. Alter the course of events
Can we change outcome?
4.Action: changes to policy and practice
Key research implications
1. Many environmental influences e.g. diet, are
not open to experimental manipulation for
practical and ethical reasons.
2. We have to rely mainly on non-experimental
research strategies
3. Possible explanations for associations apart
from causality include chance, confounding or
bias.
Meat consumption and colon cancer
incidence in various countries
Armstrong and Doll (1975) Environmental factors and cancer incidence in
different countries. International Journal of Cancer Research. 15, 617
Pairs of nesting storks and births over
time in Sweden
From association to cause
The Bradford Hill guidelines provide a useful framework for
interpreting whether associations are likely to be causal:
•
•
•
•
•
•
•
•
•
Strength
Consistency
Specificity
Temporality
Biological gradient
Plausibility
Coherence
Experiment
Analogy
Bradford Hill A (1965) The environment and disease: association or causation?
Proceedings of the Royal Society of Medicine, 58, 295-300.
Main varieties of non-experimental
research
1. Case-control studies
2. Prospective cohort studies
3. Ecological designs
4. Natural experiments
May be combined with animal models
and
randomised controlled trials
Main methods of the working group
Examples of associations from non-experimental
research which we still believe are causal
•
•
•
•
Smoking and lung cancer.
Lipids and coronary artery disease.
Folic acid and neural tube defects.
Foetal alcohol syndrome.
Examples of associations from non-experimental
research which we no longer believe are causally
related
• Caffeine in pregnancy.
• Antioxidant Vitamin supplements and mortality.
• Early alcohol use and later alcohol abuse or
dependency.
Key features of associations from
observational studies for which
causality is accepted as robust
1. Very large effect or rare and unusual outcome.
2. Detailed attention to alternative non-causal explanations.
3. Multiple research designs (including natural
experiments).
4. Causal inference tested in multiple populations and
settings differing in risk characteristics.
5. Confirmatory findings from animal models and human
experiments pointing to likely mediating biological
processes.
Key lessons from observational
studies in which associations have not
been subsequently substantiated
1. Most misleading findings result from small scale, poor quality
studies, often not subjected to peer review, nor published in
scientific journals, and not replicated.
2. Non-experimental studies are especially susceptible to bias
when there are selection effects stemming from personal
choices e.g. lifestyle, supplements.
3. Particular caution when major interests are concerned e.g
commercial considerations or strongly held clinical views.
4. Causal claims, even from good studies, need to be based on
similar findings from multiple studies using varying designs.
Main conclusions
Randomised controlled trials can minimise but not exclude
confounding and bias
Most of what we do is not supported by evidence from
randomized controlled trials for ethical and feasibility
reasons and generalizability sometimes an issue
We usually have to make policy relying on on other types of
research design and understand strengths and limitations
when interpreting and applying findings
Judgement as to what action is justified by available
evidence: quantitative assessments/ sensitivity analyses of
consequences of different options
Always uncertainty: new cohorts, new exposures,
factors, new interactions: need for continuing
surveillance and research: dynamic process
‘All scientific work is incomplete – whether it be
observational or experimental. All scientific work is
liable to be upset or modified by advancing
knowledge. That does not confer upon us a freedom
to ignore the knowledge we already have or to
postpone the action that it appears to demand at a
given time.’
Sir Austin Bradford Hill CBE FRS, 1965