Reading Science
Critically
Debi A. LaPlante, PhD
Associate Director, Division on Addictions
First Sources

Reading primary sources can be daunting
– Complexity of information
– Researchers are marketing their ideas and
findings
– Time

Benefits
– Current findings
– Promotes and enables replication
– Data (often)
What is the Purpose of
Scientific Papers?
Concisely report information, ideas, and
innovation
 Build the common knowledge-base
 Contribute to scientific debate


Resume building
Why is important to
read science critically?

Peer-review is state of the art, but
imperfect
– Author bias
– Unintentional errors
– Conflicts of interest
– Author self-marketing
More challenges to understanding
and evaluating scientific literature
Writing by scientists, not writers
 Marketing: Trojan Ns
 Marketing: Assertive Sentence Titles
 Statistical versus Clinical significance
 Publication bias

– Tough to publish negative results
Finding Articles
Citation lists of published papers
 Select journals’ table of contents
 Specialized search engines (e.g., Medline;
PsycInfo)
 Web searches (e.g., Google Scholar)
 Personal referrals
 Citation indexes (e.g., Social Science
Citation Index)

Components of Scientific Papers


Abstract
Introduction
– Hypotheses or research questions

Methods
– Participants
– Materials
– Protocol


Results
Discussion
– Interpretation of results
– Advances
– Limitations

Conclusion
How to get through a paper
Strategy depends on expertise
 General approach:

– Don’t read straight through
– Read title and abstract
– Skim Intro
– Read results
– Track back to Methods
– Read Discussion
Resources

Literature summary services
– www.basisonline.org
– http://www.cesar.umd.edu/cesar/cesarfax.asp
Greenhalgh (1997)
http://www.bmj.com/cgi/content/full/315/
7103/305
 Zaccai (2004)
http://pmj.bmj.com/content/80/941/140.f
ull.pdf

Is the study original?

Does the research advance what we
know?
– Bigger, longer, more substantial?
– More rigorous?
– New population?
– Will it inform or change clinical practice?
Greenhalgh (1997)
Whom is the study about?

What was the recruitment method?
– Representative and generalizable?
 Refusal rate? Homogeneity? Random?

What are the inclusion criteria?
– Disorder severity

What are the exclusion criteria?
– Co-existing illness, other medication, English,
literate

How “true to life” is the study setting?
Greenhalgh (1997)
Is the design sensible?

What was done?
– Appropriate comparison groups?
What was the measured outcome?
 Is there a sufficient description of the
design?

Greenhalgh (1997)
Ambiguous Research Methods
Author Statement
Actual Method
Problem
“We measured how often
GPs ask patients whether
they smoke.”
“We looked in patients’
medical records and
counted how many had had
their smoking status
recorded.”
Assumes medical records
are 100% accurate.
“We measured how doctors
treat low back pain.”
“We measured what doctors
say they do when faced
with a patient with low back
pain.”
Assumes that self-report
reflects behavior.
“We compared a nicotinereplacement patch with
placebo.”
“Subjects in the intervention Fails to provide adequate
group were asked to apply a methodological details.
patch containing 15 mg
nicotine twice daily; those in
the control group received
identical-looking patches.”
Greenhalgh (1997)
Is systematic bias avoided
or minimized?

Designs
– Randomized trials
– Non-randomized trials
– Cohort studies
– Case studies

Methods
– Blind assignment and assessment
– Validated measurement tools
– Control confounding (e.g., baseline group
differences)
Greenhalgh (1997)
Autumn Season
Falling Leaves
Student arrival to campus
Confounder of the
Season-Falling Leaves
relationship
Are the results credible?
Is there a sufficient sample size?
 Are the results clinical significant?
 How long is follow up?

– Is the follow-up appropriate to the outcome?
(e.g., post-operative pain versus pediatric
growth patterns)
– What is retention rate?
Greenhalgh (1997)
Take Away Messages
First Source publications provide important
benefits to science and practice
 Unintentional and intentional errors occur
 Readers should read critically and not
merely take such publications at face
value
