Andrea M. Landis, PhD, RN
UW LEAH
December 7, 2012
Learning Objectives
 Discuss the importance of critically examining the
literature.
 Review key issues for evaluating the literature.
 Explore in detail each section of a manuscript.
 Appraise systematic review article.
Sources:
“How to read a paper” (2010) Greenhalgh, T. [ebook]
Centre for Evidence Based Medicine (Oxford)
UW Health Sciences Library - Toolkit
Before any research project review
the literature relevant to your RQ:
 Gain a full and in-depth understanding of a subject.
 See if your intended research subject has been done before




and avoid duplication.
Avoid any errors made in similar research.
Enable you to place your study within its context (ie so that
you can show how your research will add to the existing
sum of knowledge).
Provide you with ideas to help you define or amend your
own research topic.
Provide you with information with which to compare and
contrast your findings.
Introduction
 One’s ability to review articles is something that
requires practice and experience.
 Process can be aided by following a checklist of things
to look out for and comparing the paper under review
to the criteria.
 The type of research will effect the information you are
evaluating.
 Standard journal format subheadings for research
reports: title, abstract, introduction, method, results,
discussion and conclusion.
Summary of key issues that need
to be evaluated (APA, 1983):
 Is the RQ question significant and is the work original and






important?
Have the instruments used been demonstrated as reliable
and valid?
Do the outcome measures relate clearly to the variables
with which the investigation is concerned?
Does the research design fully test the hypothesis?
Are the subjects representative of the population to which
generalizations are made?
Did the researcher observe ethical rules?
Has the research reached such a stage that publication is
justified and the results are meaningful?
Title
 Very important.
 Short and informative.
 Gives insight into what (was done), whom (it was done
to) and how (it was done).
 Gimmicky?
Author
 Some idea of the author(s) academic background and
ability to carry out valid research
 job title, qualifications, and where they work.
 Corresponding author
Abstract
 The abstract should contain a brief statement about
the study's purpose, method, results, conclusion and
clinical relevance.
 Time-efficient way for readers to determine if the
article suits their needs – return to the article later.
 Do not to accept the conclusions before critically
reading the entire article.
Source
 Assessment of where the article was published should
give some clues as to its potential value.
 Is it a peer reviewed journal?
Introduction & Statement of the
Problem
 Research problem/clinical question should be defined clearly.
 Expect to find clear descriptions of the research aims, an outline
of theoretical issues and the hypothesis should be introduced.
 Information should include the current state of knowledge about
the research topic and an indication of the gaps in knowledge
which the current study will hope to fill.
 “Why was the research done?”
 Statement of the Problem: should describe the questions and
concerns that led the author to undertake the investigation.
 “What question did the author try to answer?”
Literature Review → Purpose Statement
 The literature review should establish a theoretical and historical




basis
Survey of current knowledge highlighted by a thorough review of
the existing literature.
Identify gap of knowledge between what is known (or
previously documented) and what is desired to be known. Find
information in the literature that supports the concept and
approach of the study.
Attempt close the gap by explaining why the study was
conducted. Or point out flaws, inconsistencies or areas where
no conclusions can be drawn.
Issues:
 Up to date references
 Unbiased
 The purpose of the study should be described in a direct, clear
statement.
Methodology
 Clearly explain how the study was conducted.
 Critical readers should pretend they are going to
replicate the study:
 Is there sufficient detail in the method to conduct the
study and obtain similar results?
 Was the design of the study sensible?
 Method can be divided into the following subsections:
subjects, instrumentation and apparatus, procedure,
and data analysis.
Subjects
 Summarize and describe the subjects who participated in the study in
terms of age, sex, diagnosis and other pertinent demographic
characteristics (Table).
 Inclusion/Exclusion criteria.
 How were they recruited?
 The extent to which readers are able to use the results (generalization)
of the study depends on how the sample of subjects was selected:
 Randomly selected
 Sample of convenience
 RCT - allocation of subjects to experimental and control groups
 How many subjects were included in the sample:
 Small vs. Large
 Impact on power analysis
 Diversity (race/ethnicity reported)
 IRB approval (includes confidentiality and anonymity assured)
Instruments or Apparatus
 Any special equipment or instruments (e.g. questionnaires)
should be described.
 Indicate validity and reliability.
 Described in such a way that readers can replicate the
study.
 Specify model numbers, corporate names and addresses,
and other pertinent details about the instruments.
 Any apparatus (or questionnaire) designed and developed
by the researcher should be fully described with a drawing,
photograph and description.
 Were the instruments calibrated? How were they
calibrated? Are they repeatable day-to-day?
Procedures
 How and when the steps of the study were applied.
 How the data were collected.
 Internal validity - changes noted during the study are the
result of the devices being studied and not the result of a
sloppy procedure.
 Did the experimental treatment cause the observed
change in the dependent variable?
 Could other (extraneous) factors be responsible for that
change?
 Has the investigator taken steps to improve their internal
validity or control sources of secondary variance.
Data Analysis
 Section should describe all testing applied to the data
 NO results.
 Did authors chose the appropriate statistical tests for the
type of study and design.
 When analyzing data, arithmetic operations too frequently
are misapplied to data based on levels of measurement.
 Common error - analyzing ordinal data as though they were
quantitative (interval or ratio).
 Did authors screen data for errors in data entry, outliers
and distribution.
 Conventional parametric statistical analyses are conducted
on continuous data.
 descriptive, comparative, associative and predictive.
Results
 Reports what has been discovered.
 Reported factually and formally without commentary.
 Summary statistics may be presented in tables/figures.
 Statistical tests and measures used should be
described allowing the reader to evaluate whether the
appropriate tests were applied.
 Level of significance
 Statistical versus clinical significance
 Do the authors mention all the relevant results? - even
those that actually go against the hypothesis.
Discussion
 The issues raised by the findings should be discussed and
resolved in this section.
 Should relate back to the literature/aims of the research as
outlined in the introduction.
 The author is expected to examine, interpret and qualify
the results and draw any inferences from them.
 Large enough/long enough/followed-up enough to be
credible
 Is the research question answered? Has the author given
meaning to the results?
 Has the author considered broader implications of his/her
findings?
Conclusion & Recommendation
 Brief restatement of the experimental results and
describes the implications of the study.
 The paper should end with some conclusions about
the importance (or otherwise) of the findings.
 The author should not make any statements here
which are not supported by the facts found.
 Recommendations on the basis of the findings are
often stated here.
 comments on possible improvements
 future areas for more study.
References
 Consistent citing of references.
 Endnote, RefWorks
 Appropriately extensive and up-to-date.
Questions
1.
2.
3.
4.
5.
6.
What question did the systematic review address?
Is it unlikely that important, relevant studies
were missed?
Were the criteria used to select articles for inclusion
appropriate?
Were the included studies sufficiently valid for
the type of question asked?
Were the results similar from study to study?
What were the results? How are the results
presented?