Practice Matters —
How to appraise quantitative
research articles
By Kenneth J. Rempher, PhD, RN, MBA, CCRN, APRN, BC, and Cory Silkman, MAR, BSN, RN, C
WHATEVER
THEIR SPECIALTY
Don’t let the research process
intimidate you. Use these
guidelines to critically
appraise scientific studies.
or practice area, all nurses
should strive to become more
sophisticated consumers of
nursing research by learning
how to critically appraise, synthesize, and communicate research findings. Such critical appraisal shows your commitment to evidence-informed practice and empowers you to create a practice
culture based on the best available evidence.
To critically appraise nursing research, you must
ask focused, meaningful questions to determine the
overall integrity and applicability of the research.
This article will help you better understand—and undertake—the process of critical appraisal. With practice, even nurses who once were intimidated by the
research process should be able to efficiently and effectively determine the clinical relevance of scientific
studies. (See Questions to ask yourself when evaluating a scientific study.)
Problem statement
problem statement can cause
flaws in a study’s methods, protocols, samples, and analyses.
Be wary of broadly stated or
overly generalized problem
statements, as well as those
whose research questions can’t be answered by the
methods proposed. (See Parsing the problem statement.)
Literature review
The literature review is a systematic, critical review of
the most important scholarly literature on a given topic.
It should:
• highlight critical weaknesses in previous studies
• identify previously studied concepts or variables
• relate the current research project to historical research
• identify the current knowledge deficit about a particular phenomenon and state what more needs to be
done to overcome that deficit.
Look for a broad range of references—for example,
peer-reviewed journal articles, systematic reviews of
relevant research, professional standards, position statements, dissertations, and conference proceedings. In
general, references should be no more than 5 years
old, unless the research cited is a classic or historically
important work.
The problem statement should appear at the beginning
of the article and should include enough information for
you to determine if the study results can be generalized
to a specific patient population. Effective problem statements include independent and dependent variables, population of interest, and key concepts of the study.
The research problem should provide a clear rationale for the study.
The problem statement may be in
To better illustrate the importance of a clearly written problem statement, conone of two forms:
sider these two examples.
• a research question that indicates
the who, what, when, where, and
#1: “The primary purpose of this study was to determine the influence of clinical
why of the study
pathways on heart failure outcomes.”
• a “purpose” statement that de#2: “The primary purpose of this study was to assess the impact of a nurse-driven
scribes the researcher’s purpose in
clinical pathway on length of stay, costs, charges per patient day, documentation
of smoking cessation advisement, and administration of ACE inhibitors for
conducting the study.
Medicare patients diagnosed with left ventricular systolic dysfunction.”
Also, the research problem should
fill a gap in the current body of nursing
Statement #1 is broad and general and doesn’t identify the independent and deresearch or theory or should pinpoint a
pendent variables, population of interest, and key concepts being studied. Statesingle, relevant nursing issue that’s
ment #2 provides all relevant information, including dependent and independmeaningful to nurses and patients.
ent variables, population of interest, and key concepts.
A poorly worded or inappropriate
Parsing the problem statement
26 American Nurse Today
January 2007
Conceptual framework
Depending on the nature of the study, a conceptual or
theoretical framework may be presented near the beginning of the article.
• Theoretical frameworks are narrower in scope and
can be tested directly.
• Conceptual frameworks express assumptions and
can’t be tested directly.
A well-defined conceptual framework allows the reader
to better understand the relationship between major concepts of the study and more fully explicates the relationship
between the variables. Additionally, a conceptual framework may help you better understand how the researcher’s
hypothesis and research question were developed.
Methods
The methods section of the article tells you how the
principal investigator went about answering the research question. It includes information about the sample selection, study design, data collection, and data or
statistical analysis. This section should also provide sufficient information to permit duplication of the study
and should address the protection of human subjects.
Sample selection
Sample selection occurs on the basis of eligibility criteria that the researcher establishes in accordance with
the study’s objectives. The sample’s eligibility and exclusion criteria, as well as its demographic composition,
should be appropriate to achieve the study’s objectives.
The study should have an adequate number of participants and a low dropout rate to protect against compositional and statistical bias and make the study more
representative of the population.
Study design
The study design should be clearly stated and appropriate
for the research question being asked. The most common
design associated with quantitative research is experimental design—commonly considered the most rigorous design and, for many researchers, the gold standard. In this
design, the researcher controls both the selection of study
subjects and introduction of the independent variable.
Also, in experimental design, subjects are randomly
Questions to ask yourself when evaluating a scientific study
Use this checklist to help you determine if a study is valid and
applicable.
Problem statement or research question
■ Does the problem statement include the population of interest?
■ Does it identify key study concepts?
■ Does the research question fill a gap in the present body of
nursing knowledge or theory?
■ Does the question clearly define the relationship between the
variables?
■ Does the question address the who, what, where, when, and
why of the research study?
Literature review
Does the literature review:
■ highlight weaknesses of previous studies?
■ relate the current study to historical research?
■ address the study variables and hypotheses?
■ make evident the current deficit in knowledge about the phenomenon of interest?
■ state what needs to be done to overcome the existing deficit?
■ cover a broad array of resources?
■ cite studies no older than 5 years?
Conceptual or theoretical framework
■ Is the framework described?
■ Does it identify the relationship between the variables?
■ Does it support the study hypothesis?
Sample selection
■ What method was used to select the sample?
■ Do the eligibility and exclusion criteria help achieve the study’s
objectives?
■ Is the sample size appropriate and representative of the intended population?
■ Is the dropout rate significant enough to skew results?
Methods and design
■ Are all instruments used to collect data adequately described?
■ Is there evidence of validity and reliability for the instruments?
■ Is the design appropriate for the study?
■ Is the design sufficiently described so it could be easily replicated?
■ Are both descriptive and inferential statistics presented (if appropriate)?
■ Is the effect size presented?
■ Are the statistical tests used for analysis appropriate?
Protection of human subjects
■ Did the researcher obtain informed consent from subjects or
their legal representatives?
■ Did an institutional review board approve the study?
■ Do the study’s benefits outweigh its potential risks to participants?
Results and discussion of findings
Does the author:
■ relate statistical findings to the study’s independent and dependent variables?
■ clearly identify statistical tests and methods used to analyze
the data?
■ relate study findings to the larger conceptual or theoretical
framework?
■ adequately address the study’s limitations and weaknesses?
January 2007
American Nurse Today
27
assigned to treatment and control
groups, thereby reducing study bias
because the researcher can’t influence the assignment of subjects. Ideally, groups in an experimental study
Be aware that the “p” value tells you only that a difference exists between the
are similar in all respects, and any
experimental and control groups. It doesn’t tell you the magnitude of the effect on
differences between them result from
the experimental group. Understanding the magnitude of effect helps distinguish
the intervention administered by the
clinical significance from statistical significance. The article should report statistics
researcher. Experimental design is
known as the effect size and confidence intervals to help you determine the
commonly used when new drugs or
intervention’s magnitude of effect.
medical products are being studied.
In contrast, the nonexperimental
Protection of human subjects
study design is more qualitative and is used when the reTo protect research subjects, the study must be approved
searcher wants to observe a particular phenomenon but
by an institutional review board. The researcher also
lacks the ability and desire to manipulate the independent
must obtain informed consent from participants after givvariables. The quasi-experimental design is more closely
ing them oral and written information on the nature of
related to experimental design but lacks random assignthe study and potential safety risks or conflicts of interest.
ment of subjects and subsequently may introduce bias.
What the “p” value doesn’t tell you
Data collection
Data collection procedures should be fully explained in
the methods section and should provide a clear understanding of how data were collected and who collected
them. Issues such as inter-rater reliability, instrument
reliability and validity, and training of data collectors
should be addressed. A clear explanation of data collection lends credibility to the study.
Statistical analysis
A thorough review of the statistical analysis is important. (If you’re uncomfortable or unfamiliar with basic
statistics, consult a clinical nurse specialist, nurse researcher, or other advanced practice nurse for assistance.) Study results should be presented in a logical,
systematic format. For quantitative studies, both descriptive and inferential statistics should be provided.
• Descriptive statistics, including measures of central tendency (mean, median, and mode) as well as measures
of dispersion or variability (variance, standard deviation,
and range) provide information about the characteristics
of the subjects or phenomenon being studied.
• Inferential statistics allow researchers to make assumptions about the population based on a sample.
Through the use of significance tests and other
measures, inferential statistics help researchers understand the probability that the results of their
study occurred by chance. The level of significance,
expressed as a “p” value, represents the probability
of obtaining the computed value by chance. If “p” is
less than 0.01, the probability of obtaining the computed results by chance is less than 1%. In other
words, if the study were repeated 100 times, the difference between groups would be attributed to the
study intervention 99 times and to chance only one
time. (See What the “p” value doesn’t tell you.)
28 American Nurse Today
January 2007
Results and discussion of findings
Here the author should succinctly relate any descriptive
and inferential statistical findings to the study’s independent and dependent variables (as stated in the original research question or research hypothesis). This section
should state clearly whether the data analysis supports, or
fails to support, the research hypothesis. You should be
able to read the study’s results and determine the relationship between the independent and dependent variables.
Some articles may contain a separate “discussion” section in which the author interprets, analyzes, and summarizes the study’s conclusions and its relevance to the larger theoretical framework. Expect the author to objectively
state any limitations or weaknesses in the study’s design,
method, sample, or data collection procedures. The discussion also should identify any conceptual or theoretical
relationships in need of further investigation.
✯
Selected references
Dunning M, Abi-Aad G, Gilbert D. Experience, Evidence and Everyday Practice: Creating Systems for Delivering Effective Health Care.
London: King's Fund; 1999.
Greenhalgh T. How to read a paper. The Medline database. BMJ.
1997;315(7101):180-183.
Griffin-Sobel JP. Research in practice: immersing yourself in research.
Gastroenterol Nurs. 2003;26(5):219-220.
Hudson-Barr D. From research idea to research study: the how. J
Spec Pediatr Nurs. 2005;10(3):147-150.
LoBiondo-Wood J, Haber J. Nursing Research: Methods, Critical Appraisal, and Utilization. St. Louis, Mo: C.V. Mosby; 2002.
Valente S. Critical analysis of research papers. J Nurs Staff Dev.
2003;19(3):130-142.
Kenneth J. Rempher, PhD, RN, MBA, CCRN, APRN, BC, is Director of Professional
Nursing Practice at Sinai Hospital of Baltimore in Baltimore, Md. Cory Silkman,
MAR, BSN, RN, C, is a Clinical Leader in the Comprehensive Inpatient
Rehabilitation Unit at Sinai Hospital of Baltimore.