Critiquing Quantitative
Research
Cynthia L. Russell, PhD, APRN, BC
There is no conflict of interest in the creation of this education program.
Learner Objectives
1. Describe 2 reasons for evaluating
quantitative research.
2. Identify 5 sections of quantitative
research reports.
3. Discuss the 5 components of the
methods section of a quantitative
research report.
Reasons for Evaluating
Quantitative Research Reports
•
To determine if the findings can be
used in practice
(Caelli, Ray & Mill, 2003)
Reasons for Evaluating
Quantitative Research Reports
•
To determine the current state of
science on a particular topic to direct
your next research study
Components of a Quantitative
Research Report
Quantitative research report components:
• -- Introduction/Research problem
•
•
•
•
•
-- Review of literature
-- Methods
-- Data analysis
-- Results
-- Discussion
(Caelli, Ray & Mill, 2003)
Introduction
• Describes the background of the
problem
– Prevalence and/or incidence of the
problem
– Impact of the problem on patient outcomes
– Importance of the problem to health care
providers and nurses, in particular
• Builds a case for the need for the study
• Develops the Research Problem
Research Problem
•
Broadly defines what needs to be
studied
– Flows directly from the introduction
– Concludes the Introduction section
(Caelli, Ray & Mill, 2003)
Review of the Literature
Summary of pertinent literature
Organized and clear
Evaluate quality of literature
Strengths and weaknesses
Organized from broad to specific
Identify gaps in literature
Need for the study should be clear
Discuss theoretical/conceptual framework
Methods
Describes the steps used by the
researcher to conduct the study
Design
Sampling plan
Instrumentation
Procedures
Protection of human subjects
Design
Describes the plan for carrying out the study
Non-experimental designs
Types
Descriptive
Correlational
Experimental designs
Types
Randomized control trials
Quasi-experimental designs
Sampling Plan
Describes the sample for the study
Who will participate
How they will be identified
Characteristics of the target population (the
population to which the findings are
generalized)
Sample procedure
Sample size
Instruments
Describes the instruments used to gather the
data for the study
Researcher should identify which instruments
measured which study concepts
May involve
Interviews
Questionnaires
Scales
Observation
Biophysiological measures
Instrument Reliability and
Validity
• Reliability is the instrument’s ability to
consistently measure the concept
• Validity is the instrument’s ability to
measure the concept that is supposed
to measure
Protection of Human
Participants
• Report should state that Institutional
Review Board Approval was obtained
• In the Procedure section the report
should state that informed consent was
obtained
Informed Consent
Why Am I Being Approached?
Why Is This Study Being Done?
How Many People Will Take Part In The
Study?
What Is Involved in the Study?
How Long Will I Be in the Study?
Informed Consent
What Are the Risks of the Study?
Are There Benefits to Taking Part in the
Study?
What Other Options Are There?
What about Confidentiality?
What Are the Costs?
Informed Consent
•
•
•
•
Will I be Paid for Participating in the Study?
What if I am Injured?
What Are My Rights as a Participant?
Whom Do I Call if I Have Questions or
Problems?
• Subject’s signature
• Signature of Study Representative
• If patient is incompetent to give consent,
additional steps are taken
Procedure
• Recipe for the research process
• Sufficient detail should be provided
• Should be written clearly and flow
logically
• All steps of the procedure should be
clearly stated
Data Analysis and Results
• Can be intimidating
• Confirm that the researcher has
presented results that clearly answer
the proposed research question(s)
• Researchers frequently organize this
section by research question to
facilitate readability
Data Analysis and Results
Review the research design
Descriptive design should use descriptive statistics
such as:
Mean, mode, and median (which measure how the data
tend to be similar or grouped together) and variance,
Standard deviation,
Range (which measure how the data tend to be spread out)
Correlational design should use correlational
statistics such as a correlation coefficient
(identifies the strength and the direction of the
relationship between two variables. Examples are:
Person r
Spearman rho.
Discussion
• Should clearly flow from the data
• Should place the study’s findings in context with
what is already known
• May be linked back to theoretical or conceptual
framework
• Author interpretations should be clearly identified
as such
• Reviewer must determine if the author’s
interpretations are justified
• Should present study limitations
• Should present implications for practice, as
appropriate
• Should present directions for future research
Study Limitations
Common study limitations include:
Small sample size
Sample limited to one area or one group of
people (this decreases generalizability to
other areas and other people)
Weak designs used such as correlational or
quasi-experimental
Checklist for Research Report
Evaluation
• Research Problem
– Is the problem clearly stated?
– Is the problem significant?
• Review of the Literature
–
–
–
–
–
Is the literature summarized?
Is the literature critically evaluated?
Are gaps and inconsistencies in the literature described?
Is the literature current and the review complete?
If presented, is the theoretical or conceptual framework
clearly described including concepts and relationships?
– Does the problem clearly link to and flow from the
theoretical or conceptual framework?
• Design
– Is the design clearly stated?
Checklist for Research Report
Evaluation
• Sample
– Is the sample clearly identified?
– Is it clear how the sample will be obtained?
– Is the relationship between the sample and the target population
clearly delineated?
– Is the rationale for the sample size provided?
• Instrumentation
–
–
–
–
Is it clear which instruments will measure which concepts?
Is the rationale for instrument selection acceptable?
Is the reliability for each instrument described and acceptable?
Is the validity for each instrument described and acceptable?
• Procedure
–
–
–
–
Are sufficient details provided in the procedure?
Is the procedure written clearly?
Does the procedure flow logically?
Are all steps of the procedure clearly stated?
Checklist for Research Report
Evaluation
Human Participants Protection
– Has the researcher provided sufficient protection
of human participants?
• Data Analysis and Results
– Is the data analysis section well organized?
– Is the statistical method used for analysis
appropriate for the research question(s) and/or
hypothesis and level of measurement?
– Are tables and graphs clearly labeled?
– Do the tables and graphs complement the text?
Checklist for Research Report
Evaluation
Discussion
– Does the discussion clearly flow from the data?
– Does the discussion place the study’s findings in context
with what is already known?
– If a theoretical or conceptual framework is presented, are
the nature of the findings discussed in the context of the
framework?
– If the author presents interpretations of the findings are
these clearly distinguished as such?
– Are justifications offered for the author’s conclusions?
– Are study limitations provided?
– Are implications for practice and future research
delineated?
Summary
• You may see yourself as a novice
reviewer and not trust your
interpretations.
• This is a common concern.
• Read, review, and discuss research
reports with others to increase your
skills.
References
• Alderman, S. (1998). Critiquing research for
use in clinical nursing practice: A CD-ROM
review. Nurse Educator, 23(2), 8.
• Beck, A. T., Steer, R. A., & Garbin, M. G.
(1988). Psychometric properties of the Beck
Depression Inventory: Twenty-five years of
evaluation. Clinical Psychology Review, 8(1),
77-100.
• Beck, A. T., Ward, C. H., Mendelson, M.,
Mock, J., & Erbaugh, J. (1961). An inventory
for measuring depression. Archives of
General Psychiatry, 4, 561-571.
References
• Beck, C. T. (1990). The research critique:
general criteria for evaluating a research
report. JOGNN, 19(1), 18-22.
• Beyea, S. C. (1998). Critiquing research for
use in clinical nursing practice (CD-ROM).
Computers in Nursing, 16(1), 16-17.
• Brink, P. J., & Wood, M. J. (2001). Basic
Steps in Planning Nursing Research (5th
ed.). Boston: Jones and Bartlett Publishers.
• Haughey, B.P. (1994). Evaluating quantitative
research designs: part 1. Critical Care Nurse,
14(5): 100-2.
References
• Haughey, B. P. (1994). Evaluating
quantitative research designs: part 2. Critical
Care Nurse, 14(6): 69-72.
• Holcomb, Z. C. (2002). Interpreting Basic
Statistics (3rd ed.). Los Angeles, CA: Pyrczak
Publishing.
• Pieper, B. (1993). Basics of critiquing a
research article. Journal of ET Nursing, 20,
245-250.
• Polit, D., & Beck, C. T. (2004). Nursing
Research: Principles and Methods (7th ed.).
Philadelphia, PA: Lippincott, Williams, &
Wilkins.
References
• Polit, D., & Beck, C. T. (2008). Nursing
Research: Principles and Methods (8th ed.).
Philadelphia, PA: Lippincott, Williams, &
Wilkins.
• Russell, C. L., Brown, K. (2002). The effects
of information and support on individuals
awaiting transplant. Progress in
Transplantation, 12(3), 201-207.
• Russell, C. L. (2005). Evaluating quantitative
research reports. Nephrology Nursing
Journal, 32(1), 61-64.
References
• Ryan-Wengar, N. M. (1992). Guidelines for
critique of a research report. Heart & Lung,
21(4), 394-401.
• Soeken, K. L. (1985). Critiquing research:
steps for complete evaluation on an article.
AORN Journal, 41(5), 882-893.
• Summers, S. (1991). Defining components of
the research process needed to conduct and
critique studies. Journal of Post Anesthesia
Nursing, 6(1), 50-55.