Introduction to selecting and using
critical appraisal tools/frameworks
How to interrogate research papers
Alan Glasper and Colin Rees
How to Write Your Nursing Dissertation, First Edition. Alan Glasper and Colin Rees.
© 2013 John Wiley & Sons, Ltd. Published 2013 by John Wiley & Sons, Ltd.
Critical appraisal tools/frameworks
• Sue is meeting with her
supervisor to discuss
the selection of an
appropriate critical
appraisal tool to
evaluate her selected
papers.
What is critical appraisal? What are
critical appraisal tools?
• Critical appraisal is the
process of carefully and
systematically examining
research to judge its
trustworthiness, value and
relevance in a particular
context.
Health care practitioners need
to:
• Decide if the research being
reported in the papers they
read has been conducted in
such a way to ensure that
the finding are both valid
and reliable
• Be able to understand and
make sense of the results
• Decide if the research is
strong enough to suggest
changes to practice.
What is the best critical appraisal tool to use?
• There are many different
critiquing tools available, some
target specific types of study (e.g.
qualitative) and some are generic
and can be used to appraise any
paper.
• Critical appraisal is the systematic
and unbiased detailed
examination of all the reported
elements of a published paper or
study to allow judgement of both
the merits or strengths and the
weakness or limitations in order
to facilitate both the meaning
and relevance to practice (Burns
and Grove 2009).
• The University of South Australia
International Centre for Allied
Health Evidence
(http://www.unisa.edu.au/cahe/R
esources/CAT/default.asp) offers
a full and comprehensive
discussion of the attributes of
many popular health care
critiquing tools. Additionally the
Scottish Intercollegiate Guidelines
Network (SIGN) has developed
evidence based clinical practice
guidelines for the National Health
Service
(http://www.sign.ac.uk/methodol
ogy/checklists.htm).
What is the best critical appraisal tool to use?
Although there are many
critiquing tools available,
this presentation will only
consider critiquing tools
which have been
developed by:
• Crombie
• CASP
• Parahoo
• Rees
Sam supervisor has advised him to use
Parahoo’s critiquing approach, whereas
Sue has been advised to use Rees’
model.
Using highlighter pens
• Sue has been advised by
Sam to buy a large pack
of children’s highlighter
pens including some
exotic fluorescent
colours. She now
believes that this will be
useful as she now has a
different colour for each
of the critiquing tool
questions.
Riley (1996) first used
highlighter pens used as a
method of identifying
commonalities in
transcribed qualitative
interviews.
Highlighter
pens
Using colour highlighter pens
Sam gave me
a good tip
Using colour highlighter pens
• Have at least three or four copies
of each your selected papers, one
to carry with you in your
handbag, briefcase or rucksack to
read on the bus, train or plane or
in your coffee break, one to
identify specific details using the
colour highlighter pens. (Some
students like to use one copy of
the papers to cut up with scissors
after highlighting and reassemble
piece by piece to match the
criteria of the specific critiquing
tool they are using). Finally, keep
a spare copy for insurance – just
in case!
glad the
colours
work for
you
Using Savage and Callery grids to undertake a
preliminary critique
Author
(Year)
Country
Aim(s) of study
Methodological Issues
Sample
Relevant/key findings
Design, data collection
and analysis,
rigour/reliability and
validity
The Crombie Model of critiquing
1 Why was it done? (Objectives)
What was the rationale for undertaking this study?
What was the purpose of the study?
Was the research question clearly stated?
Crombie’s framework can help practitioners
think about the process of critiquing a
research paper (qualitative or quantitative)
2 How was it done? (Methods)
Was the research design appropriate to address the
research question?
How was the sample selected?
How was the data collected?
How was the data analysed?
Were ethical issues discussed?
3 What has it found? (Results)
Are the findings clearly presented?
Do the findings answer the research question?
4 What are the implications? (Conclusions)
Were the findings discussed?
Are implications for future practice summarized?
Sam and his supervisor discuss a
range of models
The CASP models of critiquing
• The Critical Appraisal Skills
Programme (CASP) has
been designed by Solutions
For Public Health
http://www.sph.nhs.uk/wh
at-we-do/public-healthworkforce/resources/critical
-appraisals-skillsprogramme and provides 7
different critiquing tools,
each designed for different
types of study and these are
freely available to download
from their website.
Yes I am
using
CASP
The Parahoo model of critiquing
Title of study
Does the title convey the study clearly and accurately?
Abstract
Does the abstract give a short and concise summary of the following
aspects of the study?
Background
Aim
Designs
Results
Conclusions
Literature review/Background
Is the importance of study justified?
What is the context of this study?
Does the literature review show the gap/s in knowledge which this study
seeks to fill?
Aims/objectives/research questions/hypotheses
Are the aims of the study clear?
Design of study
What is the design of the study? Is it the most appropriate for the aims of
the study?
Are the main concepts (to be measured) defined?
What are the methods of data collection? Are they constructed for the
purpose of the current study or do the researchers use existing ones?
Who collected data? Can this introduce bias in the study?
In studies where there are more than one group, is there a description of
what intervention/treatment each group receives?
Is the setting/s where the study is carried out adequately described?
Who was selected? From what population were they selected? What
was the precise method of selection and allocation? Was there a sample
size calculation?
Was ethical approval obtained? Are there any other ethical implications?
•
Data analysis
Was there a separate section in the paper that explained the planned
analyses prior to the presentation of the results?
Which statistical methods were relied on?
Is it clear how the statistical tests were applied to the data and groups?
Results
Are the results clearly presented?
Are the results for all the aims presented?
Are the results fully presented?
Discussion
Is it a balanced discussion? Has all possible explanations for the results
given?
Are the results discussed in the context of previous studies?
Are the results fully discussed?
Are the limitations of the study discussed?
Are the results discussed in the context of previous studies?
Are the results fully discussed?
Are the limitations of the study discussed?
Conclusions/Recommendations
Are the conclusions justified?
Are there recommendations for policy, practice or further research?
Are the results/conclusions helpful for my practice?
Are the results generalisable?
Funding
Is there potential conflict of interest (if information on funding is
provided)?
The Rees (2011) model for critiquing
quantitative and qualitative research
•
•
•
•
•
•
•
•
•
Focus
Background
Aim
Study design
Data collection method
Ethical considerations
Sample
Data presentation
Main findings
• Conclusion and
recommendations
• Readability
• Practice implications
NB Depending on the level of the
course i.e. undergraduate or post
graduate, the amount of
information concerning statistical
information will vary but for all
dissertations, at least a
preliminary understanding of the
tests used by the researchers is
mandatory.
Assessing the strengths and
weaknesses of critiqued papers
Study
1
2
3
4
Strengths
Limitations
• Sue and Sam have
completed their critiquing
chapter of their
dissertations. They are
now in the final aspects
of their dissertations and
are planning to write a
short chapter summing
up the relative strengths
and weaknesses of the
papers they have read
and comprehensively
critiqued.