Worksheet 1: Critical appraisal of qualitative research
Teacher answer key
Criteria
1) There is congruity
between the stated
philosophical
perspective and the
research
methodology.
Maliski Paper
Perspective is social constructionism (p 1610 introduction first
paragraph / p 1617 discussion second paragraph / p 1618
conclusion first paragraph)
Methodology is grounded theory approach (p 1611 methods
first paragraph).
Yes: Grounded theory is a methodology capable of capturing
perceptions and interpretations of people and to deepen
understanding of social processes influencing their
interpretations.
Note: constructionism deals with the ways in which
individuals and groups participate in the creation of their
perceived social reality and looking at the ways social
phenomena are created, institutionalized, and made into
tradition by humans. Reality is reproduced by people acting on
their interpretation and their knowledge of it.
Kelly Paper
Perspective is theory of embodiment (p 151 centrality of
embodiment) as opposed to social contructivism (p 152 first
column second paragraph)
Methodology used is ethnography (p152 Methods).
There is a degree of congruity: Ethnography is a methodology to
explore social meaning and culture. These concepts can be linked
with embodiment as a concept that goes beyond the physical
characteristics of the human body, although traditionally we would
expect embodiment to be initiated via phenomenology or
feminism.
Note: Embodiment is not a concept that pertains to the body
grasped as a physiological entity. Rather it pertains to the
phenomenal body and to the role it plays in our object-directed
experiences refers to the biological and physical presence of our
bodies, which are a necessary precondition for subjectivity,
emotion, language, thought and social interaction. Studies are not
'about' the body per se. Instead they are about culture and
experience insofar as these can be understood from the standpoint
of bodily being-in-the-world.
Ethnography is a methodological strategy used to provide
descriptions of human societies, which as a methodology does not
prescribe any particular method (e.g. observation, interview,
questionnaire) although it is often associated with observation and
‘writing up stories’. It is holistic, founded on the idea that humans
are best understood in the fullest possible context and focused on
tracing the doings of people.
2) There is congruity
between the research
methodology and the
research question or
objectives.
Objective of the study is development of a model of processes
used by low-income African/American Black and Latino men
to maintain masculine identity with prostate cancer treatment
related symptoms (p 1610 second column last paragraph /
model at p 1616)
Methodology used is grounded theory (p 1611 methods first
paragraph)
Both are mentioned together (p 1612 analysis first paragraph)
Yes: Grounded theory is a suitable methodology to develop a
theoretical model and new theory in general. It is inductive in
nature.
Objective of the study is to explore physical, emotional and social
(gendered) experiences associated with prostate cancer (p 152 first
column first paragraph).
Questions concerned impact of cancer on masculine embodiment,
changing concerns over the course of disease, managing the
embodied consequences and implications for service development
and future research (p 153 first column last paragraph).
Methodology is ethnography (p152 Methods).
There is a degree of congruity: Ethnography is mainly interested
in social aspects, which are part of the objective stated. The study
seems to focus more on perspectives and meaning. There have
been examples of studies taking an ethnographic approach, while
focussing on meaning. They are not common.
Note: Clifford Geertz’ fieldwork on winking used elements of a
phenomenological approach, tracing not just the doings of people,
but the cultural elements themselves. For example, if within a
group of people, winking was a communicative gesture, he sought
to first determine what kinds of things a wink might mean. Then,
he sought to determine in what contexts winks were used, and
whether, as one moved about a region, winks remained
meaningful in the same way.
3) There is congruity
between the research
methodology and the
methods used to
collect data.
Method used is interviews (p1611 data collection first
paragraph): in-person interviews and telephone interviews for
long distance participants (no differences found between the
groups)
Methodology used is grounded theory (p 1611 methods first
paragraph)
Yes: the research technique of interviews is suitable within a
grounded theory approach. It allows for achieving the degree
of depth aimed for.
Methodology used is ethnography (p152 Methods).
Method is interviews and observation (p 153 first column first
paragraph)
Yes: the research technique of interviewing and observing is
suitable within an ethnographic approach.
4) There is congruity
between the research
methodology and the
representation and
analysis of data.
Methodology used is grounded theory (p 1611 methods first
paragraph)
Yes: The description of the analysis matches the methodology
of grounded theory (p 1612 analysis). It is based on constant
comparison of codes and themes and appears to be a
continuing process, relationships developed were confirmed,
the model created emerged from the dataset. Attempts are
made to clarify concepts through additional contacts with the
research subjects (p 1612 first column last paragraph)
Note: it is not exactly clear from the paper whether data
collection and data analysis were concurrent. The statement
that part of the group has been recruited through advertisement
slightly indicates this, however it remains unsure (p1611
participants and setting first paragraph)
Methodology used is ethnography (p152 Methods).
Yes: The researcher documents the range of data sources accessed,
including interviews, observation of consultations and waiting
rooms, recent media accounts,…over a period of 18 months (p 153
first column first paragraph). Interviews appear to be analysed via
comparative analysis and observation used a theoretical
framework from Spradley (p 153 analysis)
5) There is congruity
between the research
methodology and the
interpretation of
results.
Methodology used is grounded theory (p 1611 methods first
paragraph)
Yes: The components used in the model are presented and
further explained in the results section. Deviant cases are
presented (ex. p 1613 second column last paragraph)
Note: The perspective of contructionism is visible in this
representation of findings.
Methodology used is ethnography (p152 Methods).
There is a low degree of congruity: The author uses a narrative
approach, drawing on findings retrieved from different
information resources (interviews, observation). The observational
data presented seem to be limited to one diagnostic situation only.
There is nothing on treatment decision making for which the
author stated that it different significantly (p 153 analysis last
paragraph). There is no clear understanding on how the recent
media accounts might have been used. Does he refer to papers
published by others that appear in the results section?
Note: The distinction between what is derived from the data and
what is purely interpretation from the author is flawed (e.g. p 156
first column first paragraph)
The importance of embodiment as a theory guiding data
representation is mentioned in the context of the categories
derived from the interviews (p 154 first column first paragraph).
6) There is a
statement locating
the researcher
culturally or
theoretically.
Unclear:
No explicit statement on beliefs or values of the researcher
where found.
Yes:
Initial interest in the topic emerged from clinical experience in
cancer care and as a man increasingly aware of other men facing
the reality of prostate cancer (p 152 first column first paragraph).
Note: It indicates the rationale of why the author came to write this
text and allows for evaluating self-awareness and reflexivity to
make judgments about the point of view.
7) The influence of
the researcher on the
research, and viceversa, is addressed.
Yes:
Teamwork: Analysis was undertaken by the first author in
collaboration with bicultural study team members and
participants (p 1612 analysis first paragraph).
Independent interviewer: All interviews were conducted by a
language and ethnicity concordant trained male interviewer.
Interpretative processes were carried out by five independent
researchers (p 1612 second column first paragraph)
Member check: Categories and relationships were confirmed
in follow-up interviews (p 1612 second column first
paragraph)
No:
Teamwork: Analysis of interviews was undertaken by the first
author in collaboration with two other researchers.
Interviewer: It is assumed but not clearly stated that the primary
researchers did the interviews himself.
Transcripts: It is unclear how the interviews were transcribed:
based on notes of the researcher of based on tape-recorded
interviews. As such it is hard to evaluate whether or not the
researcher has selected parts out of the interviews to be analysed
or not.
8) Participants, and
their voices, are
adequately
represented.
Unclear:
Words and quotes expressed by participants are included
between brackets.
Note: Quotes are secure for the theme masculine identity but
are fairly limited in the other parts, which rules out any chance
to evaluate the context in which they have been produced.
Yes:
The author clearly uses excerpts from his observational and
interview exercises to back up his statements.
Note: Quotes from other authors are used in the results section,
where we would expect them in the discussion section. It turns
reading the results section into a very confusing exercise and it is
quite uncommon in ethnography to account for others opinions.
9) The research is
ethical according to
current criteria or,
for recent studies,
there is evidence of
ethical approval by
an appropriate body.
Yes:
Institutional review board approval has been received (p 1611
participants and setting first paragraph)
Written informed consent was received (p1611 data collection
first paragraph)
Yes:
Ethical approval was granted by the research ethics committees of
the hospitals where the study took place (p 153 first column last
sentence).
Written consent was obtained from all participants (p 153 second
column first sentence).
10) Conclusions
drawn in the
research report do
appear to flow from
the analysis, or
interpretation, of the
data.
Yes:
Conclusions are in line with what the model developed
suggests. Men at risk are highlighted as well as how the
population differs from other populations (white males), based
on statements derived from the results section (as well as the
introduction).
Yes:
The author succeeds in generating conclusions about what is
happening and why. There is no incongruence between what is
reported in the discussion and in the result section.
Note: The promise made in the research questions on addressing
implications for future research is not fulfilled and the suggestions
for service development are also fairly limited.