Worksheet 1: Critical appraisal of qualitative research Teacher answer key Criteria 1) There is congruity between the stated philosophical perspective and the research methodology. Maliski Paper Perspective is social constructionism (p 1610 introduction first paragraph / p 1617 discussion second paragraph / p 1618 conclusion first paragraph) Methodology is grounded theory approach (p 1611 methods first paragraph). Yes: Grounded theory is a methodology capable of capturing perceptions and interpretations of people and to deepen understanding of social processes influencing their interpretations. Note: constructionism deals with the ways in which individuals and groups participate in the creation of their perceived social reality and looking at the ways social phenomena are created, institutionalized, and made into tradition by humans. Reality is reproduced by people acting on their interpretation and their knowledge of it. Kelly Paper Perspective is theory of embodiment (p 151 centrality of embodiment) as opposed to social contructivism (p 152 first column second paragraph) Methodology used is ethnography (p152 Methods). There is a degree of congruity: Ethnography is a methodology to explore social meaning and culture. These concepts can be linked with embodiment as a concept that goes beyond the physical characteristics of the human body, although traditionally we would expect embodiment to be initiated via phenomenology or feminism. Note: Embodiment is not a concept that pertains to the body grasped as a physiological entity. Rather it pertains to the phenomenal body and to the role it plays in our object-directed experiences refers to the biological and physical presence of our bodies, which are a necessary precondition for subjectivity, emotion, language, thought and social interaction. Studies are not 'about' the body per se. Instead they are about culture and experience insofar as these can be understood from the standpoint of bodily being-in-the-world. Ethnography is a methodological strategy used to provide descriptions of human societies, which as a methodology does not prescribe any particular method (e.g. observation, interview, questionnaire) although it is often associated with observation and ‘writing up stories’. It is holistic, founded on the idea that humans are best understood in the fullest possible context and focused on tracing the doings of people. 2) There is congruity between the research methodology and the research question or objectives. Objective of the study is development of a model of processes used by low-income African/American Black and Latino men to maintain masculine identity with prostate cancer treatment related symptoms (p 1610 second column last paragraph / model at p 1616) Methodology used is grounded theory (p 1611 methods first paragraph) Both are mentioned together (p 1612 analysis first paragraph) Yes: Grounded theory is a suitable methodology to develop a theoretical model and new theory in general. It is inductive in nature. Objective of the study is to explore physical, emotional and social (gendered) experiences associated with prostate cancer (p 152 first column first paragraph). Questions concerned impact of cancer on masculine embodiment, changing concerns over the course of disease, managing the embodied consequences and implications for service development and future research (p 153 first column last paragraph). Methodology is ethnography (p152 Methods). There is a degree of congruity: Ethnography is mainly interested in social aspects, which are part of the objective stated. The study seems to focus more on perspectives and meaning. There have been examples of studies taking an ethnographic approach, while focussing on meaning. They are not common. Note: Clifford Geertz’ fieldwork on winking used elements of a phenomenological approach, tracing not just the doings of people, but the cultural elements themselves. For example, if within a group of people, winking was a communicative gesture, he sought to first determine what kinds of things a wink might mean. Then, he sought to determine in what contexts winks were used, and whether, as one moved about a region, winks remained meaningful in the same way. 3) There is congruity between the research methodology and the methods used to collect data. Method used is interviews (p1611 data collection first paragraph): in-person interviews and telephone interviews for long distance participants (no differences found between the groups) Methodology used is grounded theory (p 1611 methods first paragraph) Yes: the research technique of interviews is suitable within a grounded theory approach. It allows for achieving the degree of depth aimed for. Methodology used is ethnography (p152 Methods). Method is interviews and observation (p 153 first column first paragraph) Yes: the research technique of interviewing and observing is suitable within an ethnographic approach. 4) There is congruity between the research methodology and the representation and analysis of data. Methodology used is grounded theory (p 1611 methods first paragraph) Yes: The description of the analysis matches the methodology of grounded theory (p 1612 analysis). It is based on constant comparison of codes and themes and appears to be a continuing process, relationships developed were confirmed, the model created emerged from the dataset. Attempts are made to clarify concepts through additional contacts with the research subjects (p 1612 first column last paragraph) Note: it is not exactly clear from the paper whether data collection and data analysis were concurrent. The statement that part of the group has been recruited through advertisement slightly indicates this, however it remains unsure (p1611 participants and setting first paragraph) Methodology used is ethnography (p152 Methods). Yes: The researcher documents the range of data sources accessed, including interviews, observation of consultations and waiting rooms, recent media accounts,…over a period of 18 months (p 153 first column first paragraph). Interviews appear to be analysed via comparative analysis and observation used a theoretical framework from Spradley (p 153 analysis) 5) There is congruity between the research methodology and the interpretation of results. Methodology used is grounded theory (p 1611 methods first paragraph) Yes: The components used in the model are presented and further explained in the results section. Deviant cases are presented (ex. p 1613 second column last paragraph) Note: The perspective of contructionism is visible in this representation of findings. Methodology used is ethnography (p152 Methods). There is a low degree of congruity: The author uses a narrative approach, drawing on findings retrieved from different information resources (interviews, observation). The observational data presented seem to be limited to one diagnostic situation only. There is nothing on treatment decision making for which the author stated that it different significantly (p 153 analysis last paragraph). There is no clear understanding on how the recent media accounts might have been used. Does he refer to papers published by others that appear in the results section? Note: The distinction between what is derived from the data and what is purely interpretation from the author is flawed (e.g. p 156 first column first paragraph) The importance of embodiment as a theory guiding data representation is mentioned in the context of the categories derived from the interviews (p 154 first column first paragraph). 6) There is a statement locating the researcher culturally or theoretically. Unclear: No explicit statement on beliefs or values of the researcher where found. Yes: Initial interest in the topic emerged from clinical experience in cancer care and as a man increasingly aware of other men facing the reality of prostate cancer (p 152 first column first paragraph). Note: It indicates the rationale of why the author came to write this text and allows for evaluating self-awareness and reflexivity to make judgments about the point of view. 7) The influence of the researcher on the research, and viceversa, is addressed. Yes: Teamwork: Analysis was undertaken by the first author in collaboration with bicultural study team members and participants (p 1612 analysis first paragraph). Independent interviewer: All interviews were conducted by a language and ethnicity concordant trained male interviewer. Interpretative processes were carried out by five independent researchers (p 1612 second column first paragraph) Member check: Categories and relationships were confirmed in follow-up interviews (p 1612 second column first paragraph) No: Teamwork: Analysis of interviews was undertaken by the first author in collaboration with two other researchers. Interviewer: It is assumed but not clearly stated that the primary researchers did the interviews himself. Transcripts: It is unclear how the interviews were transcribed: based on notes of the researcher of based on tape-recorded interviews. As such it is hard to evaluate whether or not the researcher has selected parts out of the interviews to be analysed or not. 8) Participants, and their voices, are adequately represented. Unclear: Words and quotes expressed by participants are included between brackets. Note: Quotes are secure for the theme masculine identity but are fairly limited in the other parts, which rules out any chance to evaluate the context in which they have been produced. Yes: The author clearly uses excerpts from his observational and interview exercises to back up his statements. Note: Quotes from other authors are used in the results section, where we would expect them in the discussion section. It turns reading the results section into a very confusing exercise and it is quite uncommon in ethnography to account for others opinions. 9) The research is ethical according to current criteria or, for recent studies, there is evidence of ethical approval by an appropriate body. Yes: Institutional review board approval has been received (p 1611 participants and setting first paragraph) Written informed consent was received (p1611 data collection first paragraph) Yes: Ethical approval was granted by the research ethics committees of the hospitals where the study took place (p 153 first column last sentence). Written consent was obtained from all participants (p 153 second column first sentence). 10) Conclusions drawn in the research report do appear to flow from the analysis, or interpretation, of the data. Yes: Conclusions are in line with what the model developed suggests. Men at risk are highlighted as well as how the population differs from other populations (white males), based on statements derived from the results section (as well as the introduction). Yes: The author succeeds in generating conclusions about what is happening and why. There is no incongruence between what is reported in the discussion and in the result section. Note: The promise made in the research questions on addressing implications for future research is not fulfilled and the suggestions for service development are also fairly limited.