Qualitative Approaches for Food Security Assessment 687318203 1.3. Qualitative Assessment Methods Session-at-a-glance Content Approximate Time (Minutes) Instructional Activity Fruit Salad 10 Energizer Exercise Session Introduction 5 Presentation In-Depth Look at Qualitative Assessment Methods 30 Group Work Qualitative Methods Compared 45 Group Presentations and Plenary Discussion Total Time 90 minutes / 1hour 30 minutes Session Objectives After the session, participants will be able to: Describe and compare the most common qualitative data collection methods; Know characteristics of each method of data collection; and, Demonstrate an understanding of when to use the different qualitative methods. Session Supplies Power-point 1.3: Qualitative Assessment Methods Handout 1.3: Qualitative Methods.doc, copied to all participants Flipchart and markers for three groups Key Messages The main messages that need to be stressed are the following: Page 1 of 10 Qualitative Approaches for Food Security Assessment 687318203 There are three main qualitative collection methods: observations, discussions and interviews. Each of these methods can be structured in several ways. o Interviews can be exploratory (unstructured), structured and semistructured. o Discussions can take the form of community or focus group discussions. o Observations can be structured or unstructured. They can be participant observations (usually long) or direct observations. Each method and its sub-method has its advantages and disadvantages. The use of more than one method in qualitative assessments gives greater depth to the information collected and allows for triangulation and verification of information and data gathered. A food security assessment team will typically use a mixture of interview and discussion techniques, combined with direct observation. Facilitator Guidance and session preparation This session discusses the methods used in qualitative assessments. Facilitators are strongly encouraged to review TGS#9 (pages 2-5). The following external references can be consulted as well. Journals, Handbooks and Articles Adler, P. A. and P. Adler (1994). Observational Techniques. Handbook of Qualitative Research. N. K. Denzin and Y. S. Lincoln. London, Thousand Oaks, New Delhi, Sage: 377-392. Chataway, J. and A. Joffe (1998). Communicating Results. Finding Out Fast: Investigative Skills for Policy and Development. A. Thomas, J. Chataway and M. Wuyts. London, Thousand Oaks, New Delhi, Sage: 221-236. Denzin, N. K. and Y. S. Lincoln, Eds. (1994). Handbook of Qualitative Research. Thousand Oaks, London, New Delhi, Sage. Patton, M. Q. (1990). Qualitative Evaluation and Research Methods. Newbury Park CA, Sage Publications. Roche, C. (1999). Impact Assessment for Development Agencies: Learning to Value Change. Oxford, Oxfam Novib. Stake, R. E. (1994). Case Studies. Handbook of Qualitative Research. N. K. Denzin and Y. S. Loncoln. Londo, Thousand Oaks, New Delhi, Sage: 236-247. Web resources: http://www.socialresearchmethods.net/kb/contents.php Page 2 of 10 Qualitative Approaches for Food Security Assessment 687318203 http://www.nsf.gov/pubs/1997/nsf97153/chap_3.htm http://www.idrc.ca/en/ev-106470-201-1-DO_TOPIC.html http://chsd.uow.edu.au/Publications/2007_pubs/care_planning_info_bulletin_1.pdf The facilitator should give examples from practical experience, particularly when discussing the advantages and disadvantages of the methods, and when to use the various methods. Participants should be encouraged to tell anecdotes of their experience to enrich the discussion. Room Setup Same as earlier sessions Session Activities Fruit Salad: Energizer Exercise 10 minutes Start with a short energizer of the many known in the literature. The energizer suggested here is the Fruit Salad Energizer (Box below), which will assist the facilitator in reshuffling the groups formed in the morning, while getting people to interact with each other. Page 3 of 10 Qualitative Approaches for Food Security Assessment 687318203 Fruit Salad Energizer: The aim of the fruit salad is to get everyone re-energized after lunch and to divide participants into three groups for Exercise No. 4. Create a circle of chairs with chairs facing inwards. You need 1 chair less than the number of participants. Ask everyone to sit on a chair facing the centre. Participants will realize that there are not enough chairs. Explain that 1 person is in control of the game and ask this person to stand in the middle of the circle. Now go around and tell each person that they are a type of fruit, depending on number of group you want to end with (for the purposes of the upcoming exercise use three fruits only). So, go around and assign a fruit to each person, including the person in the middle. Now whoever is in the middle has control of the game. If he shouts apples, all the apples have to stand up and switch chairs with another apple. Whilst this is going on, the person in the middle has to try to get a seat. You will again end up with one person in the middle again. Continue this using the other fruits e.g. if he shouts bananas all bananas stand up and again the person in the middle has to try to get a seat. If the person in the middle shouts fruit salad then everyone has to get up and switch seats. If nobody asked for salad or cocktail ask for it as a trainer. Warning: People tend to get quite competitive, a little loud and rough. Therefore check at the beginning if anyone has any reason why they can't jump around e.g. pregnancy, foot complaints, etc. Session Introduction and Overview of Qualitative Methods 5 minutes Introduce the session objectives (slide 3). Introduce the three main data collection methods (slide 4), noting that there are other qualitative methods such as case studies and documentary reviews but that the discussion will only be limited to the three methods most commonly used. Ask participants to regroup according to the last group they were in the Fruit Salad Exercise for a group exercise. Assign one method to each group. Introduce Exercise 4 (slide 5) . Participants are expected to familiarize themselves with and discuss the three methods when answering why the method they are representing is the best method for food security assessments. Ask participants to take a few minutes to read Handout 1.3 and review Pages 2-5 of TGS#9 before discussing the exercise. Page 4 of 10 Qualitative Approaches for Food Security Assessment 687318203 Instruct the groups to be creative in presenting their method, highlighting that they should stress the advantage of their method and downplay its disadvantages. Each group will be given five minutes to deliver its presentation. Five to ten minutes will allocated to discuss each presentation. In-Depth Look at Qualitative Assessment Methods: Exercise 4 30 minutes Move between groups to provide guidance as necessary. If one of the groups has finished its task early, ask them about their key conclusions and question these conclusions to prompt them to reconsider what they will present. For example, you can ask them if they have discussed the implications of the disadvantages they discussed and how to deal with them, or if they considered when to use the method. Qualitative Methods Compared 45 minutes The group tasked with presenting “observation” should deliver its presentation first. Encourage participants to comment or ask questions. Make sure that the following questions are discussed in a plenary discussion either by asking them directly or by probing: 1) What is the difference between participant observation and non-participant observation? What is the advantage of each? 2) What are other advantages and disadvantages than those mentioned by the group? 3) How can we respond to the limitations of inconsistent observations? Observers misinterpretations? 4) What are some examples of observations conducted in FSN assessments and what kind of information can they provide us (link with previous exercise, Exercise 3, if time permits)? 5) What do the participants think about the value proposition (why is it the best method) made by the group? Confirm/correct what was discussed (slide 7 to 9). Highlight the following points as appropriate: Slide 8: A distinction is often made between Participant observation, where the observer shares some of the activities or discussions that are being assessed in order Page 5 of 10 Qualitative Approaches for Food Security Assessment 687318203 to get a better understanding of insider views and experiences, and non-participant direct observation where the observer deliberately does not become involved in the situation under assessment in order not to influence it. In reality the distinction is often blurred as any observer is likely to influence events in some way. Slide 8: Observation occurs in all types of assessment (initial, rapid, comprehensive). Good interviewers carefully observe the nonverbal signals that a respondent may give and adapt their questions. They will observe the environment in which an interview is conducted to see if there are signs that confirm or contradict what the interviewee is saying. It is possible to make this observation process more systematic and more reliable by the careful documentation of certain aspects of an interview (for example body language, how people treat each other, how conflict is handled, the trends in agreement and disagreement between people being interviewed), and by being critically reflective on one’s own assumptions and biases. It is very important to crosscheck information from observation to avoid misunderstanding particularly when the observer is from a different culture. Videos and photographs may be extremely useful for both aiding memory and as a focus for subsequent questions. Slide 8: Observation can be combined with both quantitative and participatory methods. Some of the things observed can be quantified e.g. length of time or numbers of contributions taken by particular speakers at a meeting, numbers of houses of particular types which may indicate levels of poverty. Observations can also be recorded in the form of diagrams. Slide 9: A simple observation may be extremely useful to confirm what may seem as a subjective reporting by respondents. For example, if people say that food is not available in the market, a quick visit to the market can confirm whether food is available or not. Similarly, if a respondent reports that lands are not cultivated in a certain area, a visit to that area can confirm or refute this claim. Slide 9: What makes observations particularly advantageous is that they provide the assessor with a first-hand feeling of the situation he/she is assessing. Analyzing the results of an assessment is often strengthened by images and feelings obtained from direct observations. These bring out the human side of the story a lot more strongly. Slide 9: Observations may uncover issues that were unknown at the time of the design of the assessment and that go unreported by informants. For example, in many communities, sanitation problems have become part of the “usual” daily living conditions that people may not report anymore. Call upon the group tasked with presenting “Interviews” to deliver its presentation. Invite participants to comment or ask questions to the presenting team. Make sure that the following questions are discussed in a plenary discussion either by asking them directly or by probing: Page 6 of 10 Qualitative Approaches for Food Security Assessment 687318203 1) What is the difference between structured, semi-structured and unstructured interviews? when can they be used? 2) What are other advantages and disadvantages than those mentioned by the group? 3) How can we respond to the limitations and disadvantages of interviews, particularly respondent bias? 4) What do the participants think about the value proposition (why is it the best method) made by the group? Confirm/correct what was discussed (slide 10-13). Highlight the following points as appropriate: Slide 10: The central method in qualitative research is simply talking and listening to people, interviews are thus one of the most frequently used methods in qualitative food security assessments. In qualitative assessments every opportunity for investigation and increasing knowledge is important. The unanticipated may often be more useful in highlighting what people really think and really do than answers in a formal survey situation. Slide 10: Qualitative interviews can take many forms including: o Qualitative questions added to structured surveys and questionnaires at the end, or annotated in the margin. o Semi-structured interviews where the questions are more open and answers recorded in more detail, and where space is left for unanticipated issues arising in the course of conversation. o Open-ended but more probing interviews where the broad issues to be covered are clear, but the order or ways in which they are asked are decided in the course of conversation. o Completely open-ended ad hoc conversations with people as the opportunity arises and determined by what they are interested in talking about. Slide 11: Depending on the focus of the assessment, questions can range from microlevel details of people's daily lives to detailed questions about ways in which organisations and institutions work, or macro level policies. The distinguishing feature of qualitative interviews is their continual probing and cross checking of information and a cumulative building on previous knowledge rather than adherence to a fixed set of questions and answers. Good interpersonal skills are crucial as is careful documentation (highlight that both will be covered later). Slide 11 (when presenting the first point on the slide): Like observations, qualitative interviews can generate quantitative information. For example types of responses can be classified together and numbers of people counted. Qualitative interviews can also be conducted with groups of people. They often involve several members of one Page 7 of 10 Qualitative Approaches for Food Security Assessment 687318203 household, or neighbors who come in to hear what is going on or other members of organizations who happen to be passing by. Slide 12: qualitative interviews provide an opportunity for exchange, casual conversation and probing. The environment is more comfortable, hence generates more in-depth information and understanding. This is not always the case, however, as some respondents are reticent to openly share their thoughts and feelings (slide 13). Slide 13: the disadvantages of qualitative interviews are quite similar to the ones of quantitative (structured, questionnaire-based) interviews. Assessors must always be aware of the biases of respondents and the sensitivity of some topics. Hence, assessors must continuously check for inconsistencies and make every effort to probe respondents and question their answers in a respectful way. Call upon the group tasked with presenting “Discussions” to deliver its presentation. Invite participants to comment or ask questions to the presenting team. Make sure that the following questions are discussed in a plenary discussion either by asking them directly or by probing: 1) What is the difference in structure between community discussions and focus group discussions? when can they be used? 2) Can community discussions and focus group discussions be classified as a semistructured group interviews? Why/how? 3) What are other advantages and disadvantages than those mentioned by the group? 4) How can we respond to the limitations and disadvantages of “discussions”, particularly respondent bias? 5) What do the participants think about the value proposition (why is it the best method) made by the group? Confirm/correct what was discussed (slide 14-16). Highlight the following points as appropriate: Slide 14: In TGS#9, discussions are classified as a separate method because of the width of information they provide, i.e. they do not provide in-depth personal information as interviews do. However, in most qualitative research literature community and focus group discussions are classified as group interviews. Slide 14 (when presenting community and focus group discussions): community discussions are different from focus group discussions in the mix of participants, issues usually covered (community discussions are held with mixed group of Page 8 of 10 Qualitative Approaches for Food Security Assessment 687318203 community representatives to explore a wide range of issues/topics, while focus group discussions are organized with a group sharing at least one characteristic to explore an issue or topic in-depth), and –sometimes- timing (focus group discussions can be held anytime during the assessment, whereas community discussions are usually organized early in the assessment to gather broad information and/or –in the cases of participatory assessments- at the end to verify assessment findings). Discuss the utility of the various methods and the linkages between qualitative and quantitative methods (slide 17). Ask participants to reflect on the meaning of the arrows on the slide and to share their experience in this regard. Discuss how quantitative and qualitative methods inform each other. More specifically, note the following during the discussion: The three methods inform each other both during the assessment process itself (for example, observations can lead to additional investigation during an interview or a focus group, or a statement in an interview may prompt an observation), and later during the analysis (when the data from various methods are compared for establishing trends and confirming findings). Qualitative assessments are usually informed by secondary data. These are critical to formulate the objectives of the assessment and design the assessment tools. Indeed food security assessments generally begin with a review of available secondary information (documented studies, country office reports and public records on the country context, economic and poverty conditions and the factors that affect food security). Time should be allocated for this review at the beginning of the assessment. While certain methods can be more appropriate than others in certain conditions and for specific purposes, the three methods allow the assessors not only to crosscheck/triangulate data, but also to make data collection a dynamic process. For example, the observation of an assessor can lead to the reformulation of the interview questions. Similarly, interview results can identify the need to verify initial findings though unplanned observations or focus group discussions. The data obtained from qualitative methods can be used to confirm/refute, deepen the understanding and explain the data obtained through quantitative methods. For example, an interview can be used to gather data from households on how they cope despite having very low food expenditures, or to deepen the understanding of reported households’ coping strategies. Similarly, exploratory interviews and discussions can be used to identify indicators for a quantitative assessment. The integration between the methods is key to undertaking an objective assessment, and this is why assessors should always review their field notes, discuss initial findings, Page 9 of 10 Qualitative Approaches for Food Security Assessment 687318203 identify data gaps and inconsistencies, and find ways to reduce these (or at least explain them). Note that the use of individual methods is determined by a number of factors, and ask participants in a plenary to identify these factors. Write down the responses on a flipchart. Conclude the discussion, noting that the choice of the method is usually determined by degree to which a method is flexible (the more flexible, the better), meaningful (will the method provide the information being sought) and manageable with the resources available (slide 18). Wrap up with key messages on things to remember about qualitative assessment methods (slide 19). Stress that while qualitative methods are not as structured as quantitative methods, a significant amount of planning is needed to formulate the process and the questions these methods need to answer. This is why the term “semi-structured” or “unstructured” should never be equated to “unplanned”. Ask if anyone has any questions and invite all for a 30 minute break. Mention that the last session of the day will be dedicated to discussing qualitative assessment tools. Page 10 of 10