This toolkit was originally developed by PC Burkina Faso. The original has been revised by
OPATS for wide use and to include global health indicators. Please revise to align with your
programming as needed.
Volunteer and Counterpart Handbook
Measuring Our Results
Table of Contents
Purpose of the Handbook
Developing a way to measure results
Step One:
Assess what you are doing by topic or issue area
Step Two:
Select the key results you and your counterparts agree to measure
Step Three: Decide how to measure your key results
Step Four:
Develop tools to measure your key results
Step Five:
Practice using the measurement tools
Step Six:
Improve the tools based on how well they worked in practice
Step Seven: Pilot test the revised tools again
Step Eight:
Discuss the results of the pilot test and improve tools some more
Step Nine:
Develop an action plan to measure the key results of your work
Step Ten:
Carry out your action plan
Coordinating and supervising a survey effort
Compiling and summarizing what you learned from your survey/measurement effort
Sharing findings with your community
Sharing findings with Peace Corps
Using your findings to improve your work
Using your findings to strengthen funding proposals
Appendix
Appendix A:
Appendix B:
Appendix C:
Appendix D:
Detailed facilitator’s agenda for 5 day ‘measuring success’ workshop
Glossary
Malaria survey
HIV/AIDS survey
1
Purpose
The purpose of this draft handbook is to assist Peace Corps Volunteers, their counterparts and
members of their communities to better measure the results of their efforts. It is based on a 5 day
workshop conducted in April of 2007 with Volunteers and counterparts active in community
health promotion. As a result, this first draft of the handbook contains training session ideas as
well as questionnaires and data collection tools designed and piloted by the participants of that
workshop. Please continue to add to this handbook to make it more useful to Volunteers
working throughout Burkina Faso.
Developing a way to measure results
Step One:
Assess what you are doing by topic or issue area
Volunteers and their counterparts should work together in pairs to list their main health education
or other activities in the past year.
 What types of health education or other activities have you undertaken in the past year?
 What was the focus of the activity--what messages?
 How many times did you do the activity?
 How many people did you reach through the activity?
To see the breadth of topic areas (e.g., AIDS, malaria, hygiene, etc.) that are being covered by
Volunteers and their counterparts, return to the large group and have each Volunteer/Counterpart
list their activities by topic area. Take each topic area in turn. Record activities by topic area on
flip charts like the one below:
Paludisme
Activités
e.g. talk to mothers at the
clinic about the importance
of sleeping under nets
Step Two:
Select the key results you and your counterparts agree to measure
As a group, discuss what outcomes you hope to achieve. Chose one topic area (e.g., AIDS) to do
together as a group discussion, asking the following questions:
 What type of change are we trying to encourage? (improved knowledge, attitude,
behavior)
 Who is the target population? (pregnant women, youth, men, etc.)
Have two Volunteer/Counterpart pairs ask each other about the kind of results they want from
their work in health education or other areas. Participants then briefly present the results of their
interviews to the group. Record desired results/outcomes on flipcharts like the one below:
Paludisme
Activités
Résultats Désirés
e.g. talk to mothers at the
clinic about the importance
of sleeping under nets
e.g. more women and
children will sleep under
nets
2
Facilitator then takes a few of the desired results for one health topic or issue area and looks at
them carefully, asking these questions to the group:
 What kind of outcome is this—a change in knowledge? Skills? Attitude? Behavior?
Condition?
 Is this outcome a realistic result of the activity?
 Is this outcome something we all agree is very important?
 Is this outcome something that more than one of us is hoping to achieve?
 Should we call this outcome a key outcome?
 How will we know when this outcome has been achieved?
Step Three: Decide how to measure your key results
Look at each desired result and discuss how you would measure whether or not it is being
accomplished. If it is a change in knowledge or awareness, you will need to ask people
questions about their awareness and knowledge. If it is a change in behavior, discuss how you
can collect this data, i.e. through observation, or simply reported practices via questions to target
population. Examples of observation include checking on presence of soap where family
members wash their hands; checking to see the latrine or how drinking water is being stored in
the household. Examples of reported behavior include asking individuals about the actions they
took during an illness in the weeks preceding the survey; or child feeding in the 24 hours, or
week prior to the survey. Health providers can also be interviewed as they can speak to trends in
care-seeking behavior, and provide data showing use of health services.
If the change is knowledge, you will need to discuss these questions to decide on how you will
go about measuring the change in knowledge:
 Who will you need to interview/ask?
 Where is the best place to do this?
 When is the best time to do this?
 How should you do these interviews/questions?
 What question(s) will you ask them to assess their knowledge/awareness?
Step Four: Develop tools to measure your key results
In a big group, take one key outcome and address the following questions:
1. What will tell you that people have increased their knowledge or changed their attitudes or
behavior? What questions should you ask them?
3
Paludisme
Résultats Clefs
Par example : People will recognize the symptoms of
malaria
What information do we need?
 The level of knowledge throughout the village about
the symptoms of malaria : fever cycles every 24 hours,
headache, chills/convulsions
What question(s) should we ask ?
 “How do you know when someone has malaria ? What
are the signs ?”
Who do we ask ?
Where do we ask them ?
When do we ask them ?
Then have people work in small groups to begin drafting other questions related to assessing
other changes in knowledge.
In drafting these questions, bear in mind that in many ways “less is more.” This means that you
should only ask a question you are very sure will yield useful information. The goal should be to
ask as few questions as you can to get the information you absolutely need. This will make the
overall effort much more efficient for you and for the people you need to interview, and it will
make managing all the information you get much easier.
Step Five:
Practice using the measurement tools
Once you have a draft of the questionnaire/survey instrument, it will be useful to practice using it
before you head out to interview people for real. To practice it, have two volunteers—one
interviewer and one respondent—agree to model the interviews. The respondent should pretend
to be someone in a village, sitting at home, not expecting to be approached by someone asking
for an interview. The interviewer should approach the villager, greet her appropriately, explain
why s/he is there, and ask permission to do an interview.
The facilitator of the workshop should stop the role play at this time. Ask the large group to
reflect on how well the interviewer approached the beginning of this interview :
Q:
Q:
Q:
Q:
Q:
What did you think about how the interviewer introduced himself ?
Was the purpose of the interview clearly explained ?
Did the interviewer ask permission to do the interview ?
Did the interviewer set an appropriate tone for the interview/put the respondent at ease ?
What could the interviewer do better next time ?
After this discussion, the interview continues, question by question. The facilitator should wait
for the question to be asked and answered, and for the interviewer to try to record the response,
then stop it and ask the group to reflect again.
Q:
Q:
Q:
Q:
How clear was the question ?
How clear was the response ?
Could the interviewer have done anything to prompt a clearer or more complete response ?
Is this question in the right place ? Should it be asked earlier or later in the interview ?
4
Continue this process until all the questions have been asked and answered.
Step Six:
Improve the tools based on how well they worked in practice
Make revisions to the questions based on the lessons from the previous role play exercise. The
objective is to refine the questions to make them clear, and to put them in an appropriate order.
If there is time suggest that the group include any instructions to the interviewer on the
questionnaire. For example, if the answer is no, skip to question 5.
Step Seven: Pilot test the revised tools again
In a large group setting, discuss the purpose of the pilot test, if you can do one.
What is the purpose of your pilot test? What are you testing, exactly ?
Allow for some group discussion of this, and emphasize the following points about what the pilot
test is testing:
 The clarify of the questions. Do you have to rephrase or restate the question too much ? Is it
clear the first time you ask it ? How can you make it clearer ?
 The usefulness of each question. Is it necessary to ask all the questions you have ? Can you
eliminate any questions ?
 The length of the interview. How long does each interview take, on average ?
 The availability of people to respond at that time. Are enough people available at the time
of day/month/year for this survey ?
 The clarity of responses to your questions. If more than one person is listening to the
interview, do they each agree on how the respondent answered each question ?
 The availability of other data/information. If some of the information you are looking for
would come from the records of a health clinic or school, for example, is the specific
information you would need available to you in a form you can easily use ?
 How easy is it to accurately record responses on your data collection form(s) ?
Small group planning of pilot test
Spend some time discussing these questions in your group and coming up with a plan for the
pilot test tomorrow.
 How many interviews are you going to attempt to do ?
 Where are you going to go when you get there ?
 Who is going to do the interviews ?
 If translation is necessary, who is going to do the translating ?
Materials prep for pilot test
To type up and copy final drafts of the survey/questionnaires as well as any additional data
collection tools, get at least one volunteer from each group to do the administrative work of
preparing materials for their small group members to use tomorrow. The rest of the group can
either break early for the day or meet on other issues.
Implementing the pilot test
5
Small groups go out, each with a facilitator, to the village to test tools. Each group should try to
conduct 5 to 10 interviews. Observers give feedback after each interview. Each group should
also look for data as necessary at the health clinic, boutiques, and through observations.
Groups should use the following questions as a guide to evaluate their experience during the
field test:
 How did you determine which way to go in the village?
 How many people/households did you interview?
 How did you choose the interviewees?
 How did the introductory part go? (did you put the interviewees at ease? did you ask for their
informed consent? did you explain confidentiality? etc.)
 Were the questions understood by the interviewees? If not, why not?
 Is the order of the questions logical? If not, why not?
 Is the length of the survey appropriate? If not, why not?
 Can you eliminate any questions ?
 Do you need to add any questions ?
 Is the data recording instrument appropriate ? If not, why not?
 Did you each record the same responses to each question ? If not, why is there disagreement
or confusion about how people responded to your questions ?
 Did you find relevant data at the health center? If not, what did you do?
 Did you encounter any other difficulties?
 What are your recommendations for improving the process?
Step Eight: Discuss the results of the pilot test and improve tools some more
Small groups meet together first to review how their pilot test went, and to prepare their
summary for a debrief to the larger group. In your debrief to the larger group, focus your
remarks on :
 How many interviews you did.
 The average length of each interview
 Which questions seemed to cause confusion and how you recommend changing those
 If members of your group recorded responses differently, why did that occur and how are
you going to fix that ?
 Other challenges or lessons learned from the pilot
Step Nine:
Develop an action plan to measure the key results of your work
Each Volunteer and counterpart should put together an action plan that lays out how they are
going to start measuring results when they get back to their sites. Emphasize who is going to do
what, by when.
Step Ten:
Carry out your action plan
Coordinating and supervising a survey effort
Be sure to get approval from every village you intend to do a survey or collect data. Follow your
training as Peace Corps Volunteers in how to approach a village, who to ask for permission, how
6
to explain what you are doing and what you hope to learn from this effort, and how you intend to
share results with them.
It may be necessary to find interpreters or guides to assist your research efforts.
Compiling and summarizing what you learned from your survey/measurement effort
Facilitator leads the group through a review of the data collected by the groups. Note differences
between men and women for each survey and what the data suggest about priority areas for
health education. What do you do with data? (Analysis) If evidence shows that behavior isn’t
changing, it could be because: the message is wrong, a key piece of the message is missing, the
treatment isn’t intense enough, the wrong people are targeted.
Use a simple clear approach to summarizing the results of your research, along these lines:
# hommes :
# femmes :
Nombre Total d'Individus
qui acceptent l'interview
# enfants :
Réponses correctes
# hommes :
Nombre de personnes
qui connaissent les
symptomes du
paludisme
# femmes :
# enfants :
Sharing findings with your community
Anytime you do research in a community, be sure to share what you find with them as soon as
you have summarized them. Present your findings in as clear and simple a way as you can.
Avoid using research terms. Present what you think are the most interesting findings and ask
what they think about them. Ask if anything you found is surprising
Sharing findings with Peace Corps
7
Using your findings to improve your work
Using your findings to strengthen funding proposals
8
Appendix A:
Detailed facilitator’s agenda for 5 day ‘measuring success’ workshop English/French
Peace Corps/Burkina Faso: ‘Measuring Success’ Workshop
Ouagadougou, April 23-27, 2007
Participants: 11 Peace Corps Volunteers and 10 counterparts
Facilitators: Dr. Claude Millogo – PC/BF APCD Health; Jerry Black – PC/W Evaluation
Specialist; Kathy Jacquart – PC/W HIV/AIDS Programming and Training Advisor; Joachim
Ouedraogo– CNLS Representative, Dr. Joanne Capper – PC/W Evaluation Specialist
Workshop Purpose: To train and equip workshop participants to measure the outcomes of their
health education interventions
Workshop Outcomes:
o Draft handbook for staff, Volunteers and counterparts on how to measure outcomes of
health education and disease prevention activities (e.g. malaria, HIV/AIDS, tuberculosis,
polio, nutrition, guinea worm, other?)
o Set of simple measures/instruments that can be used by literate or semi-literate health
educators/promoters/community members.
Workshop Format: Interactive presentations, small group work, field practice.
Materials Needed: Flipcharts and markers; tape; post-it note pads; paper, pencils, pens,
markers.
Monday, April 23
8:30-8:45
Welcome and Purpose of the Workshop
Marily Knieriemen, PC/CD and Dr. Millogo open the workshop.
8:45-9:15
Introductions/Icebreaker/volunteers for energizers and daily evaluations
9:15-9:45
Overview of Workshop Objectives, Agenda, Expectations and Norms
Walk through what we hope to achieve each day, and what the approach will be each day. Ask
participants what they hope to get out of the workshop; record answers on flipchart. Note
how/where the proposed agenda is going to address participant expectations. If some
expectations are not part of the agenda, propose modification to agenda, or explain why
expectations can’t be met through this workshop. Reinforce these ideas:
 This workshop is building on the work they are already doing by developing a simple
approach for measuring the results of their health education and promotion activities.
 Workshop participants will create this simple evaluation approach, as well as some draft
measurement tools (surveys, data collection forms, etc) together to ensure that the approach
is relevant and appropriate.
 Over the next 5 days we will design a practical approach to evaluation that will help
Volunteers with their counterparts learn what is working well as well as where, with whom,
9
when and how to focus their health promotion messages and activities to have a more
powerful effect in their communities.
Norms for the next 5 days include:
 Speak slowly and loudly in French
 Do not interrupt anyone who is speaking
 Turn off all cell phones
 Be on time
 Raise your hand before you ask a question or make a point
 If any word or expression is not clear to you, ask for clarification. We will keep a running list
of definitions on a flipchart on the wall.
9:45-10:00
What are our health education activities?
Volunteers and their counterparts work together in pairs to list their main health education
activities in the past year.
 What types of health education activities have you undertaken in the past year?
 What was the focus of the activity--what messages?
 How many times did you do the activity?
 How many people did you reach through the activity?
10:00-10:15
Coffee break
10:15-10:45 What are our health education activities?
To see the breadth of topic areas (e.g., AIDS, malaria, hygiene, etc.) that are being covered by
Volunteers and their counterparts, return to the large group and have each Volunteer/Counterpart
list their activities by topic area. Take each topic area in turn. Record activities by topic area on
flip charts like the one below:
Paludisme
Activités
e.g. talk to mothers at the
clinic about the importance
of sleeping under nets
10:45-11:15 What are we trying to accomplish through our work?
As a group, discuss what outcomes they are hoping to achieve through their activities. Chose
one topic area (e.g., AIDS) to do together, asking the following questions:


What type of change are you trying to encourage? (improved knowledge, attitude,
behavior)
Who is the target population? (pregnant women, youth, men, etc.)
11:15-12:30 Small group work and presentations on what we’re trying to accomplish
Have two Volunteer/Counterpart pairs ask each other about the kind of results they want from
their work in health education. Participants then briefly present the results of their interviews to
the group. Record desired results/outcomes on flipcharts like the one below:
10
Activités
Paludisme
Résultats Désirés
e.g. talk to mothers at the
clinic about the importance
of sleeping under nets
12:30-1:30
e.g. more women and
children will sleep under
nets
Lunch
1:30-2:00
Looking more carefully at a few outcomes
Facilitator takes 3 to 4 of the intended outcomes for one health topic and looks at them carefully,
asking these questions to the group:
 What kind of outcome is this—a change in knowledge? Skills? Attitude? Behavior?
Condition?
 Is this outcome a realistic result of the activity?
 Is this outcome something you all agree is very important?
 Is this outcome something that more than one of you is hoping to achieve?
 Should we call this outcome a key outcome?
 How will you know when this outcome has been achieved?
2:00-3:00
Participant analysis and discussion of outcomes
Break group into small groups, giving each group a health topic area. Ask each small group to
go through a similar exercise of analyzing the intended outcomes on their flipcharts. Assign
someone to record the group’s decisions about the outcomes. This person will present back to
the larger group your small group’s recommendations. Be prepared to explain which outcomes
you believe the group should consider the key, or core ones.
Co-facilitators should sit with each small group and make sure they understand what to do and
get through this exercise, prepared to talk about their decisions.
On flipchart
Small Group Work: Outcome Analysis Questions
 What kind of outcome is it? (knowledge, skill, attitude,
behavior, condition)?
 Is the outcome or result possible and realistic?
 How many of us are working to achieve this outcome?
 How important is this outcome to all of us?
 Do we agree that this outcome is a key outcome we all want to
measure?
 How will we know when this outcome has been achieved?
3:00-3:15
Break
3:15-4:45
Small group presentations and discussion
11
Each small group gets 10 minutes to present their discussion and proposal to the group, and
about 5 minutes for group discussion. The goal of this discussion is to arrive at a group
agreement about the key desired results (outcomes) for all their health work. These key
outcomes will then become the focus for the development, testing and refining of related
measures.
Facilitator keeps each group to the time limit, and pulls out the key questions for the larger group
to consider, which are:
 Do we all agree that this/these are the key outcomes we’re hoping to achieve in this
health area?
Paludisme
Résultats Clefs
Par example :






people will recognize the symptoms of malaria
people will understand how malaria is transmitted
people will understand how to prevent malaria
more women and children will sleep under nets
families will take their children to the clinic at first
signs of malaria
pregnant women will take malaria prevention
medication at pre-natal visits
4:45-4:55
Recap of the day and preview of Day 2
Summarize the highlights of the day.
Tomorrow we will explore the question: “How do I know when my work has been successful?”
4:55-5:00
Day 1 evaluation
Take a few minutes to write your thoughts about what worked well today as well as what could
be improved; put your post-it notes on the flipchart. Can we get a Volunteer or two to agree to
summarize this for us by tomorrow morning?
Day 1 evaluation
What worked well
What could be improved
Thanks for good work today.
12
Tuesday, April 24
8:00-8:15
Introduction and recap of Day 1
Energizer.
Recap Day 1. Ask volunteers to present summary of participant feedback from Day 1. Ask
group if there are issues or questions about yesterday.
Tell the group that today we are going to discuss why it’s important to provide convincing
evidence of what you have accomplished. We are then going to design some draft data
collection instruments together. These are the tools you will use to measure your successes and
provide convincing data. The idea here is that we will design these tools together and all agree
to use them, and to encourage other Volunteers or counterparts we know to also use them. The
more people use them, the more powerful a sense of your outcomes you will start to get. Also it
should help you to have more informed and productive conversations about where the priority
health education needs are in Burkina Faso and how Peace Corps can best meet them.
Ask for volunteers to lead the afternoon energizer and evaluation for Day 2.
8:15-9:00
Accomplishments to date
Q: Before we go further, who wants to share with us a result that they have seen from their
work?
Q: How have you measured your successes so far? What approaches have you taken to figure
out how successful your work has been?
If no one mentions anything, be prepared to cite the following outcomes from Claude’s FY06
PSR last year, namely:






PCVs observe more pre-natal visits to health clinics.
PCV(s) noted that the outcome of theatre groups work in malaria education included
increasing knowledge of malaria and increasing use of mosquito nets. Who did this?
How was the study done?
Collaboration between COGES, clinics and theatre groups has improved: What does this
collaboration look like? Is this something PCVs are actively working on? How do you know
collaboration is happening the way you want it to?
Self-sufficiency of theatre groups has improved in some cases: How is self-sufficiency
measured? Who does this exercise?
PCV(s) found increases in community members’ knowledge after theatre groups did
their shows. Was this a pre-post test with the crowd? Whose knowledge increased?
More use of water pumps (Guinea worm): What does this statement ‘more use of water
pumps’ mean—where is this happening? How much more use?
Acknowledge that this group and their colleagues are achieving a lot in Burkina Faso, and that
what we’re trying to do in this workshop is figure out a better way to identify the most important
accomplishments and successes. We made some progress here yesterday. We have agreement
on the key outcomes you all are trying to achieve.
13
9:00-10:00 The value of good measures
The next question is: How will we know when our work has succeeded?
Look at the outcome about use of mosquito nets. Some Peace Corps Volunteers did a study of
the results of the work of theatre groups in Burkina Faso and found that more people understand
how to prevent malaria, more people bought mosquito nets, and more people are using mosquito
nets.
Q. Why is it important to have a good way to measure the effectiveness of your activities?
Allow a few minutes for group to respond to this question. If no one responds, say something
like: The value to you of having a good way to measure the effectiveness of things like the
theatre group’s malaria messages is that:
 you will know what messages are really working;
 you will know how to improve your malaria prevention work so that it is more effective;
 it can motivate you to do more;
 you will be able to convince other people that your work is successful – to get funding, to get
support from community leaders, etc.
Q: Which statement is more convincing? (write statements on flip chart)
A. As a result of the theatre group’s work in malaria education, more people are using
mosquito nets.
B. Before the theatre group’s work in malaria education in the village of Linkinkele, 10
people in the village slept under mosquito nets, including just 2 women and 3 children.
Two months after the theatre group’s education activities and performances in
Linkinkele, 100 villagers sleep under mosquito nets, including 35 women and 40 children
under 5.
Reread them. Allow some discussion.
Q: Why is statement B more convincing to you than the first one?
Q: What would you have to do to be able to make a statement like B?
Allow participants to respond.
 Know how many people used mosquito nets before the theatre group’s performance about
malaria prevention; then know how many people use mosquito nets two months after the
performance.
Q: How would you figure out how many people sleep under a net at the beginning? (we call this
getting a ‘baseline’)
Allow participants to respond.
There are two ways to figure this out. The first thing to find out is if there is already an existing
source of data about the use of mosquito nets in those villages.
14
Q: Where might you find data like this? (e.g., Demographic and Health Survey, CSPS, NGOs?)
If there aren’t data available to you, then you should consider doing a survey of the villages you
know the theatre groups are going to, before they go there. So it’s important to know how to do
a survey.
In doing a survey, plan carefully what questions you are going to ask, who to interview, where
to interview, when to interview, and practice how you do the interview.
What questions? Crafting good questions is a skill. We will spend a lot of time in this
workshop crafting (drafting and refining) the questions you will ask to make sure they are good
ones. We’ll turn to this later today and tomorrow.
Who ?
Paludisme: Enquete de Bas (Malaria baseline survey)
Resultat desire : people recognize the symptoms of
malaria
Who to interview ?
 Men, women, youth (in and out of school), children
Q : If you want to assess a village’s understanding of malaria, who should you interview ?
Record participant responses on the flip.
A : You should ask a range of people in the village, including men, women, students, children,
and older adults. It would be a good idea to ask as many people as you could with the time and
resources you have available for the task.
Where ?
Paludisme: Enquete de Bas (Malaria baseline survey)
Resultat desire : people recognize the symptoms of
malaria
Where to interview ?
 People who live in each part of the village
Q : If you want to assess a village’s understanding of malaria, where should you go to do these
interviews ?
Record participant responses on the flip.
A : You would want to try and interview people throughout the village so that you ask people
who live near the health clinic as well as people who live far from it. You would not get enough
information about the level of malaria knowledge throughout the village just by asking people
who live in the same neighborhood.
15
When ?
Paludisme: Enquete de Bas (Malaria baseline survey)
Resultat desire : people recognize the symptoms of
malaria
When to interview ?
 Before and after our education activities
 Before, during and after the rainy season
 Before the harvest season begins
 Just before bedtime
Q : If you want to assess a village’s understanding of malaria, when should you do these
interviews ?
Record participant responses on the flip.
A : There are several ‘whens’ to keep in mind here :
 Before and after your malaria education activities.
 Certain months of the year, or certain seasons, may be better than others. You may decide it’s
important to ask people before the rainy season when malaria is on their minds, or before the
harvest season when they may be too busy for you.
 Time of day may matter. You should also consider the best time of day for the survey. If,
for example, you want to ask about the use of mosquito nets, and you want to ask to see their
nets to verify that they are attached, you might want to try and ask people at the end of the
day when the nets are more likely to be attached.
How ?
Paludisme: Enquete de Bas (Malaria baseline survey)
Resultat desire : people recognize the symptoms of malaria
How to interview ?
 Practice (pilot) the questions before actual research begins
 Explain the purpose of the interview and how you will use
the information
 Explain that the results will be confidential
 Ask for permission to do the interview before asking any
other questions
 Make the person feel at ease. Find a comfortable private
setting for the interview, if possible
 Speak slowly and clearly
 Listen and don’t interrupt
 Don’t ask questions in a way that suggests the ‘right’
answer
 Stay neutral throughout the interview
 Prompt for more complete answers in a neutral way, « do
you have anything more to add to that ? » or « is there
anything else you want to say ? »
16
Q : If you want to assess a village’s understanding of malaria, how should you do these
interviews ?
Record participant responses on the flip.
A : Interviewing is a skill, and you will get better at it through practice. In conducting an
interview, be attentive :
 to how you introduce the purpose of the interview
 to the atmosphere you create for the interview
 to how you ask each question
 to how you listen to each response
 to how efficiently and accurately you record each response
 In asking and listening, stay neutral. It’s critical that you do not ask questions in a leading or
suggestive manner. The tone of your voice, the way you move your eyes or body can often
suggest the kind of response you want. Many people will respond to subtle verbal and
physical cues by providing the response that they think you want to hear, rather than their
opinion or understanding. When this happens the interviewer has corrupted the integrity of
the interview and essentially rendered the results meaningless. To avoid wasting your own
time or anyone else’s, it is very important to stay as neutral as you can.
10:00-10:15
Coffee break
10:15-10 :45 Developing measurement instruments
In a big group, take one key outcome and address the following questions:
2. What will tell you that people have increased their knowledge or changed their attitudes or
behavior ? What questions should you ask them ?
Paludisme
Résultats Clefs
Par example : People will recognize the symptoms of
malaria
What information do we need?
 The level of knowledge throughout the village about
the symptoms of malaria : fever cycles every 24 hours,
headache, chills/convulsions
What question(s) should we ask ?
 How do you know when someone has malaria ? What
are the signs ?
Who do we ask ?
Where do we ask them ?
When do we ask them ?
17
10 :45-12 :00 Small group work to begin drafting measurement tools
Divide into 4 small groups: Malaria A and B, AIDS A and B. Each small group asks the What,
Who, Where, When questions for each outcome and records their answers.
12:00-1:00
Lunch
1:00-2:00
Small group reports
Each small group (Paludisme A, Paludisme B, SIDA A, SIDA B) reports how they answered the
What, How, Who, Where, When questions.
2:00-3:00
First draft of Survey or questionnaire
Small groups return to complete the first draft of their survey/measurement tool
3:00-4:00
Practice/Role play of survey
Two volunteers—one interviewer and one respondent—agree to model the interviews. The
respondent should pretend to be someone in a village, sitting at home, not expecting to be
approached by someone asking for an interview. The interviewer should approach the villager,
greet her appropriately, explain why s/he is there, and ask permission to do an interview.
The facilitator of the workshop should stop the role play at this time. Ask the large group to
reflect on how well the interviewer approached the beginning of this interview :
Q:
Q:
Q:
Q:
Q:
What did you think about how the interviewer introduced himself ?
Was the purpose of the interview clearly explained ?
Did the interviewer ask permission to do the interview ?
Did the interviewer set an appropriate tone for the interview/put the respondent at ease ?
What could the interviewer do better next time ?
After this discussion, the interview continues, question by question. The facilitator should wait
for the question to be asked and answered, and for the interviewer to try to record the response,
then stop it and ask the group to reflect again.
Q:
Q:
Q:
Q:
How clear was the question ?
How clear was the response ?
Could the interviewer have done anything to prompt a clearer or more complete response ?
Is this question in the right place ? Should it be asked earlier or later in the interview ?
Continue this process until all the questions have been asked and answered.
4:00-4:45
Revise questions based on role play
Each of the four groups makes revisions to their questions based on the lessons from the
previous role play exercise. The objective is to refine the questions to make them clear, and to
put them in an appropriate order. If there is time suggest that the group include any instructions
to the interviewer on the questionnaire. For example, if the answer is no, skip to question 5.
4:45-5 :00
Day 2 evaluation
What could be improved
What worked well
18
Wednesday, April 25
8:00-8:15 Introduction
Energizer. Recap Day 2. Ask group if there are issues or questions about yesterday.
8:15 - 9:00 DHS data exercise
Look at one question from the 2003 Burkina Faso DHS. Have the group guess the results. Show
the real results. Ask what the results tell them. Use a table like the one below.
Percentage of people who know that you can not become infected with HIV by eating with
someone with AIDS
Total Men
Women Rural Urban No
Primary
Secondary Rich
Poor
education Education Education
M
M
M
M
M
M
M
W
W
W
W
W
W
W
9:00 – 9:30 Why do we measure our results?
Brainstorm with the group the reasons for measuring our results. Reasons include:
 to see if activities worked
 to organize future activities
 to monitor if people are understanding the messages of an activity
 to change our work strategy
 to motivate ourselves
 to mobilize resources
 to mobilize support from the community
9:30 – 10 :00 Recording responses in an interview
Look at how well the questionnaire is formatted to allow the interviweer to efficiently and
accurately record the responses to his/her questions.
If the question is open-ended, include a list of likely responses below the question, with blank
space next to each likely response where you can put a check mark. This is going to work better
than trying to write down each response. In testing or piloting the questionnaire, one of the
things you’re trying to do is come up with the best list of likely (potential ‘correct’ and
‘incorrect’ responses) to a question.
For example:
Comment est-ce qu’on peut attraper le SIDA?
Ne pas lire ces réponses possibles. Notez-les seulement si l’enquêter les dit.
Correctes
Incorrectes
________Sang
_________Moustique
________Mère-Enfant
_________Poussière
________Relations sexuelles
_________Manger avec les gens infectés
_________Salive
_________Utiliser la même toilette
_________Etre proche/dormir à coté
_________Embraser une personne infectée
19
10:30 - 12:00 More practice and role playing of interviews
Give the groups time, if needed, to make additional revisions to their questionnaires. Then have
someone from each small group interview someone from another small group, with the objective
of modeling complete interviews without interruption, start to finish. Each member of the group
should be trying to record the responses, independently. Then have the group reflect on these
questions :
 How well did the interviewer introduce himself/herself ?
 How clear was the purpose of the interview ?
 Did the interviewer ask for ‘informed consent’ before proceeding with the interview
questions ?
 Did the interviewer set an appropriate, respectful tone for the interview ?
 How long did the interview take ?
 Was each question necessary ? Can you eliminate any questions ?
 Do you need to add any questions ?
 Are the questions in the right order ?
 What was the response to question 1 ? Question 2 ? etc. Does everyone agree what the
reponse was ? Why did you record different response, if you did ?
Each group should model their survey/questionnaire in this fashion and agree on any additional
revisions to their survey/questionnaire.
12:00 – 1:00
Lunch
1:00-1:45
Pilot testing the survey/questionnaire and sampling
What is the purpose of your pilot test? What are you testing, exactly ?
Allow for some group discussion of this. Record on flip and emphasize the following points as
well :







The clarify of the questions. Do you have to rephrase or restate the question too much ? Is it
clear the first time you ask it ? How can you make it clearer ?
The usefulness of each question. Is it necessary to ask all the questions you have ? Can you
eliminate any questions ?
The length of the interview. How long does each interview take, on average ?
The availability of people to respond at that time. Are enough people available at the time
of day/month/year for this survey ?
The clarity of responses to your questions. If more than one person is listening to the
interview, do they each agree on how the respondent answered each question ?
The availability of other data/information. If some of the information you are looking for
would come from the records of a health clinic or school, for example, is the specific
information you would need available to you in a form you can easily use ?
How easy is it to accurately record responses on your data collection form(s) ?
20
1:45 – 2:00 Small group planning of pilot test
Spend some time discussing these questions in your group and coming up with a plan for the
pilot test tomorrow.
 How many interviews are you going to attempt to do ?
 Where are you going to go when you get there ?
 Who is going to do the interviews ?
 If translation is necessary, who is going to do the translating ?
2 :00-2 :15
Day 3 evaluation
What worked well
What could be improved
2 :15-5 :00
Materials prep for pilot test
To type up and copy final drafts of the survey/questionnaires as well as any additional data
collection tools, get at least one volunteer from each group to do the administrative work of
preparing materials for their small group members to use tomorrow. The rest of the group can
either break early for the day or meet on other issues.
21
Thursday, April 26
8:00-1:00
Pilot test of data collection tools in village
Small groups go out, each with a facilitator, to the village to test tools. Each group should try to
conduct 5 to 10 interviews. Observers give feedback after each interview. Each group should
also look for data as necessary at the health clinic, boutiques, and through observations.
Groups should use the following questions as a guide to evaluate their experience during the
field test:
 How did you determine which way to go in the village?
 How many people/households did you interview?
 How did you choose the interviewees?
 How did the introductory part go? (did you put the interviewees at ease? did you ask for their
informed consent? did you explain confidentiality? etc.)
 Were the questions understood by the interviewees? If not, why not?
 Is the order of the questions logical? If not, why not?
 Is the length of the survey appropriate? If not, why not?
 Can you eliminate any questions ?
 Do you need to add any questions ?
 Is the data recording instrument appropriate ? If not, why not?
 Did you each record the same responses to each question ? If not, why is there disagreement
or confusion about how people responded to your questions ?
 Did you find relevant data at the health center? If not, what did you do?
 Did you encounter any other difficulties?
 What are your recommendations for improving the process?
1:00-2:30
Return to Ouagadougou/lunch
2:30-3:00
Debrief of pilot test
Small groups meet together first to review how their pilot test went, and to prepare their
summary for a debrief to the larger group. In your debrief to the larger group, focus your
remarks on :
 How many interviews you did.
 The average length of each interview
 Which questions seemed to cause confusion and how you recommend changing those
 If members of your group recorded responses differently, why did that occur and how are
you going to fix that ?
 Other challenges or lessons learned from the pilot
3:00-4:00
Finalizing questionnaires
Form two groups by theme (i.e., AIDS and malaria) to create final draft questionnaires. This will
be the version that all Volunteers and counterparts agree to use.
4 :00-4 :15
Day 4 evaluation
What worked well
What could be improved
22
Friday, April 27
8:00-8:15
Introduction
Energizer. Recap of Day 4.
8:15-8:45
Review of final survey/questionnaires
Someone from each group (SIDA1/2 and Palu ½) presents the final version of their tool.
Group approves or suggestions final, minor changes.
8:45-10 :00 Summarizing and using data from the pilot
Facilitator leads the group through a review of the data collected by the groups. Note differences
between men and women for each survey and what the data suggest about priority areas for
health education. What do you do with data? (Analysis) If evidence shows that behavior isn’t
changing, it could be because: the message is wrong, a key piece of the message is missing, the
treatment isn’t intense enough, the wrong people are targeted.
# hommes :
# femmes :
Nombre Total d'Individus
qui acceptent l'interview
# enfants :
Réponses correctes
# hommes :
Nombre de personnes
qui connaissent les
symptomes du
paludisme
# femmes :
# enfants :
Summarize all the results, by key result, of the pilot test and prepare a few summary statements
to share back with the village where the pilot took place.
10:00-10:15
Coffee Break
23
10 :15-11 :00 ‘Measuring Success’ Handbook
Lead group through a discussion of the contents of a handbook/guide for measuring outcomes.
Brainstorm issues to be addressed before collecting data, during data collection and after data
collection.
Each of these on a separate flip (before, during, after).
Before Data Collection
 Who will collect data?
 How will these people be trained? motivated?
 What resources will be needed for data collection?
 What are other steps to follow to prepare for data collection?
During Data Collection
 What should happen when data collectors arrive in a community?
 What community resources may be called upon to facilitate the data collection?
After Data Collection
 How will results be analyzed?
 Who will analyze the data?
 What will we do with the results?
 With whom will results be shared?
11:00-11:45 Next steps and individual action planning
Dr. Claude outlines her expectations about getting data from this effort, and how she plans to use
it. The point is to clarify for the group:
 When you would like to receive data for each of the health areas/topics.
 How you would like to receive this data—from whom?
 What you plan to do with all this data.
 How you plan to share results back with the group.
 Other issues/questions related to expectations and use of this data
Each Volunteer and counterpart should put together an action plan that lays out how they are
going to start measuring results when they get back to their sites. Emphasize who is going to do
what, by when.
11:45-12 :20 Final workshop evaluation and closing ceremony
12:20-1:30
Lunch
24
Atelier du Corps de la Paix/Burkina Faso :
Mesurer Les Succès
Ouagadougou, le 23 au 27 avril 2007
Le but de l’atelier : Améliorer nos capacités d’évaluer les résultats dans le domaine de la
promotion de la santé
Pendant l’atelier nous allons :
-
élaborer des outils d’évaluation
tester les outils
développer un guide pour l’utilisation de ces outils
Agenda
Journée 1
- Présentation et discussion de des activités dans le domaine de la promotion de la santé.
Pendant les 12 derniers mois, quelles sont les activités que vous avez faites ?
o le thème de l’activité (le paludisme, le VIH/SIDA, le PF, etc.)
o le type d’activité (le théâtre, les visites à domicile, les causeries)
o la fréquence de l’activité (une fois par mois, chaque semaine, tous les 6 mois)
o le nombre de personnes touchées (le nombre de participants)
-
Présentation et discussion des résultats désirés de ces activités. Suite à vos activités quels
sont les résultats que vous avez souhaités ?
o Chez qui avez-vous voulu effectuer des changements ? (les femmes enceintes, les
hommes, les jeunes, etc.)
o Quel type de changement ? (des connaissances, des attitudes, des comportements)
-
Sélectionner les résultats clés que nous allons mesurer. Analyser les résultats désirés :
o C’est quel type de résultat ? (connaissance, attitude, comportement)
o Est-ce que nos activités peuvent produire ces résultats ?
o Est-ce que le résultat est parmi les messages clés du Ministère de la Santé ? du
CNLS ?
o Comment peut-on savoir si on a atteint nos résultats?
Journée 2
- Développement des outils d’évaluation
o Comment savez-vous que vous avez atteints vos résultats désirés ? Quelle
information vous l’indiquerait ?
o Qui pourrait donner cette information ?
o Où collecteriez-vous cette information ?
o Quand collecteriez-vous cette information
Journée 3
- Finalisation du développement des outils
25
-
Préparation pour l’essai sur le terrain
Points à Considérer pour l’Enquête
Avant de commencer l’entretien, saluez votre interlocuteur/interlocutrice, présentez vous et
décrivez l’objet de votre visite. Dites-lui que vous avez besoin de lui poser quelques questions,
s’il vous donne la permission. Rassurer le (la) de la confidentialité et de l’anonymat de
l’entretien. Retirez-vous avec lui dans un endroit calme et discret. Rappelez-lui encore que sa
participation est volontaire, qu’il (ou elle) n’est pas tenu de répondre à une question qu’il (elle)
juge trop intime mais que vous souhaitez dans la mesure du possible qu’il (ou elle) réponde sans
se gêner. Enfin, informez le (la) que sa participation s’inscrit dans le cadre d’un vaste projet qui
vise à améliorer le bien-être et la santé des populations que cela ne peut se faire sans
informations sur leur niveau de connaissance, leurs attitudes, comportements et besoins sur les
questions de santé.
-
Apprentissage de comment utiliser les outils
Journée 4
-
Test des outils sur le terrain
Utiliser les questions suivantes comme guide pour évaluer le pratique sur le terrain :
1. Quelle direction avez-vous suivi dans le village ?
2. Combien de personnes /ménages avez-vous intervuvé ? (au moins 5 à 10 personnes)
3. Comment est-ce que vous les avez choisis ?
4. Est-ce que la phase introductive de l’enquête est correcte ? (mis la personne à l’aise,
demandé le consentement informé, expliqué la confidentialité, etc.)
5. Est-ce que les questions sont claires ? Si non, pourquoi ?
6. Est-ce que l’ordre des questions est logique ? Si non, pourquoi ?
7. Est-ce que la longueur du questionnaire est appropriée ? Si non, pourquoi ?
8. Est-ce que le formulaire pour enregistrer les réponses est approprié ? Si non, pourquoi ?
9. Est-ce que vous avez trouvé les données souhaitées au CSPS ? Si non, qu’est-ce que vous
avez fait ?
10. Difficultés particulières ?
11. Recommandations ?
-
Discussion des leçons apprises de l’expérience sur le terrain
-
Amélioration des outils
Journée 5
- Discussion de l’analyse et l’utilisation des données collectées
- Révision du guide d’utilisation des outils
- Elaboration des prochaines étapes
26
Appendix B: Glossary
English
Results/outcomes
Data
Measures/tools/instruments
Survey
Respondent
Interviewer
Baseline data
Workshop
To measure
Knowledge
Attitude
Skill
Behavior
Sample
Sampling
Process
French
Les résultats
Les données
Les outils
L’enquête
L’enquêter
L’enquêteur
Les données de base
L’atelier
Mesurer
La connaissance
L’attitude
La capacité
Le comportement
L’échantillon
L’échantillonnage
La méthodologie
27
Appendix C : Malaria Survey
Questionnaire sur le Paludisme
Village _________________________________________
Enquêteur(s)____________________________________
Date: __________________________________________
Le Nombre Total d’Individus qui acceptent l’interview
Hommes
Femmes
Enfants
Notes:
28
Résultat Désiré #1: Les personnes connaissent les Symptômes
du Paludisme
Hommes
Femmes
Enfants
Le nombre de
personnes qui
connaissent les
symptomes du
paludisme
Quels sont les symptômes du Paludisme?
Réponses Correctes:
Symptômes
Hommes
Femmes
Enfants
La fièvre
Le coma
La fatigue
Les vomissements/la
nausée
Les
frissons/tremblement
s
Maux de tête
La douleur musculaire
Maux de ventre
Le vertige
Manque d’appétit
Autres:
29
Réponses Non Correctes:
Hommes
Femmes
Enfants
Un oiseau a
attrapé l’enfant
Autres:
Résultat Désiré #2: Les personnes connaissent la voie de
transmission du paludisme
Hommes
Femmes
Enfants
Le nombre de
personnes qui savent
comment on attrape le
paludisme
Comment est-ce qu’on attrape le paludisme?
Réponses Correctes:
Hommes
Femmes
Enfants
Moustique
Piqûre de moustique
Réponses Non Correctes:
Hommes
Femmes
Enfants
Les mouches
La première pluie
Les mangues
Le mais frais
30
L’huile
L’eau sale
Nourriture (sale)
Etre Sale
Manger Grave
(beaucoup)
Autres:
Résultat Désiré #3: Les personnes connaissent les moyens de
prévention du paludisme
Hommes
Femmes
Enfants
Le nombre de
personnes qui savent
comment éviter le
paludisme
Comment est-ce quelqu’un peut se protéger contre le paludisme?
Réponses Correctes:
Symptômes
Utiliser/Dormir sous
moustiquaire
imprégnée
Utiliser/Dormir sous
moustiquaire
Eliminer l’eau
stagnante
Eviter l’eau sale
Hommes
Femmes
Enfants
Porter les manches
longues
Prendre les
medicaments (les
femmes enceintes)
Eviter les moustiques
Utiliser les spirals
31
Pomper
Insecticide
Autres:
Responses Non Correctes:
Hommes
Femmes
Enfants
Porter les
amulettes
Eviter les
mouches
Eviter les pluie
Eviter les
mangues
Médicament
Traditionnel/Marc
he
Autres:
Résultat Désiré #4: Les femmes enceintes et les enfants moins
de 5 ans dorment sous moustiquaires (imprégnées)
Ménages
Le nombre de ménages
ou on dit que les
femmes enceintes et
les enfants moins de 5
ans dorment sous
moustiquaire
imprégnée
Est-ce que vous avez une moustiquaire (imprégnée)?
Hommes
Femmes
Enfants
Hommes
Femmes
Enfants
Oui
Non
Si non - Pourquoi?
32
Pas d’argent
Ce n’est pas disponible
Autres:
Si oui - Est-ce que je peux voir vos moustiquaires?
Utilisation de Moustiquaire
Moustiquaire Attachée
Moustiquaire non
attachée
Apres avoir vu les moustiquaires imprégnées - Qui dorment sous les
moustiquaires?
Hommes
Femmes
Enfants
Père
Mère
Enfants (moins de 5 ans)
Enfants (plus de 5 ans)
Résultat Désiré #5: Les femmes enceintes prennent la chimio
prophylaxie contre le paludisme pendant la CPN (question pour les
femmes seulement)
Oui
Non
Le nombre de femmes
enceintes qui prennent
le chimio prophylaxie
contre le paludisme
pendant la CPN
Aux Femmes - Est-ce que vous avez un enfant moins de 1 an?
Si “oui” - Avez-vous suivi des CPN?
Si “oui” - Est-ce que vous avez pris la chimio prophylaxie contre le paludisme aux CPN
pendant votre dernière grossesse?
33
Femmes
Oui
Non
Aux Femmes Enceintes - Est-ce que vous faites les CPN?
Si “oui” - Au CSPS, avez-vous pris la chimio prophylaxie contre le paludisme?
Femmes Enceintes
Oui
Non
Où avez-vous appris cette information sur le paludisme ?
Hommes
Femmes
Enfants
Performance de Teatre
CSPS/Infirmier
Sensibilisation
Un Ami
Autres :
34
Appendix D : Enquête sur le VIH/SIDA
Village/Quartier:___________________
Nombre de personnes enquêtées:
hommes
Age : 10-14 ___________
15-20 ___________
21-24 ___________
25+ ___________
femmes
__________
__________
__________
__________
1. (Tout le monde) Qu’est ce que c’est que le SIDA? -(Si nécessaire, reformuler : Avez-vous
déjà entendu parler du SIDA ?)
2. (Tout le monde) Comment est-ce qu’on peut attraper le SIDA?
Ne pas lire ces réponses possibles. Notez-les seulement si l’enquêter les dit.
Correctes
Incorrectes
________Sang
_________Moustique
________Mère-Enfant
_________Poussière
________Relations sexuelles
_________Manger avec les gens infectés
_________Salive
_________Utiliser la même toilette
_________Etre proche/dormir à coté
_________Embraser une personne infectée
_________Par les vêtements
_________Autres moyens
_________Je ne sais pas
3. (Tout le monde) Comment est-ce qu’on peut éviter le SIDA?
Ne pas lire ces réponses possibles. Notez-les seulement si l’enquêter les dit.
Correctes
Incorrectes
_______Utiliser les condoms
___________Utiliser les condoms chaque fois
_______Etre fidèle
___________Dormir sous une moustiquaire
_______Eviter les objets
___________Utilisant le même condom plusieurs fois
souillés par le sang
___________Exciser les filles
_______S’abstenir
___________Utilisant plusieurs condoms à la fois
___________Ne s’approcher pas aux personnes infectées
___________Ne pas manger avec personnes infectées
___________Autres moyens
___________Je ne sais pas
4. (Tout le monde) Est-ce que vous pensez que la mère VIH+ peut accoucher un enfant
VIH-?
_________Oui
____________Non __________Je ne sais pas
35
5. (Tout le monde) A quel moment est-ce que la mère peut donner le VIH à son bébé?
Ne pas lire ces réponses possibles. Notez-les seulement si l’enquêter les dit.
Correctes
Incorrectes
_______pendant la grossesse
_______Jamais
_______pendant l’accouchement
_______autres moments
_______en allaitant le bébé
_______Je ne sais pas
6. (Tout le monde) Est-ce que vous avez entendu parler d’un programme à l’hôpital pour
protéger un bébé contre le SIDA ?
_______________Oui
________________Non
__________Pas sure
7. (Tout le monde) Pensez-vous qu’il y a des hommes ici au village qui portent les capots?
_______________Oui
________________Non
__________Pas sure
8. (Tout le monde) Pensez-vous qu’il est important d’utiliser les capots pendant les
rapports sexuels avec les partenaires en dehors du mariage ?
_______________Oui
________________Non
__________Pas sure
8.a. Si oui…Pourquoi?
8.b. Si non…Pourquoi pas?
9. (Tout le monde) Est-ce qu’il y a des gens avec le SIDA dans ce village, ou les villages
voisins?
_____________Oui ____________Non __________Pas sure
9.a. Si non, pourquoi les hommes portent les condoms ?
10. (Tout le monde) Est-ce que tu pourrais saluer quelqu’un qui a le SIDA?
_____________Oui ____________Non __________Pas sure
11. (Tout le monde) Est-ce que tu pourrais manger avec quelqu’un qui a le SIDA?
_____________Oui ____________Non __________Pas sure
12. (Hommes et Femmes) Si quelqu’un dans votre village avait le SIDA, comment est-ce que
on traiterait cette personne?
Pas de stigma
Stigma
________comme n’importe qui
_______on la chasserait
________on la soutiendrait
_______on la mettrait à l’écart
________autre
_______autre
13. (Elèves) Accepteriez-vous qu’un élève VIH+ fasse la même classe que vous ?
_____________Oui ____________Non __________pas sure
14. Où est-ce que vous avez appris l’information sur le SIDA?
_________au pièce de théâtre
_________à l’école
_________des amis
_________à la radio
_______au CSPS
_______autre
36