Monitoring Quality of
Social & Behavioral
Change Activities
Tom Davis
Senior Specialist for SBC (TOPS)
Senior Director for Program Quality
Improvement (FH)
What methods do you use
to monitor the quality of
your SBC activities?
• Spot checks
• Pre/Posttests
• Health Facility Assessment (especially
Exit Interviews)
• Mini-KPCs
• Quality Improvement and Verification
Checklists
What is
performance?
Performance is how
well someone does a
task or job.
What are the processes that
you do over and over again
in your projects?
• Monitoring a child’s growth
• Demonstrating proper
seedling spacing
• Promoting immediate
breastfeeding.
From a 20% improvement…
• We know that ITN use can cut child deaths by
about 20%
• A 20% improvement of promotion of ITNs and
prenatal care seeking 
• 20% increase in the number of households using
an insecticide-treated net
• Since ITN use could drop the U5MR in Haiti from 87 to
70 (20% lower)…
• …if you are working with 100,000 children, and
increase coverage from 60% to 72% (20%↑), an
additional 12,000 would be sleeping under a net.
• Result: 204 children’s lives are saved
([12,000 * 87/1000] – [12,000*70/1000] = 204
• … And many more if including other effects
of antenatal care.
REMEMBER: Small changes
in often repeated tasks...
...can cause large
changes in impact.
Is what we do “dangerous”?
• YES! How we do it can mean saved lives. If we do it
wrong, people can die.
• There are already high “failure rates” associated with the way things are
done in communities where we work with (e.g., poor crop yields, high
maternal and child mortality rates).
• There’s a great opportunity to avoid these failures, and do things
optimally, and checklists can help assure we do it right each time.
• FH project in Mozambique saved an estimated 6,848 children’s lives in
five years at a cost of $441/life saved. A lot of things had to go right for
that to happen, and checklists were used to help assure that the main
things (key processes, espec. HP) were done as they should be done.
Why did Juan die?
Was there something that the CHW should have done differently to
avoid the death?
YES: The CHW should have:
• Asked the mother about the child’s illness and dietary history.
• Gave the mother advice on ORS and feeding during illness.
• Explained to the mother dangers signs during illness.
• Asked the mother to repeat back the main messages.
• A checklist could have saved Juan’s life!
A Quality Improvement
and Verification Checklist
(QIVC) is a tool that has
three purposes:
• to encourage,
• to monitor, and
• to improve a development
worker and the work that he
or she does.
Reviewing the Checklist and
Watching the Process
• English CDW QIV Checklist (click to download):
www.caregroupinfo.org/docs/CDW_Practice_Pr
omotion_QIVC_(English).doc
• French CDW QIV Checklist (click to download):
www.caregroupinfo.org/docs/CDW_Practice_Pr
omotion_QIVC_(French).doc
• Watch Skit #1 (Click Here to watch home visit to
Community Development Worker) [2 mins]
• Watch Skit #2 (feedback session with
Community Development Worker) [15 mins]
• Click Here to proceed to next slide (after
watching skits)
What should you say to the
health or ag worker when
you visit him or her and
want to use a quality
checklist?
Don’t worry! This is not a test, but a tool to
help you improve your work. Do your work as
you would normally do it. Generally, you should
not talk much with me during the educational
session (or home visit, etc.) or ask me any
questions. I can answer questions when you
are through, but not during the educational
session. I am going to observe only.
What comments did the
Supervisor make during the
visit?
No comments were made. You should explain
to the mothers or farmers that you are
visiting the development worker to see the
work being done, but you should not make any
comments during the process. When you are
through, you can compliment the development
worker in public if she did the process more
or less well. Do not give any criticism, though,
in public. Your role is to observe the process.
Where did the Supervisor
talk about each of the points
in the checklist?
In private with the development
worker, not in front of other
people
Why did the Supervisor
explain the checklist to the
worker if it is a monitoring
and evaluation tool?
Because it is a method for improving the
worker’s performance, not just to monitor
the worker’s performance. The thing that
we consider to be perfect performance
should not be a secret. This definition of
perfect performance should be clear and
consistent in the minds of all the personnel
involved with the work. Because of that,
you should work jointly to create and
perfect a checklist.
Did the Supervisor speak to
the person in a very
threatening or reprimanding
way? Why?
No. The Supervisor needs to be very
gentle so the development worker does
not feel shame. Even if the development
worker did very poorly on the checklist,
you should not emphasize the score, but
instead emphasize changes since the last
performance evaluation – if he or she is
improving or not. The purpose of the
checklist is to help the worker see how
s/he is improving his or her work, little by
little, or quickly, as the case may be.
It is important to be very
gentle because:
1. Instructing gently works.
2. We are often working with
volunteers, people who are
donating their time to the project
and will not continue to work for
us unless they are made to feel
good about their work.
3. We are modeling how we want
people to treat each other.
It is important to be very
gentle because...
4. To use a new skill, a person has to have selfconfidence, which is usually nurtured by other
people. Self-confidence is key to behavior
change in mothers, health and ag workers, and
ourselves. If a health or ag worker feels shame
after an evaluation, s/he may not have the
confidence to change his or her behavior (i.e.,
performance).
5. The development workers in your charge will
teach and instruct in the way that they are
taught and instructed. If you are very gentle
with them, they will learn to be more gentle -and thus effective -- with others.
If you are using a checklist for a
process that does not happen
everyday (e.g., the checklist for case
management of diarrhea), how could
you use the checklist to monitor the
person’s performance?
You can use the checklist in the form of a
simulation – you pretend to be a mother
with a child with diarrhea or a farmer who
needs to learn how to do something. With
you are doing the evaluation this way, the
Supervisor should fill out the checklist
after the simulation. Afterwards, the
Supervisor and development worker should
discuss the results in the usual way.
Principles of Giving
Feedback
1. The person needs to be reminded of
the purpose of the evaluation.
2. Share the evaluation tool with the
workers. Give them a copy before
the evaluation.
3. Generally, give feedback in private.
You can give positive feedback
publicly.
Principles of Giving
Feedback
4. Acknowledge the positive. In
general, positive feedback needs
to outweigh negative feedback by
about three to one for the person
to feel that the remarks were
balanced. Look for good things to
say about the person’s
performance. Be encouraging.
Principles of Giving
Feedback
5. For things done wrong or in a
substandard way, ask the worker
about the item. It is much easier
emotionally for a person to identify
his or her own mistakes than to have
someone else point them out. This
also gets the person in the habit of
reflecting on their own performance,
which they will need to do to have
perfect performance.
Principles of Giving
Feedback
6. Ask the worker to summarize what
s/he is doing that is most
encouraging (correct), and then
what s/he can do to improve
further; assure that the person
heard everything.
7. Ask the worker to summarize what
s/he needs to do better; assure
that the person heard everything.
Principles of Giving
Feedback
8. Ask the worker for
commitment to improve those
things. People are more likely
to do what they commit to do
verbally in front of others.
What sort of management
information do you think this
tool can provide?
1.
Information on System-wide
Problems: What problems do
most workers have with the
process? What parts of the
process need to be restructured?
On what parts of the process do
workers need to be retrained?
2. Information on People Problems:
Which workers are having the most
problems? Who needs additional
supervision or support?
Another Example of a System Problem
Which of these things is more
important to determine:
System problems or people
problems?
Probably the system problems. If
everyone or almost everyone is doing
part of the process poorly, that could
jeopardize everyone’s performance
and the project’s success, especially
if the part that they are doing poorly
is essential to the process’
effectiveness.
In order to calculate a
worker’s QIVC score:
1. Count how many questions have the
answer, “YES.” This is the numerator.
2. Now count how many questions that you
used during the evaluation. You don’t
always use each line. This number is
the denominator.
3. Divide the numerator by the
denominator. This is the person’s
quality score.
Quality Scores for GM/P, By Health Agent
(FOCAS/MEI Haiti CS Project)
Quality Score
105%
100%
95%
90%
85%
80%
75%
1
2
3
4
5
6
7
8
9
Series1
93%
97%
92%
94%
100%
100%
97%
100%
100%
Series2
88%
94%
97%
97%
100%
97%
100%
100%
Series3
80%
97%
94%
100%
100%
100%
100%
100%
Series4
86%
97%
97%
97%
100%
97%
100%
100%
Measurement No.
Series1
Series2
Series3
Series4
Click here to download all training files
QI Checklist Theme (process evaluated)
Immunizations, Management of Diarrhea,
Conducting Training Sessions, Individual
Counseling, NFE Methods: Songs/Poems,
Stories, Puppetry, and Guided Testimonies.
Conducting Educational Sessions
Growth Monitoring & Promotion
KPC Survey Interviewing
IMCI Home Visits (Children 2m to 4 years)
Rally Post activities (immunization, vitamin
A/iodine dosing/education, deworming, iron
supplementation), Acute Respiratory Infection
(ARI) Management
Clinic/Hospital Management of Severe
Malnutrition
Monitoring Manager’s Tool (for evaluating and
improving supervisors use of QI checklists)
Available in These
Languages
English only
English, Spanish, Haitian
Creole
English, Spanish, Haitian
Creole
English, Spanish, Haitian
Creole
Spanish only
Haitian Creole
English (draft)
English, Spanish, Haitian
Creole, Portuguese
How to use the data from the QIVC to
Evaluate and Improve Your Program
•
You should use a quality checklist monthly for
the workers that have an unacceptable score
(e.g., less than 80%). You should continue to use
the checklist each time that you visit those
workers until their scores are acceptable.
Some options for improvement:
 Suggest that the worker look over
the training notes from a previous
workshop.
 Suggest the worker review the
checklist.
 Suggest the worker pair up with
another more experienced worker.
How to use the data from the QIVC to
Evaluate and Improve Your Program
• You do not have to continue to use the
checklist on a monthly basis with a worker
once s/he achieves a high score (e.g., over
90%). You could then use it on a semiannual or yearly basis, either with all of the
workers, or a randomly chosen sample (e.g.,
30 Ag. Extension Agents chosen at random)
assuming that you are using it strictly for
evaluation of quality and not improvement
of quality.
How to use the data from the QIVC to
Evaluate and Improve Your Program
• You should use the list less frequently
(e.g., every 2 or 3 months) for the workers
who have acceptable scores.
How to Know if a Worker is
Progressing, and Follow-up when
Opportunities for Improvement
(OFIs) are found
• You should maintain a file on each
worker. This file should have each
workers scores on the quality
checklist. The best way to keep the
scores is with a graph (similar to the
growth monitoring card: months on x
axis, score on Y axis).
Click here to download
QIVCL Worksheet
Data
 Looking at the November
data only, who has the best
performance? (Lemonjello)
 Who needs the most help? (Julia)
 Who improved the most
over the three-month
period?
(Julia)
 Who improved the least?
(Lemonjello, but he had a high
score to begin with.)
If you were the Project
Manager or a Supervisor, what
would you do when you saw this
November data?
 Supervisors should devote more time to
Julia and Oranjello.
 Supervisors should ask themselves why
these two are performing more poorly
than the others in order to improve future
training/support.
 Supervisors should figure out why
Lemonjello learned the process so well so
that the others can learn from him.
How to Identify System Problems,
and Opportunities for Improving
the Worker’s Skills and Follow-up
when OFIs are found
 Identify parts of the process that the
majority of workers are not doing properly.
 Talk to workers about reasons for failure
(e.g., training was unclear; not enough
supervised practice; no job aide to remember
steps/correct procedure).
 Implement changes and remeasure.
In general, QIV Checklists
are used to evaluate
things that:
• a worker does
• that have multiple steps;
and
• that can be observed (seen
and/or heard).
Can this be monitored
with a QIV Checklist?
Teaching a farmer how to use leguminous
plants for fertilization and mulching
Yes No
Management of acute respiratory
infection Yes No
Being honest in your work Yes
No
Prenatal education during a home
visit Yes No
Can this be monitored
with a QIV Checklist?
A Ag workers ideas about problems
in the community
Yes No
The number of farmers who have
improved silos Yes No
Dosing of children with vitamin A
Yes No
Can this be monitored
with a QIV Checklist?
How many children have signs of
anemia
Yes No
Teaching treatment of receptacles
so insects or other pests won’t
Yes
damage grain
No
Teaching members of an association
to write bylaws
Yes No
Several Guidelines for
Development of QIV
Checklists
 Do not base quality checklists only on
“problem areas” (i.e., something that is
being done poorly), but on oftenrepeated processes.
 Develop the evaluation tools jointly
with workers
 Each line is a question.
 Number each question.
Several Guidelines for Development
of QIV Checklists
 Order the questions in their order in
the process.
 If there are parts of the process that
are not always done, separate them out
with two lines.
 Each questions should be able to be
answered with a YES or NO where YES
is the correct behavior (e.g., “did not
make fun of the mother”).
Several Guidelines for Development
of QIV Checklists
 You can use questions that require the
person filling in the form to grade
performance using a scale.
 Each question should be something that
can be observed (seen or heard).
 The questions should be detailed enough
to identify specific problems and
specific things being done properly.
This is extremely important.
 Generally, checklists should be no longer
than two pages.
Several Guidelines for Development
of QIV Checklists
 You should include a space at the end
for comments.
 Periodically review and modify each
quality checklist. Change the process
based upon what seems to work and not
work, and new technical knowledge.
Experiment.
 When asking people who are illiterate or
have low literacy, try using pictures for
each question.
Several Guidelines for Development
of QIV Checklists
 Work towards getting uniform
performance on a particular process
first, THEN make changes to it to
improve it.
Creating draft QIVCs
• Work on your own to create a QIVC
for an often repeated task
Other Resources
• List of Social & Behavioral Change
Key Competencies (handout)
• List of FSN Network SBC Task Force
Approved Methods and Tools
• Please SHARE your QIV Checklists
with the FSN Network! Email me your
copies and we will have the
appropriate task force review it.
Review Questions on
QIVCs
1. What is performance?
2. For what type of processes should QIV
checklists be developed?
3. What are the three main purposes of
QIV checklists?
4. With QIV checklists, where should the
supervisor provide the majority of the
feedback to the development worker?
Review Questions on
QIVCs
5. What is “positive monitoring”?
6. In order for a person to feel that
positive and negative feedback are
properly balanced, what needs to
happen? What should the balance
between positive and negative
feedback be like?
7. What is the best way to give feedback
about something that a worker did
wrong during a session? What should
you do first?
Review Questions on
QIVCs
8. What sort of management data is
provided through the use of QIV
checklists?
9. In general, QIV checklists are
used to evaluate what types of
things?
Thanks!
This presentation was made
possible by the generous support
of the American people through
the United States Agency for
International Development
(USAID). The contents are the
responsibility of Food for the
Hungry and do not necessarily
reflect the views of USAID or the
United States Government.