Multiple Indicator Cluster Surveys
Survey Design Workshop
Survey Quality Control
MICS4 Survey Design Workshop
Objectives
• Identify factors affecting the accuracy and reliability
of survey data
• How to prevent and correct errors
• The essential role of supervision in the field and
providing feedback to the team
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Preparation for fieldwork
• Questionnaire
– Adaptation (relevant, tailored to needs)
– Translation (for uniformity)
– Pretest (good flow, capture relevant information)
• Sample design and size
– For reliability, accuracy, feasibility
• Interviewers
– Selection (systematic process, have surplus)
– Training (trainers, length, practice)
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Data Collection
• Logistics & discipline
• Implementation of sample
• Monitoring of fieldwork
MICS4 Survey Design Workshop
Data Collection: Logistics & discipline
•
•
•
•
•
•
Material & equipment (questionnaires/forms)
Transport
Accommodation
Payments to field staff
Discipline & reasonable expectations
Keep the morale of the “troops”
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
• Non-response: failure to obtain information for
selected households, eligible women, or children
• A potentially serious bias that can be minimized
• Interviewers will need to make return visits to
households (call-backs)
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
Types of non-response:
1. Interviewer is unable to do the selected
household
2. Interviewer unable to meet eligible
respondents
3. Respondent refuses to be interviewed
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
Types of non-response:
1. Interviewer is unable to do the selected
household.  Why?
– Structure not found
– Occupied structure inaccessible
– Structure non-residential, vacant, or demolished
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
Types of non-response:
2. Interviewer unable to meet eligible
respondents.  Why?
– No one at home at time of call
– Respondent temporarily absent
Will need call-backs
MICS4 Survey Design Workshop
Call-backs
• Interview that is not completed requires a “callback” or follow-up visit
• Three call-backs required (different times and
days)
• Supervisors and interviewers keep track using
control sheets
• Requires good tracking of work to ensure that all
planned interviews are completed before leaving
cluster
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
Types of non-response:
3. Respondent refuses to be interviewed.
 What to do?
– Approach respondent from her point of view
– Postponed interview to another day
– Have field editor carry out the interview
The number of refusals should be closely
monitored. Need to find out reasons for
frequent refusals.
MICS4 Survey Design Workshop
Data Collection: Monitoring field work
• Training is a continuous process
• Observation and supervision throughout the
fieldwork are a part of the training
• Team supervisors and field editors play very
important roles in continuing this training and in
ensuring the quality of MICS4 data
MICS4 Survey Design Workshop
Data Collection: Monitoring field work
•
•
•
•
Fieldwork control sheets
Direct observation of interviews
Review of completed questionnaires (editing)
Spot-checking households
MICS4 Survey Design Workshop
Fieldwork Control Sheets
• Interviewers’ work is monitored and evaluated by
keeping accurate record of assignments
• Both supervisors and interviewers have control
forms to maintain
• These forms should be returned to the director of
field operations along with the completed
questionnaires
• Interviewer is responsible for ensuring that control
sheet is up-to-date
MICS4 Survey Design Workshop
Observing Interviews, part 1
• To evaluate and improve interviewer
performance
• To look for errors and misconceptions that cannot
be detected through editing
• Why? Precise but inaccurate answers
• Who? The supervisors or field editors (females
only for women’s questionnaire)
MICS4 Survey Design Workshop
Observing Interviews, part 2
• How often? 5-6 interviews per week, more at
start of fieldwork
• How? Just take notes without disrupting the
interview
• What to do after? Supervisors speaks to
interviewers
MICS4 Survey Design Workshop
Editing Questionnaires in the Field
• Fundamental to survey quality
• Need to ensure:
– Accurate and complete information in each
questionnaire
– Correct count of questionnaires
MICS4 Survey Design Workshop
Editing Questionnaires in the Field
• Done daily by field editor; supervisor can
assist
• Finish all questionnaires before leaving
cluster
• Provides an opportunity to identify and
correct mistakes before leaving cluster
• Results and errors are discussed with
interviewers
MICS4 Survey Design Workshop
Systematic Spot Checking
• How to spot-check household
composition?
– Supervisors should complete columns 2-8 in the
HH questionnaire and compare with that of the
interviewer
– Check about 5% of households (5-6 per week)
– All team members must be spot-checked;
provide feedback if necessary
MICS4 Survey Design Workshop
Evaluating Interviewers Performance
• Daily discuss the quality of interviewers work
• Point out mistakes discovered during observation of
interviews or noticed during editing
• Discuss examples of actual mistakes, but be careful not to
embarrass individual interviewers.
• Re-read relevant sections from the Interviewer's Manual
with the team to resolve problems.
• Encourage the interviewers to talk about any situations
they encountered in the field
• Discuss whether situations are handled properly, and how
to do it in the future
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Field supervision
• Staff from survey headquarters
– Senior staff
– Quality control team
• Purpose
– Visit teams to observe interviews, review work
– Re-visit selected clusters; spot-check households
• Very important during first few weeks of fieldwork! (Possible
to start teams in one location?)
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Data Processing
• Integral aspect of survey quality (DP programmes,
double-entry, secondary editing)
• Problems detected in DP should be communicated
back to teams (phone calls and visits)
• Field-check tables
• Observations from reviewed questionnaires
MICS4 Survey Design Workshop
GENERAL INSTRUCTION TO EDIT
QUESTIONNAIRES
According to MICS Manual
Under no circumstances should we
make up an answer
MICS4 Survey Design Workshop