Multiple Indicator Cluster Surveys
Survey Design Workshop
Survey Quality Control
MICS4 Survey Design Workshop
Objectives
• Identify factors affecting the accuracy and reliability
of survey data
• How to prevent and correct errors
• The essential role of supervision in the field and
providing feedback to the team
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Preparation for fieldwork
• Questionnaire
– Adaptation (relevant, tailored to needs)
– Translation (for uniformity)
– Pretest (good flow, capture relevant information)
• Sample design and size
– For reliability, accuracy, feasibility
• Interviewers
– Selection (systematic process, have surplus)
– Training (trainers, length, practice)
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Data Collection
• Logistics & discipline
• Implementation of sample
• Monitoring of fieldwork
MICS4 Survey Design Workshop
Data Collection: Logistics & discipline
•
•
•
•
•
•
Material & equipment (questionnaires/forms)
Transport
Accommodation
Payments to field staff
Discipline & reasonable expectations
Keep the morale of the “troops”
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
• Non-response: failure to obtain information for
selected households, eligible women, or children
• A potentially serious bias that can be minimized
• Interviewers will need to make return visits to
households (call-backs)
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
Types of non-response:
1. Interviewer is unable to do the selected
household
2. Interviewer unable to meet eligible
respondents
3. Respondent refuses to be interviewed
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
Types of non-response:
1. Interviewer is unable to do the selected
household.  Why?
– Structure not found
– Occupied structure inaccessible
– Structure non-residential, vacant, or demolished
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
Types of non-response:
2. Interviewer unable to meet eligible
respondents.  Why?
– No one at home at time of call
– Respondent temporarily absent
Will need call-backs
MICS4 Survey Design Workshop
Call-backs
• Interview that is not completed requires a “callback” or follow-up visit
• Three call-backs required (different times and
days)
• Supervisors and interviewers keep track using
control sheets
• Requires good tracking of work to ensure that all
planned interviews are completed before leaving
cluster
MICS4 Survey Design Workshop
Data Collection: Implement. of sample
Types of non-response:
3. Respondent refuses to be interviewed.
 What to do?
– Approach respondent from her point of view
– Postponed interview to another day
– Have field editor carry out the interview
The number of refusals should be closely
monitored. Need to find out reasons for
frequent refusals.
MICS4 Survey Design Workshop
Data Collection: Monitoring field work
• Training is a continuous process
• Observation and supervision throughout the
fieldwork are a part of the training
• Team supervisors and field editors play very
important roles in continuing this training and in
ensuring the quality of MICS4 data
MICS4 Survey Design Workshop
Data Collection: Monitoring field work
•
•
•
•
Fieldwork control sheets
Direct observation of interviews
Review of completed questionnaires (editing)
Spot-checking households
MICS4 Survey Design Workshop
Fieldwork Control Sheets
• Interviewers’ work is monitored and evaluated by
keeping accurate record of assignments
• Both supervisors and interviewers have control
forms to maintain
• These forms should be returned to the director of
field operations along with the completed
questionnaires
• Interviewer is responsible for ensuring that control
sheet is up-to-date
MICS4 Survey Design Workshop
Observing Interviews, part 1
• To evaluate and improve interviewer
performance
• To look for errors and misconceptions that cannot
be detected through editing
• Why? Precise but inaccurate answers
• Who? The supervisors or field editors (females
only for women’s questionnaire)
MICS4 Survey Design Workshop
Observing Interviews, part 2
• How often? 5-6 interviews per week, more at
start of fieldwork
• How? Just take notes without disrupting the
interview
• What to do after? Supervisors speaks to
interviewers
MICS4 Survey Design Workshop
Editing Questionnaires in the Field
• Fundamental to survey quality
• Need to ensure:
– Accurate and complete information in each
questionnaire
– Correct count of questionnaires
MICS4 Survey Design Workshop
Editing Questionnaires in the Field
• Done daily by field editor; supervisor can
assist
• Finish all questionnaires before leaving
cluster
• Provides an opportunity to identify and
correct mistakes before leaving cluster
• Results and errors are discussed with
interviewers
MICS4 Survey Design Workshop
Systematic Spot Checking
• How to spot-check household
composition?
– Supervisors should complete columns 2-8 in the
HH questionnaire and compare with that of the
interviewer
– Check about 5% of households (5-6 per week)
– All team members must be spot-checked;
provide feedback if necessary
MICS4 Survey Design Workshop
Evaluating Interviewers Performance
• Daily discuss the quality of interviewers work
• Point out mistakes discovered during observation of
interviews or noticed during editing
• Discuss examples of actual mistakes, but be careful not to
embarrass individual interviewers.
• Re-read relevant sections from the Interviewer's Manual
with the team to resolve problems.
• Encourage the interviewers to talk about any situations
they encountered in the field
• Discuss whether situations are handled properly, and how
to do it in the future
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Field supervision
• Staff from survey headquarters
– Senior staff
– Quality control team
• Purpose
– Visit teams to observe interviews, review work
– Re-visit selected clusters; spot-check households
• Very important during first few weeks of fieldwork! (Possible
to start teams in one location?)
MICS4 Survey Design Workshop
Overview of presentation
•
•
•
•
Preparation for fieldwork
Data collection
Field supervision
Data processing
MICS4 Survey Design Workshop
Data Processing
• Integral aspect of survey quality (DP programmes,
double-entry, secondary editing)
• Problems detected in DP should be communicated
back to teams (phone calls and visits)
• Field-check tables
• Observations from reviewed questionnaires
MICS4 Survey Design Workshop
GENERAL INSTRUCTION TO EDIT
QUESTIONNAIRES
According to MICS Manual
Under no circumstances should we
make up an answer
MICS4 Survey Design Workshop
Download

Survey Quality Control