Designing Surveys
and Reporting the Results
Workshop for Human Rights Researchers and
Campaigners
AAAS Scientific Responsibility,
Human Rights and Law Program
Eric Vance
Director of LISA
Assistant Research Professor
Department of Statistics
Material from this workshop will be
available for download at
www.lisa.stat.vt.edu/?q=workshops
June 8, 2012
Washington, DC
Module 1: Introductions
Key concepts you learned:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
______
Questions you still have:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
~ Individual Exercise ~
Write down examples of using statistics/data in your work (especially surveys):
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_____________________________________________________
_____
~ Team Exercise ~
Share your best example with your team.
Module 2: Survey Fundamentals
Key concepts you learned:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
______
Questions you still have:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
~ Team Exercise ~
As a team, pick the answer choice you think would be the best method of data
collection for the following scenario:
The Millennium Challenge Corporation wants to know the impact on water
quality and health of their program to drill borewells and install handpumps in
villages in Mozambique that don’t have access to clean water.
A. Convene focus groups of village women to ask them in-depth questions
about the quality of their water and the health of their families before and
after the installation of the handpumps (Before & After).
B. Survey households in multiple villages about water quality and health
(Before & After).
C. Sample the old wells and the new handpumps and measure levels of
pathogens, organic matter, dissolved O2, and salts in the water at the
source and in the home.
D. Examine local hospital records for incidences of diarrhea (Before & After).
E. Use the results from a recently published study in a neighboring country to
estimate the impact of the new handpumps.
Module 3: Questionnaire Design
Key concepts you learned:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
______
Questions you still have:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
~ Quiz ~
Answer the questions below, first individually, then as team. Team answers
should be recorded on the provided scratch-off answer sheet.
1. “Governments have a moral obligation to do what's best for the majority of
their people and protect minority rights” is a double-barreled statement:
A. True B. False
2. Asking respondents their opinion on the statement above is likely to lead to
A. Selection bias B. Coverage error C. Nonresponse bias D. Measurement error
3. Which of the following is NOT a benefit of pilot testing a survey:
A. Check for selection bias B. Estimate variability in responses C. Reduce
measurement error D.Change the wording of the questions E.None of the above
4. Open-ended questions are used more often than closed-ended questions in
surveys because the responses are more genuine: A. True B. False
5. Which of the following factors into a sample size calculation:
A. Budget B. Time constraints C. Required precision
D. Expected variability of responses E. All of the above
Module 4: Survey Implementation
Key concepts you learned:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
______
Questions you still have:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
~ Team Exercise ~
Pick one example of an upcoming (or past) survey your team discussed after
Module 1 and decide, as a team, how to implement this survey. If this was a
past project, how would you modify it using what you’ve learned today?
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
Module 5: Reporting Results
Key concepts you learned:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
______
Questions you still have:
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
~ Team Exercise ~
Which presentation of data or the reporting of a result in Module 5 was new to
you and might be something you will use in the future? Discuss as a team.
Module 6: Questions and Answers
~ Individual Exercise ~
Review your questions from the previous modules and pick the one you most
want answered.
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
______
~ Team Exercise ~
Dr. Vance will answer one question from each team. As a team, decide what is
your most pressing question, write it on a blank sheet of paper, and hand it to
him.
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
__
_________________________________________________
_
_________________________________________________
_
EXTRA NOTES
_________________________________________________
_________________________________________________
__
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
_________________________________________________
________
_________________________________________________
_________________________________________________
_________________________________________________
___
_________________________________________________
_________________________________________________
_________________________________________________
___
_________________________________________________
_________________________________________________
_________________________________________________
___
_________________________________________________
_________________________________________________
_________________________________________________
___
_________________________________________________
_________________________________________________
_________________________________________________
___
_________________________________________________
_________________________________________________
_________________________________________________