ONLINE SURVEYS
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
1
Online surveys are relatively new – since
the mid 1990’s
Why are they so popular?
– They are easy to develop and implement
– They are relatively low cost
– You can access large #s of people at low
marginal costs
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
2
But, is it the best method?
Shall I use an online survey?
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
3
Things to think about before you
decide on an online survey
• Do your potential respondents
have easy access to the internet?
• Are they comfortable using the
internet?
• Do they trust internet communications and
interactions?
• Do they have the skills to open and complete the
online survey?
• Will their hardware and software affect how the
questionnaire will be seen?
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
4
What do we know about internet access
and use?
• We know that different audiences (age, cultural
background, etc) interact with technology differently
• 80% of American homes have personal computers
but only 57% have access to high-speed internet
– Issue of coverage and speed
• Access and use is affected by
– Educational level
– Household income (56% of homes with combined
incomes of less that $40k do not have internet
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
5
About internet access and use…
• Access and use is affected by
– Age: 18-34 year olds - 78% have internet
– Nonwhites have lower access
– Many who use internet do so only at work (there
may be restrictions on use); at school
•
Comfort and safety
– Lack computer skills; familiarity with internet
– Distrust of internet
– Fear viruses and other malicious software
Source: Thomas, 2009, wwww.thefreelibrary.com
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
6
This means that…
• Many people may be left out when using an
internet survey
• Those who respond may differ considerably from
those who do not respond
= bias in results
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
7
So, who do you want to hear from?
What do you want to know?
Is a survey the best method?
Is an online survey the best type of
survey to use?
Remember, your choice of data collection
method depends upon your evaluation purpose
and be appropriate for the audience.
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
8
Online survey tools – what they can do:
• Help you define your survey questions
• Provide possible response options
• Send your respondents to a link to answer the
survey online
• Allow you to customize the look and feel of the
questionnaire
• Provide for skip logic
• Help you with analysis: most provide simple
reports that summarize answers to each
question; some let you download into Excel;
more advanced packages allow for more
complex analyses
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
9
Online survey tools
There are many popular tools for
conducting online surveys including
• SurveyMonkey
• Zoomerang
• SurveyGizmo
• Checkbox
• LimeSurvey
Each has its strengths and weaknesses. Do a Google search to
learn more about each. For a comparison, see Survey of Survey
Tools, the OSU Web Accessibility Center at http://www.wac.ohiostate.edu/workshops/survey_of_surveys/
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
10
So, you’ve decided that an online survey is best. For
UWEX Cooperative Extension employees, Computer
Technology Services will help you with
Go to CE Technology Services at
http://www.uwex.edu/ces/techservices/webdev/surveys
Follow their instructions; OR, obtain your own online
survey tool and implement your surveys yourself.
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
11
Planning your online survey
Planning an online survey is similar to planning any
type of survey. There are a series of steps to
consider:
1. Involve others – who should be involved in
planning and conducting the survey?
2. Define what information you will collect – what do
you want to know?
3. Determine respondents – get email addresses
– Set sampling strategy, if appropriate
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
12
Steps continued…
4. Plan for data analysis - What do you want to be
able to report? Do you want percentages,
comparisons, words?
5. Develop the questionnaire
6. Pilot test the questionnaire , cover letter, etc.
7. Design the analysis plan
8. Plan for communications
9. Set budget, timeline and responsibilities
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
13
A note on anonymity and confidentiality in
online surveys
Anonymous means that NO ONE can identify who
provided the information
– This is difficult to assure when the survey is administered
online (internet or intranet)…so, don’t promise anonymity!
Confidentiality means that you are able to identify the
person but you guarantee that the information will not
be identified with the person
– This applies to all aspects of data collection, analysis and
reporting
– When reporting and communicating, ensure that no names
or other identifying information is used
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
14
A few tips
• Follow same pre-survey and follow-up processes
as for other surveys
• Use email for the pre-survey notification and
post-survey follow-up; add other methods
(telephone)
• Personalize the email message vs group
message or group address
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
15
Online survey implementation steps
•
•
•
•
•
•
Personalize all contacts as much as possible
Consider incentives – improves response rate
Use multiple contacts and vary the message
Strategically time the contacts
Use another mode to contact respondents if
possible
Keep email contacts short and to the point
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
16
Online survey implementation steps,
cont…
•
•
•
•
Make sure email is not flagged as spam
Make sure your name and address are
clear/reputable; Include an informative subject
line
Provide clear instructions for how to access the
survey
Make sure the opening screen informs
respondents that they are in the proper place
and describes steps to take
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
17
Online survey implementation steps,
cont…
• Assign each respondent a unique ID number
• Keep within the limits of the web server’s
capabilities
• Create a system for dealing with bounced emails
• Establish procedures for dealing with inquiries
and issues
• Monitor progress and early responses
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
18
Two things we want to accomplish with
our online survey
1. To get the highest response possible
2. To get as accurate information as possible
So, you need to pay attention to:
 Increasing response rate
 Good questionnaire design
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
19
Increasing response rate
There is a separate powerpoint slide set for
“increasing your response rate”. Please find it
on the same web site.
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
20
Response rate may be affected by how
the survey is distributed. In zoomerang,
there are two options:
1. Use email addresses. you need individual names with current
email addresses. Send invitation plus other communications reminders, track respondents, and receive detailed deployment
information.
– This is the only option that allows you to calculate the
response rate
2. Post the url in a public place – when you don’t have individual
addresses. E.g., in newsletter, on web site, bulletin board; or
send to an email list (not individual addresses).
– You may not know who sees/received the url – you do not
know the total number of people who received the
questionnaire.
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
21
Questionnaire design
There is a separate powerpoint slide set and
various materials to help with creating the
questionnaire for your online survey. Go to the
section on this web site, Common methods for
collecting data
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
22
Checking in…
How would you rate your comfort level NOW in
being able to develop and implement a useful
online survey?
•
•
•
•
•
Very poor
Poor
Fair
Good
Very good
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
23
Resources
• Dillman, D., Smyth, J., Christian, LM. 2009.
Internet, Mail and Mixed-Mode Surveys: The
tailored design method. John Wiley and Sons.
© 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation
24