Online Surveys
as
Data Collection Instruments
in
Education Research:
A Feasible Option?
EMASA 2012 - Dr Lorinda Minnaar
Introduction
Study Title
School Governance and Leadership for Sustainable
Quality Education for All
Study Theme
School governance and leadership
Key Purpose of the Study
To explore and establish the ability of parent members
of school governing bodies to govern and manage
schools’ finances
Research Design
Approach
Large-scale, exploratory, quantitative study
Sample
All (1433) schools in the Western Cape (WCED)
Participants
School principals
Data Collection Instrument
Online survey
Motivation for Choice of an
Online Survey
• Large sample – 1433 schools
• Financial and human resource
constraints
• An online survey is a convenient
and cost-effective data collection
method
• Access to“Checkbox”
Review of Literature on Online Surveys
Problem
Despite an extensive
search, existing
literature
documenting the
feasibility of this data
collection method in
an education research
context, seems
deficient
Solution
Adapt existing
literature to the
purpose and design of
the study, located
within a South African
educational research
context
Types of Online Surveys
E-mail an introductory letter or invitation to
respondents with a Uniform Resource Locator
(URL) hyperlink to a web-based survey
Send respondents a survey embedded in an e-mail
message with or without an attachment
Place a general request for respondents in an
electronic communication environment such as a
web page to complete a survey
Advantages of Online Surveys
 Global reach
 Large samples easily accessed
 Time-efficient and convenient
 Low preparation and administration costs
 Environmentally friendly
 Question diversity
 Question sequencing capabilities
 Question branching capabilities
 Ease of data entry and analysis
Disadvantages of Online Surveys
Perceptions as Spam
Lack of participant online experience
and expertise
Computer configuration
Impersonal
Security and confidentiality
Planning the Online Survey
Planning the online survey entailed:
• Drafting the hard copy
• Placing instructions, questions and answer options in
simple tables
• Continually revising instructions and questions to
ensure they did not contain design errors such as
confusing ambiguities, double, split and negative
questions and value-laden concepts
• Consulting an expert in the field of school governance
to seek confirmation that questions were valid,
meaningful, grammatically correct and appropriate to
the research theme
Designing the Online Survey
Designing the online survey entailed the following
methodological considerations:
• Selecting a visual layout comprising design elements
such as background colour, font face, size and colour
• Uploading Stellenbosch University’s oak leaf logo,
which not only lent colour and formality to the survey
but immediately informed participants of the survey’s
origin
• Avoiding and eliminating complex, vague and
ambiguous instructions that may perplex and frustrate
participants to such an extent that they exit the survey
before completing it
Designing the Online Survey (Cont).
• Checking the survey’s length to ensure it was not too
long and that participants would not spend too much
time completing it
Long surveys with matrix type
questions that have numerous
items and alternatives, increase
respondent fatigue and boredom
and frequently result in
respondents adopting a
“satisficing” behaviour where
they select the same scale-point
to rate all items without paying
them much attention (Mora 2010)
Question Formats
Radio Buttons
A list from which a participant selects one option
Question Formats (Cont).
Checkboxes
A list from which the participant may select multiple options
Question Formats (Cont).
Drop-Down Lists
A space-saving format
Question Formats (Cont).
Open-Ended Single-Line Text
Allows freeform text with formatting rules
Question Formats (Cont).
Open-Ended Multi-Line Text
Allows freeform text within a box with a specified number of
rows and columns
Question Formats (Cont).
Matrixes and Rating Scales
Explore the frequency of participants’ behaviour or their
attitudes toward certain phenomena
Activating the Online Survey
Activating the online survey included the following
aspects:
• Sourcing the e-mail addresses of schools in the sample
• Checking e-mail addresses for errors
• Deliberating on which day, week and month to activate the
survey
• Determining the period permitted for participants to respond
and the final date for submission
• The frequency of reminders to be sent to participants
• Compiling an appropriate invitation to principals to
participate in the survey
Activating the Online Survey (Cont).
The invitation to participate in the online survey
Response Rates and Results
Activation
and
Reminders
Number of
Surveys sent
Number of
Surveys
Completed and
Returned
Response Rate
1st
1 433
64
4%
2nd
1 433
44
3%
3rd
1 433
37
2%
4th
1 433
220
15%
24%
Overall Response
Rate
Low response rates appear typical of online surveys
Response Rates and Results (Cont).
Participants’ Responses to the Quality of the Survey
Bar/Column Plot of Count of 1
Spreadsheet14 3v*6c
45
40
35
18%
15%
15%
30
25
20
15
10
5
1%
0%
0
3 Relevant
1 Quick
2 Interesting
0%
4 Long
Other
Quality of survey
5 Complicated
Count of 1
Conclusion
“How feasible are Internet-based online surveys used as data
collection instruments for educational research?”
• Denscombe (2009, 281) asserts “There are still
several unanswered questions concerning the use of
online surveys as data collection instruments for
research purposes”.
• Despite implementation of salient methodological
considerations, the relatively low response rate
alerts educational researchers to use online surveys
as data collection instruments circumspectly.