What are Online Research Methods?
Tristram Hooley & Jane Wellens
What are online research methods?
• Research methods that utilise the internet as
medium for research
• Sometimes referred to as internet mediated
research
• Includes research methods designed to
investigate both online & offline phenomenon
Types of ORM
•
•
•
•
•
•
online questionnaires
synchronous and asynchronous interviews
virtual ethnographies
online experiments
web analytics & content analysis
+ others
Online Research Methods - Challenges
• Draw on a wide range of methodological, theoretical
and disciplinary traditions
• Understanding the range of tools, environments and
online cultures is a precursor to successful research
• Technological and social environments change year on
year
• Increased blurring of the online and offline
environments creates a need for new composite
methodologies
• Potentially poses new and different ethical
considerations
Advantages of online research?
• Huge uptake of ORM, especially online questionnaires
• Useful in particular situations e.g. researching online
populations and previously difficult to contact groups
• Can mitigate distance and space and so
internationalise research
• Can save time and money
• Decrease personal risk
• Different dynamics of communication:
- more neutral venue?
- more thoughtful responses?
- different participants open up/contribute?
But….disadvantages
• But issues of digital access and digital literacy can limit
who we talk to
• Drop off rates for questionnaires high
• Issues of identity verification
• Challenges to building rapport in interviews
• Circumstances of interview beyond interviewers control
• Technical obstacles
• Still need degree of technical competence and
institutional support (although this is getting easier all
the time)
Online Questionnaires - key questions
What are
your
research
questions/
aims?
Who are
you talking
to?
How are
you talking
to them?
Sampling
Recruitment
Non-response
Question type
Survey length
Language
Advantages of online questionnaires
• Increasingly common approach (familiar to
respondents)
• Speed and volume of data collection
• Savings in costs (to researcher)
• Flexible design
• Data accuracy
• Access to research populations
• Anonymity
Online questionnaires - challenges
•
•
•
•
•
•
•
•
•
•
Sample bias
Measurement error (Sax et al, 2003)
Non-response bias
Length, response and dropout rates
Technical problems
Ethical issues
Recruitment
Identity verification
Response rates
Identifying appropriate tools & implementation
Design issues
Will your questionnaire be easy to use?
Usability
Or will the interface, colours and question
types frustrate people and put them off?
Will your questionnaire be accessible?
Accessibility
Or will it crash on old computers, render
weirdly on different operating systems and
be impossible to use with a screen reader?
Will your questionnaire be doable?
Doability
Or are your multi-media dreams running
ahead of your technical ability and are your
data arriving in an unusable format?
Improving response rates
•
•
•
•
•
•
•
•
•
•
•
Make contact before mailing the survey e.g. an introductory
letter
Provide information that builds trust e.g. names and photos of
the researchers
Engage gatekeepers and encourage them to endorse the survey
Think about how your brand (e.g. the university) will be
perceived by the subjects
Provide clear instructions on how to complete the questionnaire;
Limit the amount of personal information you request
Use simple questionnaire format and avoid complex or openended questions
Design survey so it takes approximately 10 minutes to complete;
Do not include more than 15 questions
Send one or two follow up reminders
Emphasise confidentiality (if appropriate)
Key resource: Fan, W., & Yan, Z. (2010). Factors affecting response rates of
the web survey. Computers in Human Behavior , 26 (2), 132-139.
Some Software Options
•
•
•
•
•
•
•
Google Forms http://docs.google.com FREE
Lime Survey http://www.limesurvey.org FREE
BOS http://www.survey.bris.ac.uk CHEAP
SurveyMonkey http://www.surveymonkey.com
Snap http://www.snapsurveys.com
Using the functionality of your website or VLE
Bespoke systems
So….
The quantity of information that may be
generated, and the speed at which responses
can be collected, can result in pleasing piles of
data- but we should be wary of being seduced by
sheer quantity; data is only useful if it is
representative of the larger population.
Wakeford (2000, 33)
Online Interviews – key questions
• Why use online interviews?
• What type of interview (synchronous or
asynchronous)?
• One-to-one or group?
• Text based or using audio or video?
• What software/service will you use?
• How are you going to record, retrieve and
analyse data?
Online Interviews: advantages
• Carry out interviews with a very geographically
dispersed population
• Interview individuals or groups who are often difficult to
reach, such as the less physically mobile or the socially
isolated or those living in dangerous places.
• Provide savings in costs to the researcher
• Supply ready transcribed interview data, quickly,
providing fast and cheap alternatives to face-to-face
interviews
• Reduce issues of interviewer effect as participants
cannot 'see' each other
Online interviews - challenges
•
•
•
•
•
•
•
Trust and rapport building
Design of interview schedule
Keeping people on topic
Dealing with technical hitches
Maintaining momentum
Dealing with ‘silence’
Guaranteeing the ethical rights of respondents
including informed consent, confidentiality and
privacy
Some software options
• Adobe ConnectNow
http://www.adobe.com/acom/connectnow/
• TinyChat http://tinychat.com/ (probably need to
upgrade for anything serious)
• GoogleTalk http://www.google.co.uk/talk/
• Many VLEs also have chat and virtual
classroom tools that can be used for this kind
of purpose.
So...
• The data collected by virtual interviews can be rich and
valuable to the researcher, but the potential of on-line
research should not be exaggerated: many of the
issues and problems of conventional research methods
still apply in the virtual venue
• Moreover, it is unlikely that online interviewing is going
to replace face-to-face interviewing but rather it is
another option in the methodological ‘toolkit’
Online research ethics
• There is currently a big push to embed research ethics
more formally in the culture of the social sciences.
• Yet there is far less of a consensus about online
research ethics.
• This asks the question “is there anything special about
the online environment that requires new set of ethical
guidelines?”
Key resource: Economic and Social Research Council (2005). Research
Ethics Framework.
Ethical decision making and Internet research
• Ess & AoIR (2002) stresses “Ethical pluralism” and
argues that “there is more than one ethical decisionmaking framework used to analyze and resolve those
[internet research] conflicts.”
• Note: the world of chatrooms, MUDs and MOOs and
USENET newsgroups, described in this report in 2002
is now radically transformed.
Key resource: Ess, C. and AoIR Ethics Working Committee (2002) Ethical
decision-making and internet research http://www.aoir.org/reports/ethics.pdf
Where does it fit in?
One of the problems of mapping existing research ethics
onto the online world is the difficulty of deciding how to
categorise the spaces that you are observing/interacting
with.
Public
Private
Published
Informal
Writing
Speech
Personal
Anonymous
Broadcast
Identified
Key resource: Internet Research Ethics: Digital Library, Resources Centre,
and Commons - http://internetresearchethics.org/
Ethical Challenges
“The great variety of human inter/actions
observable online and the clear need to study
these inter/actions in interdisciplinary ways
…This interdisciplinary approach to research
leads, however, to a central ethical difficulty: the
primary assumptions and guiding metaphors and
analogies - and thus the resulting ethical codes can vary sharply from discipline to discipline,
especially as we shift from the social sciences.
Ess, C. and AoIR Ethics Working Committee (2002)
Careful and critical use ORMs
•
•
•
•
So ORMs are not a shortcut `easy option’
Many issues and problems of onsite research remain
Divide between onsite/online methods inappropriate
Must be used, carefully and critically and appropriately
in light of each specific research topic
Attempting to undertake online data collection is far easier than
successfully accomplishing it. For those who chose to perform it, they
must do so deliberately and cautiously.
(Best and Krueger, 2004)
While online methodological frameworks are in constant flux, change
is not necessarily always progressive: there is a need for online
researchers to practice their 'craft' with reflexivity.
(Madge and O’Connor, 2005)
Further reading
Exploring online research methods
http://www.geog.le.ac.uk/orm/
Tristram’s ORM bibliography on CiteULike
http://www.citeulike.org/user/pigironjoe/tag/online_research_methods
Tristram’s blog
http://adventuresincareerdevelopment.posterous.com/
I talk about ORM sometimes and technology often on this
blog.