Survey Research using
Emerging Technologies:
Considerations for Design,
Data Collection, Sampling
and Recruitment for
Smartphone and Tablet
Based Surveys
JOS Conference June 10 and 11, 2015
Trent D. Buskirk, Ph.D.
1
JOS AAPOR
30th Anniversary
Short Course,
Conference
2014 Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Instructor: Trent D. Buskirk, Ph.D.
Current Position: Vice President of Statistics
and Methodology @ Marketing Systems Group
Lives in: Saint Louis, MO
Focus on:
Use of new technologies for data collection
Mobile Phone Sampling
Dual Frame Designs and Weighting
Mobile Analytics and Paradata Collection and Analysis
Addicted to:
Playing Prince to my little Princesses
APPS and iPhones
SAS, SUDAAN and R
Tennis
Boost 101.9 (www.boost1019.com)
http://www.m-s-g.com/Web/Index.aspx
2
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Acknowledgements and Thanks!
Colleagues at MSG for their assistance and support and for
providing thumb drives for participants
My Family for their patience as I persisted through course prep
JOS Conference Organizing Committee for the Invitation to
Present
3
3
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Day
1
Agenda & Goals
Background: Nuts and Bolts of our new Mobile Universe
Understanding the landscape that is before us to set the stage
New types of data available – burden, privacy and availability
Framework for Data Collection on Mobile Devices
Active versus Passive data collection methods
Developing Emerging Best Practices: A blend of old and
new
Types of Mobile Surveys (and what we know now from the
Literature)
Free Resources for Mobile Web Surveys
Recommendations for Internet/App based survey solutions (Top 14
EBPs)
4
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Day
2
Agenda & Goals
Developing Emerging Best Practices: A blend of old and
new
Recommendations for Internet/App based survey solutions (Top
14 EBPs)
Computing – just how do you make these surveys?
From Websites to Mobile Web Surveys (Web design methods for
Mobile)
Overview of “Mobile Versions” for Popular Online Survey
Packages
Goal: Leave with a solid appreciation of:
Potential role of new technologies in data collection;
Different ways you can use smartphones to collect survey data
Some practical “best practices” for deploying mobile surveys
5
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Reference Note
I have included web links as well as QR codes for
nearly every data source we will discuss today.
Some popular QR code reader APPS for your
smartphone include:
I-nigma
Kaywa Reader
BeeTagg
The following website allows you to enter your exact
mobile provider and provides a recommendation as to
the QR code that is optimized for your phone’s camera
http://www.mobile-barcodes.com/qr-code-software/
6
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
6:59
Section 2:
Emerging
Technology and
the Rise of
Smarter Cell
Phones
7
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
What are Feature Phones?
A feature phone is a mobile phone that
combines the functions of a PDA and a
mobile phone. These models serve as
portable media players, web browsers and
camera phones.
Feature phones generally refer to devices
that are more low-end compared to higherend Smartphones.
The most important difference between a
feature phone and a Smartphone is the kind
of operating system installed in it.
8
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
What is a Smartphone?
A Smartphone is a high-end mobile phone that
combines the functions of a personal digital
assistant (PDA) and a mobile phone.
In general, a Smartphone will be based on an
operating system that allows it to run applications,
use e-mail, and surf the internet.
Apple's iPhone runs the iOS
BlackBerry Smartphones run the Black -Berry OS
Other devices run Google's Android OS, HP's webOS, and
Microsoft's Windows Phone.
9
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
For the Visual Learners in the Room – a Summary
The Picture
Summary!
10
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Smart vs. Feature Phones Worldwide
11
JOS
30th
Source:http://www.nielsen.com/us/en/newswire/2013/how-the-mobile-consumer-connects-around-the-globe.html
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Smartphone penetration for US Mobile Subscribers
over the past two years
“To every thing there is a season, and a time to every purpose under the heaven,”
Ecclesiastes 3:1
12
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
What is a Tablet Computer?
A tablet personal computer (tablet PC) is a portable
personal computer equipped with a touchscreen as
a primary input device, and running a modified
desktop OS designed to be operated and owned by
an individual.
Tablet PCs are larger than smartphones. Common form
factor dimensions are (height, width, thickness and
weight):
5-inch Tablets: 6" x 3.2" x .4" @ .75 lbs.
7-inch Tablets: 7.5" x 5" x .5" @ 1 lbs.
9-inch Tablets: 9.3" x 6" x .5" @ 1.2 lbs.
10-inch Tablets: 9.8" x 7" x .5" @ 1.4 lbs.
13-inch Tablets: 12.5" x 8" x .5" @ 2 lbs.
13
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Devices come in all sizes (and shapes)
Screen dimensions are measured in pixels (px) both for
height and width. Another measure that combines these to
give an overall measure is dots per inch (dpi)
1 megapixel = 1,000,000 pixels.
Common widths (in pixels) for various devices:
768
px
1024
px
1680
px
320
px
14
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile Devices introduced in 2013
15
JOS
30th
Source: Millennial Media, 2013: http://www.millennialmedia.com/mobile-intelligence/mobile-mix/
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile Devices Covering the USA
240 Million Mobile Users (age 13+) in the U.S.
91% of All U.S. Adults have a Cell Phone
156 Million Smartphone Owners in the U.S.
(age 13+)
64% of U.S. Adults have a Smartphone
10% of American Adults are Mobile-Only
Internet Users; 15-21% among 18-29 YOs
82 Million Tablet Owners in the U.S.
42% of U.S. Adults have a Tablet
16
Sources: comScore (2015) and Pew Research (2015)
http://www.comscore.com/Insights/Presentations-and-Whitepapers/2015/2015-US-Digital-Future-in-Focus
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile Devices Covering the World
4.77 Billion People Worldwide Using
Mobile Phones
2.03 Billion People Worldwide Using
SmartPhones
400 Million People Worldwide Own
Tablet Devices
788 million mobile-only internet users
by 2015 worldwide (Cisco, 2014)
17
Sources: http://www.emarketer.com/Article/Smartphone-Users-Worldwide-Will-Total-175-Billion-2014/1010536
http://www.techvibes.com/blog/tablet-adoption-rate-exceeding-that-of-smartphones-2013-10-15
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Smartphone Users Worldwide: 2012 – 2018
18 Source: emarketer (2014): http://bit.ly/1uWDLtg
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile device penetration continues to rise in the U.S.
19 Source: Comscore, 2014 http://bit.ly/1hh9j8L
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Tablet Penetration in the EU-5 (2012-2018)
UK
Spain
Germany
France
Italy
70.0%
Share of population
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
2012
20
2013
2014
2015
2016
2017
2018
Source: http://www.statista.com/statistics/271001/penetration-rate-of-tablets/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Smartphones in the U.S. - They’re almost everywhere!
Just over three in four
(77%) mobile
subscribers in the U.S.
owned a smartphone
during the most recent
three-month period
(Oct-Dec 2014),
Nielsen, 2015
Android Operating
System most popular
smartphone OS in US
followed by Apple IOS.
Source: Nielsen (2015):
http://bit.ly/1MDPB4j
21
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Most Common Smartphone Operating
System Per Country
Note: These estimates are based on Mobile
Web Activity rather than Device Ownership
22 Source: StatCounter: GlobalStats (2015) http://gs.statcounter.com/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
A different picture for Tablets
SOURCE: MILLENNIAL MEDIA’S Q2 2013 MOBILE MIX™ REPORT
http://www.millennialmedia.com/mobile-intelligence/mobile-mix/
23
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Most Common Tablet Operating System
Per Country
Note: These estimates are based on Mobile
Web Activity rather than Device Ownership
24 Source: StatCounter: GlobalStats (2015) http://gs.statcounter.com/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Smartphone Penetration in the US by Key
Demographic Features
Source: Nielsen (2014):
http://bit.ly/1pyhyfG
Source: Nielsen (2015):
http://bit.ly/1MDPB4j
25
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Age and Income Level Strongly Associated
with Smartphone Ownership
2012
Source: Nielsen Wire (2012) :
http://www.nielsen.com/us/en/newswire/
2012/survey-new-u-s-smartphonegrowth-by-age-and-income.html
2013
Source: Pew (2013)
http://www.pewinternet.org/2013/06/05
/smartphone-ownership-2013/
26
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Slightly Different Picture for Tablets Bigger
Screen, Smaller Income
The age distribution of Tablet users seems to skew older
compared to smartphone users (comScore, 2013)
The annual HH income distribution for Smartphone users appears
to skew richer than for tablet users (comScore, 2013)
27
JOS
30th
Source: comScore (2013) http://bit.ly/1msbiaV
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Internet traffic by device over the last 3 years (11-13)
28
Source: http://www.mediabehavior.com/articles/the-whole-story-internet-use-by-platform/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile Devices and Internet Use Today
According to comScore’s US Digital Future in Focus report (2014):
Smartphone and Mobile devices account for 57% of U.S. Internet usage
Smartphones alone have exceeded Desktops
http://bit.ly/1lFEDlj
According to mobiThinking (2012) approximately 25% of U.S. Mobile
Web users are mobile-only.
many mobile-only are older people and many come from lower income
households
http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats/b#mobile-only
According to Pew Research’s Home Broadband Report (2013)
approximately 10% of Americans indicate that they do not have a
broadband connection at home but that they do own a smartphone
http://www.pewinternet.org/2013/08/26/home-broadband-2013/
29
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile Only versus Desktop Only Internet Users (U.S.)
30
Source: comScore (2015): http://bit.ly/1EPWBJl
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Average daily minutes spent online by internet
users worldwide as of 2nd quarter 2014, by device
PC, desktop or tablet
Mobile
Daily time online in minutes
400
350
300
250
93
108.6
258.6
267.6
256.8
2012
2013
2014
74.4
200
150
100
50
0
31
JOS
30th
Source: GlobalWebIndex; ID 319732
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Multiple Tasks for Multiple Platforms
32
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
With the surge in Mobile Internet Use comes
the rise of unintentional mobile respondents
Source: Maritz Research
33
JOS
30th
http://www.maritzresearch.com/
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile, Mobile Everywhere! We have to care!
With Mobile Device penetration
rising along with
Increases in internet activity on mobile
Increases on the email opening
activity on mobile
Rises in unintentional mobile survey
respondents
The case is clear – we need mobile
survey research strategies, best
practices and resources to meet our
respondents where they seem to be
going! Mobile cannot be ignored!
34
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
6:59
Section 3:
Developing Best
Practices in the
New Landscape…
35
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Thinking Differently About Technology &
Survey Data Collection
Technologies often viewed as “survey-enabling” tools,
but not as “data collection vehicles” in their own right
The advancement in apps and hardware for these mobile
devices and the social networks that are accessed using
them has opened the avenue for a broader spectrum of
data that can be collected as part of a “survey”
Location data via captured GPS coordinates for both
respondents and field workers
Scagnelli, et al. 2012
Olson and Wagner, 2015
Picture data captured via an app or via phone
Scagnelli et al. 2012,
Michaud, Buskirk and Saunders, 2014
Data Collection Gigs via Crowdsourcing
Duan, Lai and Link, 2013
36
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
A new era of survey research ushers in new data
collection opportunities
In-person, in-the moment, round the clock data are now
being collected, compared and explored in the context of
surveys
Health related outcomes via apps and peripheral devices (Bluetooth) and
hardware
Gregoski, et al. 2013
Abroms, et al., 2011
Kumar, 2012
Au-Yeung et al., 2012
(http://www.proteus.com/assets/AuYeung_NetworkedSystemForSelfManagement1.pdf)
Real-time diaries for wellness, activities and time use and trip/location data
Koskinen and Salminen, 2007
Fernee and Sonck, 2013
Graham and Cobb, 2013
Lai et al., 2010
Nitsche et al. , 2012
37
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
GPS analytics provided additional trip details and metrics not easily
captured via survey (Source: Nielsen, 2012)
38
Source: Scagnelli, Bailey et al. 2012
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
“Grab & Go” Purchase Locations
Each green dot is
a Grab & Go store
where a Nielsen
panelist made a
purchase
39
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile Devices aren’t just for Respondents!
We are also using these devices ourselves to collect data
In-person data collection via tablets is also on the rise
Seal, 2012
– http://www.burke.com/Library/Articles/John%20Seal_Article_CASRO%202012-2013%20Journal.pdf
Paudela, et al. 2013
– http://www.ghspjournal.org/content/1/2/277.full.pdf+html
https://www.dooblo.net/
http://www.surveypocket.com/
Using Mobile phone–assisted personal interviewing (MCAPI)
van Heereen et al., 2014
van Heereen et al., 2013
– http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3713928/
40
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Interface & Flow
User interface: usually an important consideration in data collection,
with new technologies it is essential
New & extended human-computer interaction concepts:
Visual appeal
Intuitive flow (more than simply “logical”)
Ease of use /functions are readily apparent
Few, if any, text instructions
Optimize mode capabilities and reduce user anxiety and frustration
Study of why people participate in mobile surveys (Bosnjak et
al 2010):
Perceived enjoyment; attitudes toward participation; self-expressiveness, &
trust factors most important
Blend current understanding of CAI systems with expanding
literature on human-computer interactions (use & interaction)
41
JOS
30th
Source: Bosnjak et al, 2010 http://ssc.sagepub.com/content/28/3/350.short
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Thinking about Mode Choice on Multiple Devices
The emergence of devices with internet access has created
unprecedented choices for online survey completion:
computer (lap/desk top),
Tablet,
Smartphone,
FeaturePhone (for the brave)
Other device (Sony PSP, your Smart TV, your Fridge, etc.)
42
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Thinking about Mode Choice on Multiple Devices
Buskirk, Walton and Wells (2013) developed a study to investigate
factors related to survey mode choice
Study employed a subset of smartphone, tablet and computer users from a
nationally representative probability online survey panel and replicated this
study for a large opt-in panel of similar users
Study preference data analyzed using Conjoint Analysis.
Factors included: Survey Mode (Smartphone, Tablet and Computer as well as
Paper and Pencil), Time/Incentive (5 to 60 minutes; 2 to 25 dollars), survey
sponsor, Incentive Type (Cash, Card, eCode, Charity) and Survey Topic).
Among the combination of survey time, incentive amt and type and mode,
Tablet and Computer mode had the top 5 combinations while smartphone only
had one combination preferred in the top 25 (ranked 21).
The degree of preference for smartphones increased as prior exposure with
survey modes (other than computer) increased. Similarly, preference for
computer mode decreased as prior survey exposure to on alternate modes
increased.
43
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Thinking Differently About Technology &
Survey Data Collection and Respondent Burden
Technologies often viewed as “survey-enabling” tools,
but not as “data collection vehicles” in their own right
What we normally ask ourselves:
How can we use technology to conduct a traditional survey?
We should be asking:
What are the questions / data elements we need to
answer/collect to provide insights into phenomena of interest?
How can technology be utilized to acquire that understanding /
those data?
What does respondent burden look like in the new landscape?
Are there new data available that we can collect passively using
new technologies that haven’t been easily collected before?
44
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Respondent Burden
The degree to which a respondent
perceives participation in a task as
difficult, time consuming, or
emotionally stressful.
Typically measured in terms of length
or administration time of a survey.
Most common approach to countering
respondent burden: incentives
“The concept of
respondent burden is
like that of the weather:
everyone talks about it,
but no one does
anything about it.”
Norman Bradburn,
1978
Focus on “traditionally defined”
Respondent Burden is no longer
enough …
45
Source: Norman Bradburn
http://www.amstat.org/sections/srms/proceedings/papers/1978_007.pdf
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Respondent Burden in the Mobile World
Respondent burden in mobile world is similar to respondent
burden in the regular world with a few differences
Two Groups: Personal and Technoogical
Personal: Safety
Respondents aren’t just responding at home anymore!
Steeh, Buskirk and Callegaro (2007) report respondents completing cell
phone surveys while boarding a helicopter.
Personal: Privacy
New data types like photo/voice may mean information is collected from
more than the consenting respondent
Age of assent on internet is younger than consent via phone (13 vs 18/19)
Personal: Time
To complete survey tasks
To load survey pages
46
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Respondent Burden in the New Mobile World, Cont.
Technological
Data consumption
Both receiving and sending information (survey questions/content and
submitting responses)
Nielsen (2012, Q3/Q4) Estimates that 96% of mobile phone users have
a data plan and that the average monthly bill is $66 (all mobile) and $93
for smartphone users.
Bandwidth
Slower bandwidth increases time required to send and receive data
Higher data consumption may lead to throttling by the provider –
meaning reduced bandwidth in future
Battery Drain
Persistent GPS capture drains mobile device batteries
Extra HTTP requsts can drain mobile device power
47
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Internet Access Method varies by Mobile Device Type
Internet access for US smartphone users seems to be
predominantly through Mobile Connections rather than wi-fi
connections;
However this pattern is different across operating system
48
Source: comScore (2013) http://bit.ly/1msbiaV
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Smartphone Use Statistics by Country
https://think.withgoogle.com/mobileplanet/en/
49
Source: Our Mobile Planet, Google, 2013: http://bit.ly/1HSHFwO
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Data Plan Limits and Smartphone Users
50
Source: Pew Research Center (2015) http://pewrsr.ch/1FlmRbv
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Respondent Expectations
Being involved in a scientific survey is a relatively rare occurrence
Little, if any, expectations by respondents
Respondent experience with new technologies is much different
Respondents are more savvy
Technology changes rapidly – some respondents will find some tasks difficult
or not easy to use.
[Example: Michaud, Buskirk and Saunders, 2014 Voice Data Entry]
Developing expectations from these experiences
Ease of use/intuitive
Speed
Usefulness/utility or fun/entertainment or both
Ability to share experiences w/ others
Location awareness
Auto detection/passive collection
51
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
The new paradigm: MOBILE Surveys
Smartphone Survey
Design Research
Social Science
Research
Buskirk and Andrus (2012)
Link and Buskirk (2012)
Kinesis White Paper (2010)
Buskirk et al. (2011)
Callegaro (2010)
Callegaro and Macer (2011)
Peytchev and Hill (2010)
Zahariev et al. (2009)
Smith (2011)
Bosnjak, Metzger, Graf (2010)
Lai et al. (2010)
Couper (2010)
Millar and Dillman (2012)
Human Computer
Interaction
Parush and Yuviler-Gavish
(2004)
Jones et al. (1999)
Sweeney and Crestani
(2006)
Fitts (1954)
Computer Science
and Web
Development
Koch, P-P. (2010)
Campaign Monitor (2012)
Kriss (2013)
Medallia (2013)
EmailVision (2013)
Goldberg, Fariden and
Alterovitz (2007)
Html5rocks.com (2013)
Zichermann and
Cunningham (2011)
Return Path (2013)
Pelletier (2013)
52
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
On Emerging Best Practices in the New Landscape…
Traditional Components of Best Practice:
Length/administration time of a survey
Respondent burden (time, number of phone minutes, number of
surveys)
Follow-up approaches/frequency, survey field periods
Incentives – cash or other tokens (gift cards, minutes, etc.)
New Components of Best Practices:
Balancing user engagement with user expectations/experience
Ease of use / intuitive
Speed of tool / interface / app
Respondent privacy expectations
Respondents sharing their experiences…
Visual Appeal, user interaction with tool and gamification
53
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
16:59
Section 4:
Types of Mobile
Browser Surveys
and What we
Know so Far
about Mode
Effects
54
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Various Approaches for Survey Data Collection
via a Smartphone
Use the regular phone mode
Use Short Text Messages (SMS)
Flash polls, Invitations, Synchronous Survey Interviews
App-based/administered surveys
Online surveys accessed via Smartphone Browser
Passive and Active versions
Hybrid Approach: App-Like Smartphone Browser
Surveys
Source: Buskirk and Andrus (2012) www.surveypractice.org.
55
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
SMS Surveys via Mobile Devices (Phones)
SMS Surveys involve a series of short communications
between surveyor and respondent (i.e. text messages)
Surveyor sends a question with answer choices to the
respondent (short snipets of usually 140 characters or less)
Respondent sends back the appropriate answer
Process continues as a series of asynchronous communications
Some 79% of cell phone owners say they use text messaging
on their cells (Pew, 2012)
56
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
SMS Surveys via Mobile Devices (Phones), Cont.
Nielsen estimates that on average, 764.2 text messages
were sent/received per month among all U.S. mobile
post-paid/contract users.
http://bit.ly/1QkUUFY
Studies using SMS for survey data collection are
emerging, for example:
Conrad et al. (2013) http://bit.ly/1gNVoms
Yan et al. (2013) http://bit.ly/QAL5LV
Mavletova and Couper (2014) http://bit.ly/1RGbEdk
Schober et al. (2014) http://bit.ly/1M5sqiO
57
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Online Surveys via Mobile Web Browser
Passive: Surveys intended to be completed online are
sometimes completed via mobile phone web browser (passive
mobile web browser approach).
Active: The online web-based survey is optimized for the
mobile environment. (think mobile versions of survey
websites)
Many survey software programs (e.g. SNAP, Qualtrics, etc.)
have mobile publishing options that optimize an otherwise
online survey for the mobile browser.
If you know the type of devices on which those sampled will
complete the survey, you can optimize the survey design for
the particular browser/screen resolution (e.g. Safari / iPhone).
58
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Passive versus Active Smartphone Surveys
Passive Mobile
Survey from Hotwire
59
JOS
Survey Appearance
after a “double tap”
Active Mobile Survey
Both viewed on an iPhone 5
30th
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
The Literature on Passive Mobile Browser Survey
Mode Effects is now ACTIVE!
Passive mobile browser surveys are surveys developed and
intended to be completed via computer.
Buskirk & Andrus (2012) hypothesized that passive mobile browser
surveys could have negative survey consequences. (http://bit.ly/1f0Qvez)
Fast forward 2 years: rise of mobile unintentionals + mass use of
passive mobile browser surveys = empirical findings…
Lower overall completion rates/higher break-off rates for Passive Mobile Surveys
accessed on smartphones (Decipher, 2013; Wells et al., 2013; Poggio et al. 2013)
Overall break-off rates among tablet users appear consistent with computer users
(Decipher, 2013; Wells et al., 2013; Poggio et al., 2013)
Overall median/mean completion time longer for smartphone users compared to
computer users (Wells et al., 2013 [~2 minutes longer; Decipher, 2013 [~3 minutes longer],
Cunningham et al., 2013 [about ¼ minute longer]).
Respondents on Smartphones tend to be younger, female and non-Caucasian and
primarily internet through phone users (Peterson et al., 2013; Wells et al. 2013)
Faster Time to Hit Survey Start Page (Cunningham et al. 2013 [~170 min sooner];
Peterson, 2012 [~1-5 hours sooner (median)]
60
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
The Perfect Storm Goes Mobile: The intersection of non-mobile
surveys and the rise of the unintentional mobile respondent
A recent internal study by Maritz (2013) has corroborated
earlier findings
by Medallia
(2012):
Second
Quarter 2013
Device Analysis
Completion rates for Mobile users are lower than for computer users
Completion
for surveys designed
for completion
online
via computer browsers.
% of Total
% of Total
% of Mobile Completion Time Relative
Starts
Completes
Completes
Percentage
to Desktop
All mobile phones
14.0%Quarter12.4%
100.0%
Second
2013 Device
Analysis 75.0%
iPhone
9.3%
8.4%
67.6%
72.1%
Android
4.3%
3.7%
29.8%
72.1%
Blackberry 5.0, 6.0 & 7.0 % 0.2%
1.2%
67.4%
of Total
% 0.2%
of Total
% of
Mobile Completion
Other mobile phones
0.2%
0.2%
1.3%
68.2%
Starts
Completes
Completes
Percentage
132.4%
129.7%
137.3%
Completion
184.4%
Time
Relative
to133.1%
Desktop
All mobile phones
iPad
iPhone
Desktop
Android
Blackberry 5.0, 6.0 & 7.0
Other mobile phones
132.4%
99.7%
129.7%
7.7 minutes
137.3%
184.4%
133.1%
14.0%
7.2%
9.3%
78.8%
4.3%
0.2%
0.2%
12.4%
100.0%
7.6%
NA
8.4%
67.6%
80.0%
N/A
3.7%
29.8%
0.2%
1.2%
0.2%
1.3%
Source:
Maritz Research
7.2% http://www.maritzresearch.com/
7.6%
NA
iPad
Desktop
78.8%
61
80.0%
N/A
75.0%
88.4%
72.1%
85.7%
72.1%
67.4%
68.2%
88.4%
99.7%
85.7%
7.7 minutes
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Passive surveys being reported in the literature
Articles dealing with online surveys being made
(passively) available to respondents are making their
way into the published literature and conferences.
1. Cunningham et al. (2013)
2. Bosnjak et al. (2013)
3. Poggio et al. (2013)
4. Poduska & Johnson (2010)
5. Peterson (2012)
6. Peterson et al. (2013)
1
2
3
http://bit.ly/RIyKGI
http://bit.ly/1hckNtB
http://bit.ly/1icvRCV
http://bit.ly/PquJVf
http://bit.ly/1gP5Fim
http://bit.ly/1i5WYo0
4
5
6
62
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Active and Passive Surveys being Reported
in the Literature
Baker-Prewitt and Miller (2013) http://bit.ly/PqzePP
Experiment Compared Active and Passive Mobile Browser Surveys to
Desktops and Laptops
Survey completion time, drop-out rates, straight-lining rates were all
worse for passive version compared to active mobile version
Peterson et al. (2013) http://bit.ly/1i5WYo0
Experiment to compare several active mobile versions of the survey to
a passive version as well as an app-based version and two versions for
PCs.
Survey length (about 1 to 3 minutes longer) and break-offs (2 to 3 times) were
longer/higher among passive mode compared to any of the active mobile
versions and both types of mobile had longer completion times than PCs
Survey satisfaction (4 Qs) ratings significantly lower for passive compared to
any active mobile or pc version. (Won’t do again rating 2 to 5 times as high
for passive compared to any active mobile approach)
63
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
APP-LIKE Mobile Browser Surveys
APP-LIKE Mobile Browser Surveys (A-LMBS) represent a hybrid of
the APP and the Active Mobile Browser Approaches.
A-LMBS are completed via the mobile web, but they rely on active
browser refreshment using a combination of PHP Programming and
Javascript to create “Active” Mobile web pages.
A-LMBS can be developed to run without a persistent internet
connection on multiple mobile browsers (HTML5)
A-LMBS offer native, APP like functionality under the mobile browser
context.
In today’s browser vernacular, app-like surveys might fall under the
class of Web apps which are defined as web pages that function
and appear almost app like but are accessed via browsers
64
Source: Buskirk and Andrus (2012) www.surveypractice.org.
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
A-LMBS Example- The Got Healthy Apps Study
65
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
A-LMBS Compared to Active Mobile Web Browser
A-LMBS
Approach
(iPhone)
A-LMBS
Approach
(Android)
Active
Mobile
Browser
Approach
Active
Mobile
Browser
Approach
(iPhone)
(Android)
66
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
APP-LIKE Mobile Browser Surveys in the Literature
Buskirk and Andrus (2014) report on an experiment comparing mode
effects for app-like and online computer surveys http://bit.ly/1n8JLxA
Completion rates for computer mode was significantly higher than for iPhone
Item missing rates were very low across both computer and iPhone survey modes
Survey completion times for iPhone mode was significantly less than Computer
Significantly higher app ownership reported from iPhone mode vs Computer
Mavletova and Couper (2013) present the results of an experiment that
randomly assigned panel members (Russia) to complete a survey about
sensitive topics using an app-like mobile version or PC version. One
month later, respondents were invited for a follow-up survey on the other
mode. http://bit.ly/1rl1aD4
Higher nonresponse, break-off ad longer completion times for Mobile Web compared to PC
Measurement differences reported across the two modes for Alcohol consumption and
income with less daily consumption and lower incomes being reported on mobile vs. PC
More surveys completed outside the home in the presence of strangers on mobile vs PC
No differential affect of satisficing by type of device uses (mobile or PC).
67
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
App-based/administered surveys
App-based Smartphone surveys administer and
collect data via an (native) app that is installed on the
sampled user’s Smartphone.
Surveys can be “pushed” to the app and then
executed by the end user without the need for
perpetual internet connection and can be designed to
take full advantage of the phone’s capabilities like
camera, voice, video, image capture, etc.
Data transfer for completed surveys can occur once internet
connection is established.
One popular example of this approach comes from
Confirmit/Techneos’ Survey On-Demand Application (SODA),
see:
http://www.confirmit.com/Home/Community/Data-Sheets/January-2014/Confirmit-SODA.aspx
68
Video File Courtesy of Frits de Geest of Confirmit
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
App Attack – The rise of Mobile Device Apps
According to comScore (2013)
Engagement with apps was 7 times that of engagement with web browsing,
on average for iPhone owners compared to only 2 times for iPad owners .
According to Flurry Analytics (2014)
Time spent on a mobile device for US consumers, on average, has risen to
2 hrs and 42 minutes per day (4 minute increase over 2013)
2 hours and 19 minutes on average per day on Apps and 22 minutes per
day, on average for the mobile web.
69
JOS
30th
Source: Flurry (2014): http://blog.flurry.com/?month=4&year=2014
Source: comScore (2013): http://bit.ly/1c9vneo
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
App usage among Smartphone Owners by Country
70
Source: Our Mobile Planet, Google, 2013: http://bit.ly/1HSHFwO
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Top 15 Apps within the US Measured by % Reach
71
Source: comScore Mobile Metrix (2015): http://bit.ly/1ycKQM4
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Considerations for an APP-Based Approach
The APP based approach offers the most options in terms of
formatting, gamification, and other options for data input
And it offers more control per operating system
One disadvantage is that it might be too much technology for the
task at hand:
Does a one-time survey need an app for normal survey tasks?
Apps must be downloaded before data collection occurs
Run the risk of “App Install Breakoff”
Apps themselves count against respondent data plans/onboard
memory required (download and memory)
Potentially increases fielding time (development and deployment);
72
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
App-Based Survey Approaches in the Literature
Link and colleagues (2013) explored recruitment for an app-based diary
study that was not part of a panel, but for a cross-sectional collection of
television viewing. http://bit.ly/QCXkb4
Recruitment was conducted via telephone based on dual frame RDD
samples for two local markets.
Once screened for eligibility, respondents were sent app-registration details
Bailey and Wells (2012) compared mode effects between an app-based
survey and an online (computer-based) version of the survey http://bit.ly/1k4dBjs
Completion rates and completion times similar across the two modes
Mobile app respondents typed more characters, on average, compared to
computer respondents on a consumer behavior question
iPhone Rs consistently entered longer responses in less time compared to android Rs
73
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Resources for Mobile Surveys
An overview of the survey research to date on mobile devices,
capabilities, privacy and legal issues and some discussion of
what’s on the horizon can be found in Chapter 1 of the: AAPOR
Emerging Technologies Task Force Report
Mobile Web Best Practice Guidelines produced by the World Wide
Web Consortium (W3C)
Gives very detailed suggestions on how to create web pages for mobile
devices, in general. Some of the information on touch interfaces, font sizes
and mobile versioning of online websites may directly pertain to survey
development and practice.
http://www.w3.org/2005/MWI/BPWG/Group/Drafts/BestPractices-2.0/ED-mobile-bp2-20101202/#bp-devcap
Survey Practice eJournal
Generally has a short publishing time from submission to online posting and
contains many shorter articles highlighting experiments conducted with
mobile devices.
74
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Resources for Mobile Surveys, Take 2
Mobile Marketing Research Association
Has a Knowledge Center where content on mobile market research is
searchable by keywords.
http://www.mmra-global.org/search/all.asp?c=&bst=mode+effects
Mobile Glossaries
uSamp Mobile Glossary Contains detailed definitions for many things
mobile including API, GeoFencing and many other terms relevant to
mobile devices and use.
http://www.usamp.com/learnmore/mobileglossary/?utm_source=RFL+Communcations&
The European Society for Opinion and Market Research
(ESOMAR)
Guidance for Mobile Market Research Report
http://www.esomar.org/uploads/public/knowledge-and-standards/codes-andguidelines/ESOMAR_Guideline-for-conducting-Mobile-Market-Research.pdf
Guidance for Mobile Survey Research Report
http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Codesand-Guidelines_Conducting-survey-research-via-mobile-phone.pdf
75
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Resources for Mobile Surveys, Take 3
Our Mobile Planet Google Site
Contains data from several countries on mobile behaviors by
basic demographics of the user by year (from 2011 onward)
for a host of countries around the world.
http://think.withgoogle.com/mobileplanet/en/
Mobi-Thinking – Website Offered by dotMobi to
educate practitioners about the mobile arena
Special Section dedicated to Global Mobile Statistics
http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats
Flurry Analytics (Service and Blog)
Track user activity in the app with events that are defined by the app
owner. May be useful to collecting paradata for app related surveys.
Also have a blog that provides regular estimates of app usage across a
variety of platforms.
http://blog.flurry.com/ and http://www.flurry.com
76
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
16:59
Section 5:
Recruiting
Methods and
Coverage
77
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
From new devices to new sampling units!
The proliferation of devices has also provided new
types of sampling units and new avenues for
sampling.
Take for example universes defined by owners of a certain app (like
the Yahoo App).
The sampling unit might be the App itself or alternatively the id of the
downloader, stratified by Market Store (Google Play, App Store, etc.)
The type or version of the app (phone or tablet) may also be a
stratification variable
RDD, ABS sampling frames may not provide access to the universe at
adequate levels (low incidence of a given app).
Perhaps selection generated from the app itself,
much like push notifications, is one possible way
to implement selection.
78
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile device Reach and Coverage Biases
If you deploy a solution (such as an app) using only iOS app store,
then you effectively will not be covering a fairly moderate
percentage of smartphone users.
iOS captures 42% of Smartphone Market in US compared to Android which
captures 52% (Nielsen, 2014)
Similarly, if you design an online version of the survey with no
mobile optimization, smartphone users will be present
(unintentional mobile respondents) but those who receive internet
only through mobile devices may be missing
approximately 25% of Mobile Web users or 10% of the U.S. adult population
79
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Android Users versus iPhone Users
Annual HH income
Share by Platform
Share of Users Engaged by Activity
Share of Users
between 18 and 24
80
Source: comScore, 2013: http://bit.ly/QiM43x
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
he state, the higher the
Android mobile page views
more blue the state, the
centage of iOS views. States
0 will be a shade of purple.
Android Owners vs iPhone Owners, Part 2
Distribution among
iPhone Users
(U.S. Adults)
Distribution among
Android Users
(U.S. Adults)
Men
45%
51%
18-34
36%
43%
55+
27%
20%
High School or Less
20%
32%
College Degree (or More)
54%
39%
Non-Hispanic Black
8%
19%
Non-Hispanic White
80%
70%
<30K annual HH income
15%
28%
>75K annual HH income
51%
35%
Demographic Variable
81
Source: Estimates Recomputed from Tables Included in Pew’s
Smartphone Ownership – 2013 Update (Smith, 2013): http://bit.ly/Qnt4Rr
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Android and iDevices and Geography!
Geographic differences may also exist among the iPhone
and Android User base within the U.S.
Study conducted by Mobify in 2013
Using 20,000 partner websites with over 200 million visits through 2012
93% of mobile traffic came from either an iOS or Android mobile devices
(tablets and smartphones)
More blue=higher percentage
of views from iOS
More Red = Higher Percentage
of views from Android
82
Source Mobify (2013): http://www.mobify.com/blog/ios-vs-android-in-2012/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Survey Recruiting and Mobile Devices
There are many tools/strategies that are available for
facilitating survey data collection using mobile devices.
To point selected respondents to your survey you could use a QR
code printed on a postcard or other type written material.
The QR code (short for Quick Response Code) is a matrix
representation of bytes of information associated with your web
address (or location of a survey app in one of the app markets).
GOQR: http://goqr.me/ because “QR Codes created on goQR.me are
completely free of charge (commercial and print usage allowed).”
Requires a QR scanner to be installed however;
Use a simplified/shortened web address (URL) from a service like
bitly (https://bitly.com/)
http://surveypractice.org/index.php/SurveyPractice/article/view/63/pdf
bit.ly/14etvB3
83
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Example of a missed mobile opportunity
I recently received an invitation to complete the JD Power
and Associates Vehicle Reliability and Service Survey.
The main invitation came via postal mail
A survey reminder came via postcard here:
84
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Keeping Recruiting close to the vest
(or at least the mobile device)
An alternative to QR codes and shortened URLs
could be the use of NFC (near field communication,
within 4 inches) tags to transfer/receive data
In areas where internet connectivity is spotty or unreliable,
field workers could collect survey data loaded to their mobile
devices via NFC from central computers and then when
interviewing is complete, transfer of data back to laptop or
central computer could again be made with NFC – all
without requiring any internet connectivity.
Passive NFC tags could be embedded in survey invitation
papers/postcards or on posters or via kiosks and activated
using respondents’ mobile devices.
Similar to QR codes, but requires no additional apps or steps
to “go online” from within the QR code reader app.
85
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Recruitment for Mobile Surveys via Email
Email Invitations and Web Addresses for
Smartphone Surveys should be as simple and
straightforward as possible.
S3DR Tip
Wherever possible, email invitations should be
designed responsively for easy reading and
3
navigation on smartphone and tablets. S DS #1
Place the key information (e.g. survey sponsor, incentive…) at the
beginning of the email subject.
Place the survey link as soon as you can in the email invitation, to avoid
scrolling.
Minimize the number of “special characters” in the survey web address.
If at all possible try to avoid “aliased” web addresses (especially for
mobile web version).
86
S3DR = Smarter Smartphone (and Tablet) Survey Design Recommendation
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
The Got Health Apps Study Survey Invitation iPhone
Invitation Process (Buskirk et al., 2011)
All panelists initially invited on their lap/desktops
via email
iPhone panelists were asked to point their iPhone
web browser to:
http://mobilehealth.slu.edu/1234567890 B
87
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Email Opens by Platform…Mobile on the rise giving
way to unintentional mobile respondents
Source: Return Path (2013)
http://www.returnpath.com/wp-content/uploads/resource/emailmostly-mobile/Return-Path-Email-Mostly-Mobile1.jpg
88
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Email on Smartphones and Tablets
Email and the Mobile Device Reality:
52% of all cell phone owners use email on their phone (Pew, 2013)
78% of smartphone owners check email on their phone. IDC and
Facebook – “Always Connected” (2013)
10% Of iPad users say Mobile is their preferred device to read
email and 9% for sending it. –Perion “iPad Owners Survey” (2013)
Among mobile email users, If an email does not display
correctly, BlueHornet “Consumer Views of Email Marketing” (2012)
69.7% will delete it immediately;
17.7% will view it on a computer
7.6% read it anyway on their device .
Men open 20% more emails on mobile, but WOMEN click 10%
more often on mobile e-mail TailoredMail – “It’s time to wake up and mobilize!”
(2012)
89
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
If you send a respondent and email invitation, where
is it likely read?
According to Pew Research (2013, 2011) 52% of Cell Owners use
email on their phone and 54% of Tablet owners use email daily
The 2013 US Consumer Device Preference
Report by Movable Ink reports that:
At least 35% of Apple/Android Smartphone users spent over 15 seconds or more
viewing an email.
90
Source: https://movableink.com/downloads/us_consumer_device_preference_report_Q42013
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Email Opening by Device Varies by State
91
JOS
30th
State
Percentage of Emails
Opened on
Smartphone
(1) Texas
57.60%
(2) Mississippi
57.27%
(3) Indiana
56.22%
State
Percentage of Emails
Opened via
Desk/Laptops
(1) Maine
48.85%
(2) Vermont
47.23%
(3) Washington
47.19%
Source: https://movableink.com/downloads/us_consumer_device_preference_report_Q42013
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Email Formatting Recommendations
EmailVision provides additional recommendations
pertaining to optimizing email deliverability
For Subject lines and sender options, EmailVision
recommends:
Rather than generic emails such as info@company.com use
relevant email addresses such as:
givefeedback@company.com,
your2cents@company.edu
takesurvey@company.gov
enable the recipient to understand the intent of the email and
potentially reduces auto spam classifications.
http://www.emailvision.com/sites/default/files/email
vision-deliverability-bestpractices-2011-03.pdf
92
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Email subject: Smartphone View
Android Smartphone
resolution 480 by 800
3.7. inches screen
size
93
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Responsive Email Designs for Survey Invitations
Campaign Monitor’s E-Publication “Responsive Email
Designs” offers several considerations for developing mobile
responsive emails including:
Single column layouts (500 – 600 pixels)
Links/Buttons minimum target area of 44 X 44 pixels
Minimum font displayed on iPhones is 13 pixels – smaller font than
this will be upscaled and formatting may change
Call to action (i.e. “start survey button” or survey link) should appear
toward the upper portion of the email to avoid scrolling
For CSS programming use display:none option to limit social sharing
buttons and other extraneous details in the mobile environment.
94
SOURCE: Campaign Monitor Responsive Email E-Book (2013):
http://www.campaignmonitor.com/guides/mobile/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
9
3
Why focus on Responsive Mobile Emails?
Email Invitation on
iPhone
Survey that appears
after tapping “Yes”
95
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile Optimized and Non-Optimized Emails
Mobile Optimized
Not Mobile Optimized
SOURCE: Campaign Monitor Responsive Email E-Book (2013):
http://www.campaignmonitor.com/guides/mobile/design/
96
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Example of Responsive Email from Amazon.Com
Column Dropping
Same email
viewed on
Smartphone
Email sent to Amazon
Customers Viewed on
Desktop/Laptop (responsive)
Source: http://marketingland.com/four-responsive-email-layouts-15858
97
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
From Responsive Email to App-Like Mobile Survey…
98
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
From Email to Mobile Web…
99
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Responsive Email  Quick Survey (Square, 2014)
100
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Example of an Email Invitation – Fresh off the Servers!
S
C
R
O
L
L
101
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Internet Use, Time of Day and Device, US
102
Source: comScore, 2013 http://bit.ly/1c9vneo :
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Internet Use, Time of Day and Device, UK
103
JOS
30th
Source: comScore, 2013 http://bit.ly/1QjphfK
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Resources for Mobile Optimized Emails
Smith (2013) “Responsive Email: Avoid Defections By Improving
Mobile Experiences”
Provides an overview of three common approaches to optimize emails for
reading on mobile devices
Mobile Friendly, Mobile Optimized and Responsive
http://marketingland.com/responsive-email-creating-better-mobile-inbox-experiences-63014
Studabaker (2012) “Four Responsive Email Layouts”
Examples of responsive email invitations from major vendors.
http://marketingland.com/four-responsive-email-layouts-15858
Studabaker (2012) “Choose The Best Responsive Email Layout For
Your Message”
Offers examples of typical layout patterns that are commonly used in
responsive email templates
Shrink Wrap, Column Drop and Layout Shifting
http://marketingland.com/responsive-email-layout-patterns-29378
104
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Source: comScore,
2013
http://bit.ly/1QjphfK
:
Mobile Recruiting Survey Protocols
Once you have optimized the email content and message,
when should the email be delivered?
32% of mobile users often read email on their smartphone during the
weekend; 31% sometimes reads their mobile email on weekends
54% reads email on their smartphone just before they go to bed
sometimes or even more often
49% reads email on their smartphone immediately when they wake
up sometimes or even often.
105
Source: Apsis “The email Barometer, Email on mobile devices” (2013)
http://www.emailmonday.com/mobile-email-usage-statistics
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
16:59
Section 6:
Smartphone
Survey Data
Collection and
Design
Recommendations
106
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Smarter Smartphone Surveys Development/Design
Recommendations (S3DR)
These recommendations come from our experience in
developing surveys for smartphones from the java/html
frameworks directly and are based on extensive review
of the existing literature and practice.
Some of them are based on the current technical and
screen characteristic (i.e. size) of mobile devices –
aspects likely to change over time.
The design suggestions presented here are our
recommendations meant as guidance not guideline.
107
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #1: Paradata
Successful implementation of multimode surveys requires
that the survey software include comprehensive device detection
functionality. Device detection is needed to automatically identify
the type and brand of each respondent’s device, and then display
the survey according to that device’s specifications.
-Cazes, et al. Kinesis Mobile Landscape Report, 2010
http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf
S3DR #1
Plan to collect paradata from mobile sample
including device type, size and browser size.
Browsers matter and can be key to an overall
survey’s functionality on a respondent’s device.
108
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
1
0
7
User agent strings
“A text field in an HTTP request header that contains the
name and version of the Web browser” (PC magazine
definition)
Web resource for locating your User Agent Strings:
http://user-agent-string.info
User agent strings look like this:
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; enUS) AppleWebKit/534.7 (KHTML, like Gecko)
Chrome/7.0.517.41 Safari/534.7
Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_2 like Mac
OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko)
Version/4.0 Mobile/7D11 Safari/528.16
BlackBerry9700/5.0.0.351 Profile/MIDP-2.1
Configuration/CLDC-1.1 VendorID/123
109
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Device Feature Detection –
Moving Beyond User Agent Strings
Sorting out the contents of User Agent Strings Resource:
http://blog.trasatti.it/2010/10/sorting-user-agent-strings-out.html
In addition to phone types and operating systems, features of
devices/browsers can be obtained through various device
description repositories (DDR) maintained by third parties and
could be used in advance to determine which information (i.e.
javascript, HTML markup, etc) to send to the phone to render the
mobile survey page.
Device Atlas
https://deviceatlas.com/device-data/devices (Browse library of information
available on each phone model, for example).
www.deviceatlas.com (User Agent String tool)
WURFL (Wireless Universal Resource FiLe)
http://wurfl.sourceforge.net/
110
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
1
0
9
Using RDRs in action – WURFL Example
http://wurfl.sourceforge.net/
111
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Beyond User Agent Strings  Feature Detection
Feature detection is often deployed using a JavaScript library like
modernizr (www.modernizr.com) and is done through a series of
queries made to the phone.
If you are interested or concerned about tracking mode effects
(especially primacy or recency effects), knowing the type of
device/operating system may not be enough.
Keep in mind that while most smartphones (and tablets) use the
native browser (safari for iDevices and Android Browser for
Android devices) users can and do change these defaults.
Interpretation of the survey scripts can be interpreted differently
across browsers and some functionality will gracefully degrade or
not be offered as intended
number entry presented as text; photo upload not offered
112
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Browser Types on Desktops (April 2014- April 2015)
A closer look at the Variability in Browsers across device types using
desktop internet traffic data collected over the past year (from Stats
Counter)
113
JOS
30th
Source: StatsCounter: GlobalStats (2015) http://gs.statcounter.com/
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Browser Types on Smartphones
(May, 2014 – April, 2015)
114 Source:StatCounter GlobalStats (2015) http://gs.statcounter.com/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Here’s a look at why browsers matter
Ultimately, device detection via user agent strings helps to
deliver overall content to the correct type of device (phone
versus tablet versus computer).
However, the way the device translates the survey
programming (HTML markup etc) is determined by:
The browser in use
The features of the browser
The features of the device
(e.g. hard keyboard versus virtual, front facing camera or not)
115
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Same view – different view!
When a media query is issued for the size of the viewport
for each of these smartphone the result is 320 pixels.
However, each of these phones have extremely different
browsers with different capabilities.
116
Source: http://www.slideshare.net/yiibu/pragmatic-responsive-design
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Self-Demo 1:
Explore Your Device’s UA String and Size
Using the following web-resources,
determine your Smartphone’s user agent
string and screen size.
User Agent String:
http://user-agent-string.info
Screen Size:
http://whatsmy.browsersize.com/
117
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Specific Example: Same Phone…Different Browser
118
http://user-agent-string.info/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Why browsers matter for Mobile Surveys!
Galaxy Note 3
Android (KitKat
4.4.2) Smartphone
Android Browser
(default/native)
Chrome Mobile
Browser (activated)
119
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
HTML5 and the rise of app-like functionality
The advent of HTML5 has given programmers and
researchers a lot of flexibility and versatility in creating
mobile versions of websites with many supported input
types for mobile forms and capabilities:
Geolocation collection
Media capture (voice, photo and video)
Improved and expanded input types
Web storage
For survey researchers, this new development means that
our active mobile browser surveys can be made more and
more app like with standard, predefined enhancements that
have been include in HTML5.
But this is not yet a silver bullet!! Care must be taken!
120
http://html5test.com/index.html
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
HTML 5 functionality across mobile browsers
Mobile HTML 5 Resources are available to determine if certain
functionality of interest is available on a particular mobile browser.
http://mobilehtml5.org/
http://caniuse.com
http://detectmobilebrowsers.com/
121
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Alternate HTML 5 Browser Functionality Tool
Another comprehensive tool for tracking availability of
HTML5 functionality across browser types is
http://caniuse.com
122
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Case Study Example of HTML 5 in Action
Walmart in partnership with Mattel Hot Wheels launched a loyalty
program for shoppers.
Shoppers who purchased Hot Wheels products could receive
points redeemable for various rewards by scanning their receipts
showing the Hot Wheels item purchased.
The loyalty program required NO app, yet enabled mobile photo
capture!
Scanning/photo functionality and uploads to the Walmart program were
managed via an online website optimized for mobile visitors with (modern)
browsers
123
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Hot Wheels Redemption Program on iPhone
SOURCE: Hot Wheels (2013) https://hotwheelsrewards.com/
124
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Practical Example from New Study!
Michaud, Buskirk and Saunders
(2014) designed a study in
partnership with Decipher and
Research Now to compare various
touch and data collection
possibilities across three modes –
computers, tablets and
smartphones.
A photo capture question was
deployed to all devices detected as
either “Tablet” or “Smartphone”
using user agent string
classification.
Kindle Fire Tablets Respondents
running the native Silk browser had
NO photo uploads as expected
from browser limitations
Those with devices without this
functionality had to press continue
twice to continue past this screen.
125
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #2: Mobile Survey Approach
S3DR #2
Select an implementation
approach that:
(a) optimizes presentation of
survey content
(b) is consistent with the type of
survey data desired
(c) is appropriate for the context.
126
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Share of Demographic Audiences by Platform
http://bit.ly/1D2zXhQ
Source:
127 comScore Media Metrix Multi-Platform, U.S., Age 18+, Dec 2014 / Dec 2013
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Focusing on Digital Double and Triple Vision
Percentage of Specific Tablet Brand Owners
who own Smartphone of a given brand
Source: Pew Research, 2012
http://bit.ly/1ttquIX
128 Source2: comScore (2013) http://bit.ly/1msbiaV
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Not all Tablets are Created (and used) Equally
129
JOS
30th
Source: http://www.statista.com/chart/2075/how-different-tablet-users-spend-their-time/
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Daily Social Networking Visits and Sharing by Country
70
61
60
56
52
56 56
53 53
50
41
40
30
20
23
18 16
14
26
19
22
12
10
0
Germany
Sweden
UK
USA
Visiting Social Networking Sites (via smartphone)
Visiting Social Networking Sites (via Computer)
Social Sharing via Smartphone
Social Sharing via Computer
130
Source: Our Mobile Planet, Google, 2013: http://bit.ly/1HSHFwO
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Tablet versus Smartphone Usage by Task
(European Technographics Consumer Technology Online Survey)
131
JOS
30th
Source: Forrester Research, 2012: See: http://tcrn.ch/1SHLGaO
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Sweden Internet Users, TV + 1 Device
Source: Inizio “Multiscreen 2013” See: http://bit.ly/1FJyMSW
132
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #2b: Survey Apps Should be Developed using versions
of the Toolkit that are at least one-version back from current
While IOS devices update frequently, Android device
updates are initiated via Cell Phone Providers and can
often lag behind currently available versions.
S3DR#2b
The functionality of smartphone components such as
camera, scanners etc. will in part be governed by both
device and operating system.
By using the older version of the OS to develop the app,
the functionality will adapt in newer phones and not
excluded from older devices.
Moreover, your survey app will be “found” in Market
searches across operating system versions.
133
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Global Android version distribution since Dec. 2009
Dec. 2010
Feb. 2011
Oct. 2011
June 2012
134
Source: http://en.wikipedia.org/wiki/Android_version_history
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Thank you!
TBuskirk@m-s-g.com
314-695-1378
www.m-s-g.com
@trentbuskirk
135
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Survey Research using
Emerging Technologies:
Considerations for Design,
Data Collection, Sampling
and Recruitment for
Smartphone and Tablet
Based Surveys
JOS Conference June 10 and 11, 2015
Trent D. Buskirk, Ph.D.
136
JOS AAPOR
30th Anniversary
Short Course,
Conference
2014 Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Day
2
Agenda & Goals
Developing Emerging Best Practices: A blend of old and
new
Recommendations for Internet/App based survey solutions (Top
14 EBPs)
Computing – just how do you make these surveys?
From Websites to Mobile Web Surveys (Web design methods for
Mobile)
Overview of “Mobile Versions” for Popular Online Survey
Packages
Goal: Leave with a solid appreciation of:
Potential role of new technologies in data collection;
Different ways you can use smartphones to collect survey data
Some practical “best practices” for deploying mobile surveys
137
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #4: Survey Length
S3DR #4
Smartphone surveys should be designed to
be as short as possible, but not necessarily
abbreviated versions of longer online
surveys.
Dead spots still exist for Smartphones
In both voice and data (i.e. edge, roaming)
Internet speeds vary also by area
Sprint has both 4G and 3G speeds in St Louis, for example.
Smartphone users multitask
Three good reasons to allow for users to return to the survey site
Start over?
Begin where they left off (approximately)
138
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #5: Survey Layout
The overall survey layout should minimize the need
for scrolling (either horizontally or vertically) to the
extent possible.
Survey layout should also minimize the need for
pinching/zooming.
S3DS #1
S3DR #5
The number of questions per screen should
generally be determined in order to minimize
scrolling (implied scrolling)
Be mindful for the device and place key actions in
“good” zones
Some devices don’t support zooming
Answer choice layout may create need for scrolling on some devices.
139
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Putting Responses within Reach, for mobile devices!
Source: http://www.kickerstudio.com/2011/01/activity-zones-for-touchscreen-tablets-and-phones/
140
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Illustrating an ACTIVE Mobile Web Browser Survey:
Peytchev & Hill Study (2010)
Experiment with 92 adults
provided with a Samsung
Blackjack phone
2.2. inches and resolution of 320
by 240 pixels
Several embedded experiments to
test question wording effects,
image effects, scrolling effects and
open ended effects
141
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Horizontal Scrolling Experiment (Peytchev & Hill, 2010)
M
o
r
e
Q
u
e
s
t
i
o
n
s
More
Responses
No Significant Differences Noted
142
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
1
4
1
Example of Survey from Frontier Airlines
(2014) with Horizontal Scrolling Prompts
143
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
The Rotate Screen Prompt has Debuted!
144
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #6: Question Formats
Consider reducing the number of response options
for grid questions or reorganizing grid questions as
single questions per screen.
S3DR #6
Consider organizing response options vertically or
horizontally with scale labels provided per
occurrence (see Peterson, 2013).
If Multiple choice/select all that apply require a long
list of alternatives, consider converting question
into free response with as many single-text fields
as answers expected (i.e. what are three of your
current research interests – list up to three).
145
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Some Grid Examples on Mobile Devices
Example of Grids – reworked for mobile surveys
146
Source: Peterson et al. (2013) http://bit.ly/1i5WYo0
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Got Health Apps Study Screenshots Illustrating
Question Layout and Response Choice Feedback
Source: Buskirk and Andrus (2014)
http://fmx.sagepub.com/content/early/2014/04/08/1525822X14526146.full.pdf?ijkey=jZzKaocZyiG6YNn&keytype=ref
147
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Paging Versus Scrolling Question Presentation
Scrolling question presentation in mobile surveys
presents a series of survey items on a single page
Requires respondents to scroll (vertically) in order to see (and
answer) all survey questions on the page
Paging question presentation in mobile surveys
limits the number of questions per page
Generally one question per page and the respondent
navigates multiple pages in order to complete the survey
Buskirk and Andrus (2014) present a hybrid with two
questions per page to limit (a) the vertical scrolling required
on mobile; (b) the overall number of pages needed for survey
completion and (c) increase comparability for desktop version
148
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #7: Scrolling Versus Paging
(a) Consider an optimal combination of these
methods for presenting questions.
S3DR #7
(b) If you have a series of questions with similar
response options and type then scrolling may
help move a respondent along more quickly
and simply.
(c) If you have multiple skip pattern logic in your
survey, paging may be preferred (at least for
that section).
(d) Be careful that scrolling is suggested or implied
so respondents won’t skip (especially if
Next/Back are at the top)
149
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Closer look at Paging versus Scrolling
Survey Outcomes for Mobile Respondents
Paging
Scrolling
Ease of Completion (Mavletova and Couper, 2014)
Lower Break-off rates
(McGeeney and Marlar, 2013; Mavletova and Couper, 2014)
Shorter Completion Times (Mavletova and Couper, 2014)
Lower Overall Item Nonresponse (Mavletova and Couper, 2014)
Fewer Technical Difficulties when Completing Survey
Greater variety of question types/inputs
Maximizes stored data if internet connection is lost and respondent
returns to the webpage for completion
Easy Facilitation of skip pattern logic
150 Source: http://bit.ly/1nqe6Ib
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
An example of Scrolling in practice
Requires
tapping here;
then entering
value; then
tapping out
then scrolling
and repeating.
151
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Example of a Survey w/ a Heavy Scrolling Section
152
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #8: Question Types
Sliderbars, sum tallies and drag and drop sorters
may not work on all devices.
These question types should be tested extensively
using emulators prior to their adoption in your
mobile survey.
S3DR #8
Text entry fields should be wide enough so that
users can see their entire entries without scrolling
horizontally. Use this type of entry judiciously.
Decisions about types of input/forms or input should
include consideration of number of taps or clicks
required (NTR or NCR) to enter (and register) an
answer.
153
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
A quick note about sliders
Be wary of the default starting positions – could
grossly impact final results
Michaud, Buskirk and Saunders, 2014
Peterson et al., 2013
Can’t distinguish a “3” from missing if default start is midpoint
154
Source: Peterson et al. (2013) http://bit.ly/1i5WYo0
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Using the NTR metrics to help weigh design
choices, in practice.
Michaud, Buskirk and Saunders (2014) included an
experiment to compare radio button and list inputs for
various substantive questions.
Initially respondents that were randomized to lists received
this format for all survey questions including demographic
questions about gender, education, race, etc..
Pilot testing and the NTR metrics changed our plans for the
demographic questions…
155
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Using the NTR metrics to help weigh design
choices, in practice.
Radio Button
Version (all devices)
List/Dropdown iOS
and Stock Android
Selector Android
(some browsers)
156
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Beware of the Pagination/Scrolling Mashup!
Just because it’s optimized for smaller screens doesn’t mean it’s
optimized for respondents!
Presentation of questions only without their response options will save space
on a screen, but makes data collection/completion a mine field!
Example from Major League Baseball!
https://www.fansatbat.mlb.com/PORTAL/default.aspx
The survey question (depicted on an
iPhone 5):
“Are any of these networks included
in your current cable lineup?”
157
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Taking Advantage of New Input Types from HTML5
With the release of HTML 5 there are several new input
types available that might be of interest to survey
researchers and questionnaire designers including:
Number
Tel
Email
Range (a.k.a. sliders)
Time
These new input types when used wisely will likely reduce:
The overall amount of additional scripting that is required to format
the survey questions and response options
Reduce survey page load times
The NTR metric for a given question
158
Source: MobiForge, 2013 http://bit.ly/1aH3vnM
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Numeric, open ended questions using new HTML
input options Number and Tel
How many miles per year do you drive the car that you personally drive
most often. (Enter a number between 0 and 50000)
Question Page
Default Input
159
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Using HTML5
“number”
input type
Trent D. Buskirk, Ph.D. 6/10/15
Making Numeric Input Similar across Devices
By using the HTML5 “tel” input type, a numeric
keypad similar to what is displayed when making
calls is displayed on all devices.
Formats consistent
across devices in
terms of number entry.
Note however, that if
decimals are required,
then iOS keypad does
not offer this capability
directly.
160
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Labels for Open Ended Fields
Labels Top Aligned
Labels Within Fields
161
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Labels for Open Ended Questions Top or Within?
Pros for Labels at Top
Cons for
Labels at
top
Pros for Labels
within
Cons for Labels
within
Minimize completion time
Requires
more space
Saves vertical space
Hint text can be
misconstrued as a
completed item
Results in
longer pages
(requiring
scrolling)
Reduces page length
for pages with
multiple openended questions
Depending on tap
location the hint text
may not disappear
from the respondents
answer
Can save on
horizontal scrolling
required compared
to left aligned labels
Once respondent
begins typing, the full
question or item is
no longer fully visible
Easier coding (no indents/
tables)
Accessibility: label then
field in order
Supports multiple screen
resolutions
More space for the item
(horizontally)
Reduces horizontal scrolling
162
Source1: Wroblewski, 2012 http://bit.ly/1eFrXaG Source 2: Penzo, 2006: http://bit.ly/1iuOFC2
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Example of Label Alignment in practice
From Wroblewski, 2012
http://bit.ly/1eFrXaG
163
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #9: Navigation/Progress
Limited screen landscape requires compromise
and judgment when using logos, progress bars,
disclaimers and help links in mobile surveys.
S3DR #9
Consider using abbreviated progress bars3
S DS #1
Placement of next and back buttons should be
oriented toward the top of the screen or at the
bottom of the screen as long as they are
persistent and visible without scrolling
Disclaimers should be placed on the welcome
page.
164
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Illustrating Back Button and Progress Bar Placement
Next/Back Buttons
Progress Bars
165
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Another Take on Next/Back and Progress Bars
166
Strategic Reflections Quick Serve Restaurant Survey, 2014
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Another Example of Persistent Progress Bars
(Expedia Survey, 2015)
167
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #10: Buttons, Tabs and More Buttons
Response selections, action buttons and navigation
tabs in surveys should be made as large as possible
on mobile surveys.
Touch input differs from Mouse input.
S3DR #10
S3DS #1
Fattest Fingers – Average width of index finger is 11
mm with a range of under 7 for babies to over 19 for
tall athletes.
Make sure to include padding around important
navigation tabs (next/continue/enter) to avoid
accidental taps .
168
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Importance of Button Sizes
Make butons/tabs large enough to not be missed/avoid mistaps
40 pixels (7 mm by 7mm) GOOD
50 pixels (9mm by 9 mm ) BETTER
30 pixels (5 mm by 5 mm) OK
Give enough buffer around/between tabs/buttons
Padding of at least 2mm (10 pixels) between targets
169
JOS
30th
Source: Microsoft Touch Guidance
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Touch Reference: iPhone IOS Requirements
iOS requires app icons to be 57 square pixels for iPhones
iOS requires app icons to be 72 square pixels for iPads
Knowing the resolution of the devices is also important
for rendering. Retina display phones and tablets for
example usually have double the size allowance for icons
(and images).
Check out this icon reference chart by graphic illustrator
Vicki Wenderlich:
http://www.vickiwenderlich.com/2012/09/app-icon-size-reference-chart/
170
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
One accidental tap could be missing data!
From Michaud, Buskirk and Saunders (2014)
We included slider bars as input options for scale questions. The slider bar
button was made large enough to drag across phone and tablet devices.
On smartphones such as iPhone, the “continue” tab was too close to the
upper answer choices, so people who dragged to either 7 or 8 could have
tapped the continue button by accident on the way to a rating of 9 or 10.
In the radio button version, people attempting to tap an 8 9 or 10 could have
tapped continue on the way there and recorded no data.
171
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #11: Flash, Video, Audio, Images
Resolution of graphic images should be reduced in proportion
to expected screen resolutions of mobile devices prior to
transmission to devices (avoid using strictly responsive only
web designs for mobile surveys if these surveys will contain
images. Prefer RESS approach or adaptive RWD instead).
S3DR #11
Flash content should be limited if iPhone Users are part
S3DSof #1
sampling frame. Alternate video playing capabilities are
emerging through HTML5 but not ubiquitous across devices
yet.
Video files should be embedded via link to YouTube or like
“player” Can’t rely on QuickTime, windows media player, etc.
Consider the size of the audio, image and video file sizes as
they have implications for both bandwidth and data
consumption.
172
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
A quick overview of IMAGE Types with focus on
Mobile Device Surveys
Image
Type
Common Use
Notes
JPEG
Typically used for photographic images
A small reduction in quality (80% vs 95%) can reduce
overall image weight by more than ½. 50% quality
images can render quickly and without visible
differences on many mobile devices/websites.
PNG
PNG is superior to GIF. It produces
smaller files and allows more colors. PNG
also supports partial transparency.
However not all browsers support this
function (IE). PNG files are also the only
image type with lossless compression
supported across the web.
PNG files of photographs can be bloated in size
compared to corresponding jpeg images (5 to 7 : 1)
PNG files however can be compressed further and
still deliver the same image without loss of detail.
Try PNGCrush and OptiPNG or Smush.it
GIF
If the image has fewer than 256 different
colors or contains areas of similar color
then GIF will be optimal.
For images with greater numbers of color or detail,
GIF compression may miss some of the colors. But
feature phones may not have a palette that goes
beyond the 256 standard gif palette anyway.
WebP
Newer format developed by Google
Can provide more compression of images of
comparable quality saved as jpeg.
173
https://developers.google.com/speed/webp/docs/we
bp_study
http://www.smushit.com/ysmush.it/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Example of PNG Image Compression
BEFORE:
13.66KB
AFTER:
11.01 KB
http://www.smushit.com/ysmush.it/
174
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Additional Issues with Collected Images
Images captured by mobile devices contain meta data that
includes:
Camera information (aperture, shutter speed, focal length)
Date and Time information
Geo Location
Orientation of the image (portrait or landscape)
This information along with the contents of the photo
(respondent plus others who may not have consented to be
included in the study) have potential privacy concerns.
Consider stripping the collected images of this meta data and
save it in it’s intended orientation prior to image upload.
Not only reduces privacy concerns about location but also reduces
the overall size of the transferred file if done client side.
175
http://cloudinary.com/blog/top_10_mistakes_in_handling_website_images_and_how_to_solve_them
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #12: Recommendation for Mobile Surveys that have
a Multilingual Audience/Various Language Versions
Remember the higher incidence of mobile users
among Hispanics and Asians in the U.S.
When deciding on spacing of answer choices,
resulting font size implications and layout for a
given set of questions, make choices in light of
the languages that will be presented.
S3DR #12
For example, if the Spanish version of the
answer choices are actually longer, then
optimize for these choices first rather than
always optimize for English versions.
176
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Optimizing for English Versions may not be sufficient!
Questions or
their answer
choices (in
English) that
appear to be well
optimized and
presented using
mobile browsers
may not be
optimally
presented in
another language
ENGLISH VERSION
SPANISH VERSION
177
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR #13: If GPS location data are required, user
opt-in/out notifications should be pushed
Apps, unlike mobile web browsers, can enable GPS location
services via the Smartphone’s GPS Antenna directly.
Geolocation data can also be collected via browsers that
support the HTML5 functionality (accuracy varies by cellular
and wi-fi connectivity.
DS #1
In IOS GPS permissions are granted/controlled on aS3“per
app” basis;
S3DR #13
Android devices generally use global controls;
Should your survey require GPS data collection (passive),
the respondent should be given the option to opt out/in via a
push notification at the time of the app launch.
Don’t assume that once an opt in is given once that it
persists across survey data collection points.
178
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Geolocation Data via Mobile Devices
Respondents – where art thou?
GEOLOCATION is the capability to detect and record where a mobile
phone is located using various methods:
IP address, Mac address, RFID, Wi-Fi connection location or GPS
coordinates.
GPS or Global Positioning System is a satellite-based navigation
system. GPS uses triangulation to calculate a user’s exact location.
Smartphones that are GPS-enabled allow an app to determine a phone’s
exact location.
Using HTML5, geolocation can be estimated form mobile deices via the
mobile web browser (wi-fi versus cellular networks)
GEOFENCING Technology that provides the ability to create a virtual
fence around a geographic location in the real world.
Smartphones that are location-enabled detect when someone enters or
exits these fences, (Big: city block or Small: retail store).
179
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Geolocation detection through BROWSERS!
A number of different sources are used to attempt to obtain
the user’s location, and each has their own varying degree of
accuracy.
A desktop browser is likely to use WiFi (accurate to 20m) or IP
Geolocation which is only accurate to the city level
Mobile devices tend to use triangulation techniques such as GPS
(accurate to 10m and only works outside), WiFi and GSM/CDMA cell
IDs (accurate to 1000m).
Accuracy on mobile phones depends on several factors including:
Whether wifi is activated or not (for android phones this has a huge impact)
Is phone in an urban or rural area?
Is phone moving or stationary?
Is a VPN turned on?
http://html5doctor.com/finding-your-position-with-geolocation/
http://www.andygup.net/html5-geolocation-api-%e2%80%93-how-accurate-is-it-really/
180
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Geolocation through Browsers
There are some pros and cons for geolocation through browsers
that we need to be aware of in order to better use these types of
data
Not all browsers support the Geolocation functionality of HTML5 (IE 8
does not, for example).
Different browsers use different Geolocation services (mapping)
Chrome browsers=Google Location Services
Safari browsers on iOS devices=Apple Location Services
Requires and internet connection and explicit user permission (as per
HTML 5 Geolocation Guidelines for use).
May not be useful for persistent Geo-location tracking or for geo-aware
notifications such as used in Geo-fencing
Does not generally have the same drain on batteries as the actual GPS
hardware.
Does not require an app to be installed in order to identify location of
respondent.
181
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Accuracy of Geolocation via Browsers
Accuracy can be impacted by whether or not WiFi is turned
on, for example.
Here is a test link and the results I obtained with and
without WiFi using an iphone 5 (ios6)
http://andygup.net/samples/html5geo/
182
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Smartphone GPS Data Vendors
Locately: (http://www.locately.com/)
Point of interest matching
Trip Analytics
Data processing (No integrated API)
Panel built from anonymous user data
Loc-Aid (http://www.loc-aid.com/)
Integrated API
Allows for custom geo-fencing
183
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
7-Eleven and am/pm stores that one respondent visits
Gym
Home
3 Visits:
on way to
Gym
1 Visit
4 Visits:
on way to
School
School
184
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
S3DR#14: Pre-Testing Mobile Surveys
There is a great deal of functionality that is consistent
across mobile smartphones- but there is enough
inconsistency to require extensive pre-testing of your
mobile survey across multiple platforms.
S3DR #14
Consider using smartphone emulators to pretest the
mobile version of your survey or alternatively a virtual
device laboratory to test mobile versions of your survey.
Test availability of flash content using a resource such
as:
http://www.adobe.com/flashplatform/supported_devices/smartphon
es.html
(or on your mobile phone/emulator)
http://www.adobe.com/software/flash/about/
185
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Smartphone Emulators
Suggested Emulators for Popular Smartphones
iPhone: http://iphonetester.com/
Android Devices: http://www.addictivetips.com/windowstips/download-google-android-emulator/
Blackberry Devices:
http://www.blackberry.com/developers/downloads/simulat
ors/index.shtml
Microsoft Devices:
http://www.microsoft.com/download/en/details.aspx?displa
ylang=en&id=25191
186
Source: http://sixrevisions.com/webdevelopment/mobile-web-design-best-practices/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Mobile Emulator Software Example:
Keynote’s DeviceAnywhere
Many types of
devices centrally
located in one
source (host of
tablets and
smartphones)
http://www.keynotedeviceanywhere.com/da-free-product-overview.php
187
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
16:59
Section 7:
Web Design
Approaches
to Mobile
Surveys (DIY)
188
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Active Mobile Browser Survey Development
A multi-screen strategy is a must to succeed with today’s
constantly-connected consumers. But one size doesn’t fit all
when it comes to creating the perfect configuration. Many factors
come into play, and you know your own business needs best.
– Google 2013 http://bit.ly/1i9e3P1
and SURVEY
Three main approaches to developing an active mobile
browser survey:
Responsive Web Design (RWD) for Survey Pages
Mobile-First RWD
Adaptive Designs
Responsive Design with Server Side Components (RESS)
Separate Mobile Web site developed for Surveys
189
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Responsive and Mobile Responsive Web Design
Responsive web design (RWD) is a method for web design that
uses fluid grids, images and media queries to change the displayed
layout based on the size of the device (view port size).
Media queries govern when and how the fluid layouts appear on given
devices and is managed by the respondent’s device
Single HTML source code delivered to all devices (one size fits all)
Mobile-first Responsive Web design is a method that begins with
the mobile version first and pushes the web page to respond to
larger devices via progressive enhancement.
Additional layout attributes and survey functionality can be added
progressively, in real time for respondents accessing website using larger
devices.
References
Frost (2011) http://bit.ly/1gOcOPK
Passani (2013) http://bit.ly/1i9G8Wg
Wroblewski (2011) http://bit.ly/1gOcS2d
190
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
RWD and Mobile First RWDs for Surveys
RWD for Surveys
would be a strategy
that takes an online
(computer survey)
and thinks about how
to render it down to
tablet and then down
to smartphones.
Mobile First RWD
would take a mobile
survey and
progressively push it
to larger devices with
potentially different
functionality.
191
Source: http://web3canvas.com/what-is-mobile-first-responsive-design/
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Responsive Web Design Example: www.SweetHatClub.org
192
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Responsive Web Design Example: www.SweetHatClub.org
193
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Adaptive Designs
There are various versions of Adaptive web designs but the
approach that has a lot of traction recently is Responsive Web
Design with Server Side Components (RESS)
RESS is sometimes referred to as a hybrid approach between Serverside implementation (rendering unto Caesar what is Caesar’s) and
RWD.
RESS uses device detection on the server side to send device specific
survey web page content/scripting
The device specific code can include components that allow it to
progressively enhance the survey page depending on browser
features it detects on the respondent’s device.
References:
S. Rieger (2011) http://slidesha.re/1k1HLUH
Pratap (2013) http://bit.ly/IK6Q8A
Ronan (2011) http://bit.ly/1tuHNcJ B. Rieger (2011) http://slidesha.re/1pnzBeA
194
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Images
used in
Survey
Computer
Scripting and
HTML Markup
(all devices)
RESPONSIVE WEB
DEISGN – 10000
Feet Overview
Survey
Questions
www.takesurvey.com
195
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
What Kind of
Device is it? User
Agent String
Says:
Images
for
PHONES
Computer
Scripting and
HTML Markup
(PHONES)
iPhone
Survey
Questions
www.takesurvey.com
RESPONSIVE WEB DEISGN
with Server Side Components
10000 Feet Overview
196
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Giving your Online Survey Site a Mobile Sibling
Separate mobile version of the site is the ultimate adaptive
strategy
Special scripts are created to program survey functionality specifically
for the mobile devices
The survey website will generally be a stand-alone website and could
potentially require a different web address.
If you know a priori that you are recruiting mobile respondents, then a
unique mobile link and QR code could be provided.
If you are interested in field work or in-person data collection using
mobile devices and you don’t want to use an app, then a separate
mobile web survey optimized for mobile devices could be a cost
effective and viable solution.
Frost (2012) offers a great comparison or responsive and separate
versions of presidential websites from the 2012 election.
197
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
198
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Data COLLECTION!
199
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
How to choose a survey design approach?
Responsive Web Design Pros
All HTML scripting required for survey deployment across any device is
contained in a single packet and is delivered to every device (OSFA)
Survey content will be rendered seamlessly across a wide array of devices
(even ones that aren’t invented yet or that you didn’t plan for)
Single Survey URL serves all devices/sizes/types
Survey content will automatically adjust to Horizontal or Vertical orientations
Responsive Web Design Cons
All programming code required to render surveys on any device is downloaded
to the respondent’s device for processing (unnecessarily large data
transmission size and potentially longer rendering times)
Images are generally much larger in size than they need to be for mobile
devices (again, increased data transferred to phone/tablet than required)
Typically does not take full advantage of mobile device functionality
Survey layout cannot be controlled as easily and in many cases pages will be
longer and require more scrolling.
200
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Source: http://bit.ly/QBI4Lv
Trent D. Buskirk, Ph.D. 6/10/15
How to choose a survey design approach?
Mobile First Responsive Web Design Pros
Because programming and layout focus on mobile, images and therefore entire
HTML mark-up will be smaller compared to RWD designs
Mobile functionality can be built in to the survey coding from the ground up
Total number of HTTP requests between the mobile devices and the survey
host will be minimized in order to render the same content – thus reducing
latency and loading times.
Has been used successfully by Survey Researchers to collect data from
both smartphone and computer respondents
Bruijne and Wijnant (2013) http://bit.ly/1lxs0UX
Mobile First Responsive Web Design Cons
Requires re-engineering online survey website (computer version) from the
beginning with mobile surveys as the basis **
Survey layout cannot be controlled as easily and in many cases pages will be
longer and require more scrolling.
Source: http://bit.ly/1nl50MT
201
JOS
30th
Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
How to choose a survey design approach?
RESS Approach Pros
Because content can be packaged for devices, http requests and overall
sizes of markup tends to be smaller (decrease load time and data transferred)
Can include components of progressive enhancement to build out features
that are detected on respondents device
Single Survey URL serves all devices/sizes/types
Survey content will automatically adjust to Horizontal or Vertical orientations
RESS Approach Cons
In order to send appropriate versions of survey scripts, a device detection
data base/repository is required and would require updating
To maximize information sent from the server and to minimize subsequent
http requests for rendering survey cookies can be used
May require informed consent to place cookies (EU Cookie Directive)
Server side computing resources and specific programming is required
202
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
How to decide the approach used here?
203
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Resources for Assessing Size and Programming
Features of your Survey Webpages
YSLOW – a downloadable/executable program from
Yahoo to provide a grade for your website in terms of
Web Performance Best practices
Includes Image Assessment, Overall size assessment and more
https://developer.yahoo.com/yslow/
PageSpeed – executable online program from Google to
assess page performance and user experience criteria
based on Web performance best practices
Online version includes Image Assessment, overall size, speed
and user experience criteria for both MOBILE and ONLINE with
suggestions for improvements
http://developers.google.com/speed/pagespeed/insights/
204
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
A Worldwide Wonderland for the World Wide Web
(Mobile included!)
Responsive Web Design Example: www.disney.com
mini
205
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Let’s Look Under the “Hood” of the Disney Site.
An example of a bloated design includes
background images (skin) along with many
foreground images. www.disney.com
This took about 10 seconds to load using my
iphone 5 over the LTE network of Verizon
206
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
http://developer.yahoo.com/yslow/
Trent D. Buskirk, Ph.D. 6/10/15
From Google PageSpeed: Disney.com
207
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
The other side of RWD…LAPTOP Pinch and Zoom!
Responsive web design can be great at adapting content to smaller
viewports (or devices) but one of the drawbacks for surveys is the
bringing back the pinch and zoom issue to lap/desktops as
illustrated here – a laptop and iphone view of a question from a
sample survey from CheckMarket. https://www.checkmarket.com/mobile-surveys/
Laptop
View
iPhone
View
208
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Focus on a survey design that is technically sound
On mobile websites there is a smaller margin of error for:
Image rendering, browser requests, survey page size, waiting times.
77% of cell internet users say they experience slow download speeds
that prevent things from loading as quickly as they would like. Of those
cell internet users, 46% face slow download speeds weekly or more
frequently. (Pew, 2012)
97% of time on a single web page is spent on the front end – meaning
the loading time of the page (Everts, 2011)
Why do we need to focus on techniques that can optimize
survey websites?
Overall Respondent burden
Lag/Latency Waiting time, Time on Tasks, data consumption/exchange, effort
Break-off, partial completion, full completion
Keep in mind that a web survey is a series of consecutive and linked
web pages (each serving up survey tasks to complete).
209
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Practical Tips for Reducing Respondent Device Burden
There are several practical guidelines to think about for
reducing overall respondent burden
Image Resizing
Deliver smaller images to devices with smaller resolutions
Avoids having to transmit entire image to device and then resize at
the device level
Device Specific Scripting Only
If displaying survey contents to a smartphone, don’t send
deskop/tablet specifications, additions or modifications, for example
Connectivity Detection
Not only for cellular networks, but also for weak wi-fi networks (i.e.
laptops accessing weak signal @ airports or conference hotels)
210
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
16:59
Section 8:
Making it
Work!
Software
Considerations
211
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Development Frameworks and Resources
JQuery Mobile: (http://jquerymobile.com/) mobile framework
that allows you to design a single web site or app that will work
on all popular smartphone, tablet, and desktop platforms.
Pelletier’s E-book: Mobile App Manual: The Blueprint? Provides an
introduction to developing apps using J-Query Mobile and the PhoneGap
build environment. User friendly overview of the approach rather than
emphasis on the computer code required. http://mobileappmanual.com/
Sencha Touch: (www.sencha.com) A comprehensive
development environment for mobile apps that uses the HTML5
platform
Pure from Yahoo: http://purecss.io/
This collection of CSS modules enables the users to create responsively designed
websites (for a myriad of devices) using an entire collection of CSS pages that
have been designed to be as small as possible to avoid bloating and increased
loading/unpacking times on mobile devices.
212
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Software Services Solutions for Mobile Survey
Design/Development/Deployment
WebFirst’s PhiCollect
(Paid)
www.webfirst.com/phicolect
Open source data collection platform – flexible, scalable, cloud-based
Web, Android, iOS and SMS options (Tablet and Phone)
Location, multimedia, sms data collection
Has been deployed for mHealth and education applications
worldwide.
Qualtrics (Paid)
http://www.qualtrics.com/research-suite/mobile-surveys/
Provides feedback on survey questions and answers and formats
with recommendations for better mobile optimization.
Uses responsive web design with server side components
allows customization of content delivery based on device type detection:
ipad, android phone, palm, windows mobile, etc.
213
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Example from a Basic Qualtrics Survey
214
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Software Services for Mobile Surveys, Cont.
Columbia University Modi Research Group’s formhub (Free)
http://www.formhub.org/
Python/Django Open source, free development platform for mobile
data collection
Relies on ODK Collect app or mobile browser (modern browser that
supports HTML caching is required)
Surveys are designed and managed using Microsoft Excel
XLSForms spreadsheets
One tab is questions; the other is answer options
Surveys are presented more like data completion forms (scrolling)
and multiple completes can be managed from a single device (Tablet
and Mobile Phone Assisted Interviewing)
Formhub Demo Survey Available at
http://bit.ly/1nvbXbj
215
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
The Actual Survey in FormXLS
216
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Software Services for Mobile Surveys, Cont.
CheckMarket Surveys (Paid)
https://www.checkmarket.com/
Company headquartered in Belgium and offers responsivelike design for mobile surveys that has some app-like
functionality.
Their mobile solution has implemented several of the emerging best
practices and provides some flexibility to the survey designer (controlling
questions per page, size of windows for open ended questions)
New data collection techniques for media are available as well as a ranking
question type that is implemented smoothly in the mobile environment
Example Demo Survey is available here:
http://bit.ly/1K3hzqu
217
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
App-Based Survey Solutions
Confirmit’s SODA (Survey on Demand Application) App. (Paid)
Probably the most widely known and used App-based survey
solution among the survey research community.
Version 3.0 was released in January, 2014
Continuous GPS tracking, Push Notifications, Multiple field editing
enhancements
http://bit.ly/1lllwfQ
Conduit Mobile (Paid)
Offers app development in a hosted environment.
Can build your own app and can include data collection forms and
formats using their web-based app-building platform
http://mobile.conduit.com/
218
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
The Future of Survey Research is Coming: Fast!
The percentage of unintentional mobile respondents has
reached arguably non-negligible levels and will likely increase
as email and internet activity on mobile devices increases.
There is still a lot to learn about usability and design
considerations for surveys optimized for Smartphones.
Even if you are using a survey platform that can handle mobile
devices you will still have to make many decisions in terms of
questionnaire design.
Be mindful of variability across platforms- iPhone release may
not be sufficient for all types of Smartphones associated
members of your sampling frame.
Here’s Hoping that All our Surveys Continue to Get Smarter…
219
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
Final Comment!
I leave you with this final word from Sir Richard
Branson to encourage all of you to continue to:
invest in experimentation in the mobile survey arena and
share your results
so that we can continue to develop and use best
practices for these very cool (and smart) devices!
220
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
2
1
9
Thank you!
TBuskirk@m-s-g.com
314-695-1378
www.m-s-g.com
@trentbuskirk
221
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
References
AAPOR Cell Phone Task Force Report, 2010 available at:
http://www.aapor.org/Cell_Phone_Task_Force_Report.htm , accessed on October 28, 2011
Attfield, Simon, Gabriella Kazai, Mounia Lalmas, and Benjamin Piwowarski. 2011. “Towards a
Science of User Engagement (Position Paper).” Presented at the WSDM Workshop on User
Modelling for Web Applications, Hong Kong, China.
Bailey, Justin and Lorelle Vanno. 2011. “Market Research Association Current Mobile Research
Trends.” Webinar for the Florida Chapter of Marketing Research Association.
Bosnjak, Michael, Gottfried Metzger, and Lorenz Gräf. 2010. “Understanding the Willingness to
Participate in Mobile Surveys: Exploring the Role of Utilitarian, Affective, Hedonic, Social, SelfExpressive, and Trust-Related Factors,” Social Science Computer Review 28: 350-370.
Brick, J. M., P. D. Brick, S. Dipko, S. Presser, C. Tucker, and Y. Yuan. 2007. Cell phone survey
feasibility in the U.S.: Sampling and calling cell numbers versus landline numbers. Public
Opinion Quarterly 71 (1): 23–39.
Buskirk, T.D., Callegaro, M. and Rao, K. (2010) ‘N the Network'? Using Internet Resources for Predicting
Cell Phone Number Status. Social Science Computer Review: Special Issue on Mobile Surveys, Vol. 28,
No. 3, 271-286.
Buskirk, T.D. and C. Andrus. (2012a). Smart surveys for smartphone: exploring various approaches for
conducting online mobile surveys via smartphones. Survey Practice. Available at:
http://surveypractice.wordpress.com/2012/02/21/smart-surveys-for-smart-phones/ .
Buskirk, T. D. and Callegaro, M. “Cell Phone Sampling,” in Encyclopedia of Survey Research
Methods, Lavrakas, P.J. (editor), Sage, Los Angeles, © 2008, pp. 84-89.
Buskirk, T.D., Gaynor, M., Andrus, C. and Gorrell, C. (2011) “An App a Day Could Keep The
Doctor Away: Comparing Mode Effects for a iPhone Survey related to Health App Use.” American
Association of Public Opinion Research, Phoenix, AZ.
222
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
References (con’t)
Buskirk, T.D. and Andrus, C. (2014) “Making Mobile Surveys Smarter: Results from a randomized
experiment comparing online smartphone and desktop/laptop computers,” Field Methods,
forthcoming. See http://fmx.sagepub.com/content/early/recent
Callegaro, Mario. 2010. “Do You Know Which Device Your Respondent Has Used to Take Your Online
Survey?” Survey Practice, December: www.surveypractice.org.
Callegaro, M. and Macer, T. (2011) “Designing Surveys for Mobile Devices: Pocket-sized Surveys that
Yield Powerful Results,“ Shortcourse Presented at the 66th Annual Meeting of the Americaon
Association of Public Opinion Research, Phoenix, AZ.
Cazes, J., Townsend, L., Rios, H., & Ehler-James, J. (2010). The mobile survey landscape - Today and
Tomorrow. Impacts of mobile devices usage on current and future market research practices.
Retrieved from http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf
comScore (2012) 2012 Mobile Future in Focus. Retrieved from
http://www.comscore.com/Press_Events/Presentations_Whitepapers/2012/2012_Mobile_Future_in_Fo
cus
Couper, M. P. (2010). Visual design in online surveys: Learnings for the mobile world. Presented at
the Mobile Research Conference 2010, London. Retrieved from
http://www.mobileresearchconference.com/uploads/files/MRC2010_Couper_Keynote.pdf
Cunningham, JA et al. (2013) “Use of mobile devices to answer online surveys: implications for
research,” BMC Research Notes. Vol 6:258 available online: http://www.biomedcentral.com/17560500/6/258
Duan, Shu, Justin Bailey, and Michael Link. 2011. “Data Collection Method Innovation: A
Crowdsourcing Application to Collect Factual Information in Africa”. Paper presented at the Midwest
Association for Public Opinion Research Conference, Chicago, IL.
223
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
References (con’t)
Fuchs, Mark, 2008. “Mobile Web Surveys: A Preliminary Discussion of Methodological Implications”. In F.
Conrad & M. Schober (Eds.), Envisioning the Survey Interview of the Future (pp. 77-94). Hoboken, NJ: Wiley.
Graham, P, and C. Cobb (2013). “Comparison of Instantaneous Mobile Time Use Data Collection Methods to
Traditional Time Diary Methods.” Paper presented at the Annual Conference of the American Association for
Public Opinion Research, Boston, MA.
Gregoski, Mathew, Martina Mueller, Alexey Vertegel, et al. (2012). “Development and Validation of a
Smartphone Heart Rate Acquisition Application for Health Promotion and Wellness Telehealth Applications.”
International Journal of Telemedicine and Applications, doi:10.1155/2012/696324.
http://www.hindawi.com/journals/ijta/2012/696324/
Kennedy, C. (2007), Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys,
Public Opinion Quarterly, Vol 71 / Num 5: 750-771.
Koch, P-P. (2010) Smartphone Browser Landscape, A List Apart Article No. 320, available at
http://www.alistapart.com/articles/smartphone-browser-landscape/ (accessed on 10/30/2011)
Kumar, A. (2012) “Pilot study tests wearable sensors and smartphone-based data collection for diabetes,”
iMedicalApps, accessed at http://www.imedicalapps.com/2012/11/pilot-study-wearable-sensors-smartphonediabetes/
Kuntsche, Emmanuel and Benjamin Robert. 2009. “Short Message Service (SMS) Technology in Alcohol
Research – A Feasability Study.” Alcohol &Alcoholism 44: 423-428.
Lai, Jennie W., Lorelle Vanno, Michael Link, Jennie Pearson, Hala Makowska, Karen Benezra, Mark Green.
2010. “Life360: Usability of Mobile Devices for Time Use Surveys” Survey Practice, February:
www.surveypractice.org
224
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
2
2
4
References (con’t)
Lai, Jennie, Michael Link, and Lorelle Vanno. 2011. “Like? The Effect of Social Networking in the Digital Age
of Mobile Application Research.” Paper presented at the Midwest Association for Public Opinion Research
Conference, Chicago, IL.
Maxl, E., Döring, N., & Wallisch, A. (Eds.). (2009). Mobile market research. Köln: Herbert Von Halem Verlag.
Nitsche, P, et al. (2012) “A strategy on how to utilize smartphones for automatically reconstructing trips in
travel surveys,” Procedia-Social and Behavioral Sciences, Vol. 48, pp. 1033-1046.
Olson, K, and J. Wagner (2013). “A Field Experiment Using GPS devices to measure Interviewer Travel
Behavior.” Paper presented at the annual conference of the American Association for Public Opinion
Research, Boston (May).
Paudela, D., M. Ahmedb, A. Pradhanc, R. Lal Dangold (2013) “Successful use of tablet personal computers
and wireless technologies for the 2011 Nepal Demographic and Health Survey,” Glob Health Sci Pract August
12, 2013 vol. 1 no. 2 p. 277-284; http://www.ghspjournal.org/content/1/2/277.full.pdf+html
Perkins,J, R. Granger, and E. Saleska. 2009. “Data Security Considerations When Using Social Networking
Wesites for Locating and Contacting Sample Members.” Presented at International Field Directors &
Technology Conference.
Peytchev, Andy and Craig A. Hill. 2010. “Experiments in Mobile Web Survey Design: Similarities to Other
Modes and Unique Considerations,” Social Science Computer Review 28: 319-335.
Pew Research Center. 2011. “How People Use Tablets and What It Means for the Future of News.” Report.
Available at: www.pewresearch.org/pubs/2119/tablet-news
Scagnelli, Jeff, Justin Bailey, Michael Link, Hala Moakowska, and Karen Benezra (2012). “On the Run: In the
Moment Smartphone Data Collection.” Paper presented at the annual conference of the American Association
for Public Opinion Research (Orlando, FL).
225
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15
References (con’t)
Schober, Michael, Frederick Conrad, Christopher Antoun, Alison Bowers, Andrew Hupp, Huiying Yan (2013).
“Conversational Interaction and Survey Data Quality in SMS Text Interviews.” Paper presented at the Annual
Conference of the American Association for Public Opinion Research, Boston, MA.
Smith, Aaron. 2011. “Smartphone Adoption and Usage.” Report available at:
www.pewinternet.org/reports/2011/Samartphones.aspx
Steeh, C., Buskirk, T. D., & Callegaro, M. (2007). Using text messages in U.S. mobile phone surveys. Field
Methods, 19, 59–75.
van Heerden, A. Shane A. Norris, Stephen M. Tollman, and Linda M. Richter, “Collecting Health Research
Data: Comparing Mobile Phone-Assisted Personal Interviewing to Paper-and-pen Data Collection,” in press,
Field Methods, http://fmx.sagepub.com/content/early/recent
Vanno, Lorelle, Jennie Lai, and Michael Link. 2011. “Want to Collect Behavioral Data in an Engaging Way?
There’s an App for That!” Paper presented at the Southern Association for Public Opinion Research
Conference, Raleigh, NC.
Zahariev, M., Ferneyhough, C., Ryan, C., & Bishop, S. (2009). Best practices in mobile research. Presented at
the ESOMAR Online Research, Chicago, IL: ESOMAR.
Zickuhr, K. (2011). Generations and their gadgets. Pew Internet & the American Life Project. Retrieved from
http://pewinternet.org/Reports/2011/Generations-and-gadgets.aspx
Zuwallack, R. (2009) Piloting Data Collection via Cell Phones: Results, Experiences, and Lessons Learned,
Field Methods, Vol. 21, No. 4, pp. 388–406
Zichermann and Cunningham (2011) Gamification by Design: Implementing Game Mechanics in Web and
Mobile Apps available at: http://www.amazon.com/Gamification-Design-Implementing-MechanicsMobile/dp/1449397670
Andbergen, P. A. (2012). Comparison of WiFi positioning on two mobile devices. Journal Of Location Based
Services, 6(1), 35-50. doi:10.1080/17489725.2011.630038
226
JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies
Trent D. Buskirk, Ph.D. 6/10/15