GIST QUALTRICS: USING THE
UNT ONLINE SURVEY SYSTEM
Dr. Russell Ruffu, Office of Institutional Research and Effectiveness
TODAY’S AGENDA





Electronic surveys at UNT
Qualtrics at UNT: Brief History
Resources for Problem-Solving
Ethical Use of Data
Qualtrics Demo
Gist Qualtrics: Using the UNT online survey system
ELECTRONIC SURVEYS AT UNT







2000: IRE purchased Inquisite (not open across campus) –
hard to learn and very complex
2005: Division of Student Affairs purchased Survey
Monkey (monthly subscription)
2007: DSA moved to annual Survey Monkey subscription
2009: DSA purchased second, private Survey Monkey
account to lock down sensitive data (limited access)
2010: Colleges of Education and Business purchased
Qualtrics (limited license)
2010/2011: IRE approached campus users to purchase
university-wide license for Qualtrics
April 2011: Qualtrics added to MyUNT
Gist Qualtrics: Using the UNT online survey system
STEPS FOR PROBLEM-SOLVING



Start off at Qualtrics University
www.qualtrics.com/university
www.qualtrics.com/university/learn-qualtrics/
Access the Qualtrics self-help module at
http://institutionalresearch.unt.edu/sites/default/files/Qua
ltrics%20101Training%20Series%2010_1_12.pdf
Contact Institutional Research and Effectiveness
Qualtrics Coordinator
Dr. Russell Ruffu
russell.ruff@unt.edu
x 7894
Gist Qualtrics: Using the UNT online survey system
HELP FOR SPECIFIC TYPES OF QUESTIONS/ITEMS: GO TO
HTTP://WWW.QUALTRICS.COM/UNIVERSITY/RESEARCHSUITE/BASICBUILDING/BASIC-SURVEY-OPTIONS-SETTINGS/SURVEY-EXPERIENCE
Gist Qualtrics: Using the UNT online survey system
ETHICAL USE OF DATA






All data collected should be treated as security-sensitive
You should only access data for which you have a
legitimate work purpose
Use care when sharing data with others, whether formally
or casually
Be aware of collaborating with external parties, you may
unwittingly be compromising FERPA
National Institutes of Health training
(phrp.nihtraining.com/users/login.php) – useful even if not
IRB study
Avoid downloading individual response data to a laptop or
portable file/device
Gist Qualtrics: Using the UNT online survey system
HOW TO USE THIS GUIDE:
The following slides are not designed to show you
every detail of Qualtrics – just the most
frequently used.
 This presentation has been organized to mirror
the actual process of creating a survey, and
distributing the instrument
 Where appropriate, screen shots and notes in the
margin are designed to help first-time users.

Gist Qualtrics: Using the UNT online survey system
QUALTRICS TRAINING SERIES
PART I
Survey Creation
OPENING PAGE: USE MYUNTFACULTY  QUALTRICS
Click on Create
Survey tab or
the Create
Survey icon
Page shows all of
your surveys as
well as surveys in
collaboration
(separated by the
gray divider bars).
Gist Qualtrics: Using the UNT online survey system
CREATE SURVEY: QUICK SURVEY BUILDER
Click Quick Survey
Builder.
Create from Copy
provides a feature
that copies a
survey from your
Qualtrics file.
The Survey
Library links to
templates that you
have created or
borrowed that
exist in your
Qualtrics account,
or are available
from the Qualtrics
library.
Gist Qualtrics: Using the UNT online survey system
ENTER YOUR SURVEY’S NAME AND IDENTIFY THE FOLDER IT WILL BE KEPT.
THEN CLICK “CREATE SURVEY.”
Gist Qualtrics: Using the UNT online survey system
BLOCK AND BLOCK OPTIONS
A block can contain specific
categories of questions which
can be manipulated for
navigation and analyses
purposes. For example,
blocks might be labeled Track
1 and Track 2 for
respondents who ‘branch’ off
within the survey; or Blocks
might be labeled as
Demographics, Activities,
and Perceptions to segregate
the type of information being
sought.
(If you simply want one
question to appear at a
time, just add a Page
Break.)
Gist Qualtrics: Using the UNT online survey system
CREATE A NEW ITEM: CLICK CREATE NEW ITEM. WRITE THE QUESTION TEXT. BOTTOM RIGHT
CORNER, CLICK “ADD BLOCK.” WRITE SECOND QUESTION. WHAT IF YOU DECIDE YOU WANT BOTH
QUESTIONS DISPLAYED IN SAME BLOCK? THEN REMOVE THE LAST BLOCK. BUT…
Block deletion is a little
funky
BEST:
Plan your blocks/question groups
before you start developing the
survey.
FUNKY:
Check box for each question below
the block marker that you want to
removemove all questions up by
clicking up-arrow at far left of
questions. Remove Block option
now appears in bottom right corner
of last block.
FUNKIER:
Block OptionsDelete Block,
questions go to trash at bottom of
surveyclick on question in trash,
then click RestoreMove question to
desired location.
Gist Qualtrics: Using the UNT online survey system
MOST COMMONLY USED QUESTION TYPES
o Descriptive Text – allows you to communicate with respondent
(no active data collection)
o Graphic – allows you to upload your logo or other image
o Multiple Choice – allows respondent to pick from a list (either one
item or multiple items)
o Matrix Table – Series of statements being measured on a common
scale (e.g. strongly disagree, disagree, neutral, agree, strongly
agree)
o Text Entry – allows for write in (qualitative) responses in either
single lines, short essay or long essay formats
o Slider – Graphically interesting scale item
o Rank Order – Users can re-arrange answers in order of
importance
o Side by Side – Lets users choose answers in a forced choice model
Gist Qualtrics: Using the UNT online survey system
ITEM TYPES
Let’s start with
Text/Graphic items
Gist Qualtrics: Using the UNT online survey system
TEXT/GRAPHIC ITEM: A BIT ABOUT BUTTONS & GRAPHICS
FYI, code is available
from the Qualtric site
to serve many
functions. This is a
sample.
Button functions, like Next and Back, are automatically embedded. The Back button is inserted/selected by
going to the Survey Options icon at the top of the page.
Gist Qualtrics: Using the UNT online survey system
SURVEY OPTIONS
Items selected are inserted
automatically.
Gist Qualtrics: Using the UNT online survey system
Labels can be added to the Next and Back buttons. Go to Look and Feel, locate Next Button
Text and type in your label.
Gist Qualtrics: Using the UNT online survey system
DISPLAY LOGIC: IF ANSWER ITEM SELECTED,
If Question Q5 “In what type of h… do you reside”
tent
is Selected
(then display this question)
Force Response is activated
Gist Qualtrics: Using the UNT online survey system
THEN DISPLAY ANOTHER QUESTION
SKIP LOGIC: IF CHOICE
SELECTED, THEN
Skip Logic must be applied for
each answer selection. That is,
there is no condition for applying
“all” to all of the answer choices.
Gist Qualtrics: Using the UNT online survey system
SKIP TO QUESTION X OR END OF SURVEY
BRANCHING: A QUICK LOOK SO YOU KNOW IT EXISTS
Branching will send participants
who choose certain specified
answers down a different road to
answer a set of questions that
others will not see.
Choose Survey Flow from the tool
bar, then click Add Below from the
options provided on the right in the
blocks. Choose Branch from the
yellow menu. Define the conditions.
Notice that you can select
And…And, or you can select Or, Or,
or a combination of these.
Drag the defined conditions to “Add
a New Element Here.
Save Flow
Gist Qualtrics: Using the UNT online survey system
ELEMENTS IN BRANCHING OPERATION
Block= new section of questions
Branch=new road based on previous answer(s)
Embedded Data=information about respondents long before you survey them
Randomizer=Changes order of response choices
Web Service=bring additional information from a website into your survey
Authenticator=The Authenticator makes it possible to have respondents “log in” to a survey, along with
other features to authenticate the respondent.
Conjoint=A conjoint study usually involves showing respondents a set of features and asking them to
indicate how much they like or prefer the different attributes of that feature.
End of Survey=takes participant to end of survey
Gist Qualtrics: Using the UNT online survey system
QUALTRICS TRAINING SERIES
PART II
Survey Distribution
DISTRIBUTE SURVEY SUB-TABS
There are a couple of ways to distribute your survey. The first is called Survey Link. The
other one we will focus on is the use of Panels, which seems to be the most popular here at
UNT.
FIRST – you have to activate the survey using the link above!
Survey Link: According to Qualtrics University,
“It’s the easiest way to distribute a survey: a simple, anonymous link that can be taken by
anyone. You can:
•Send it out with your personal email.
•Post it on a web page.
•Use social media to distribute it.”
Gist Qualtrics: Using the UNT online survey system
http://www.qualtrics.com/university/researchsuite/distributing/basic-distribution/anonymous-survey-link
Gist Qualtrics: Using the UNT online survey system
The Survey Link
Highlight and Copy it.
Go to the Email Survey subtab to an email.
Paste the link here.
Gist Qualtrics: Using the UNT online survey system
MAILING LISTS/PANELS
Gist Qualtrics: Using the UNT online survey system
UPLOAD
A MAILING LIST FROM A CSV
EXCEL SHEET
There is a way to upload large (and small) sets of email recipients. It does require, however, that the
email list be in a csv Excel format that includes the first 3 column variables EXACTLY as follows:
FirstName
LastName
PrimaryEmail (the columns may be in any order)
From there, go to PanelsCreate New Panel click “Create”Import From a Fileselect fileimport
The FirstName LastName are NOT mandatory, but are commonly included in spreadsheets with
For automatic reminders for those who have not completed the survey, go to DistributeEmail
email addresses. They are not necessary for uploading survey mailing addresses.
Historyon tool bar, select ActionsSend Reminder or Thank You (thank you message can be
customized after this item is selected).
1.
2.
3.
4.
Gist Qualtrics: Using the UNT online survey system
IMPORT FROM A FILE: CHOOSE THIS ONE
Gist Qualtrics: Using the UNT online survey system
ONLY EXCEL CSV CAN BE IMPORTED
Gist Qualtrics: Using the UNT online survey system
Gist Qualtrics: Using the UNT online survey system
-OR-
1.
2.
3.
4.
Gist Qualtrics: Using the UNT online survey system
EMAIL SURVEY
You can type in, copy
& paste, select
everyone or
individuals from a
previous panel.
Gist Qualtrics: Using the UNT online survey system
CREATE CONDITIONAL SAMPLE FROM EXISTING PANEL
Set the Panel conditions
Gist Qualtrics: Using the UNT online survey system
-OR-
Enter by hand.
Note expansion of panel to
include embedded data.
You can use this to test the
integrity of emailing your
survey. Be aware that...
Gist Qualtrics: Using the UNT online survey system
Using this email
choice produces…
…NOT ALL FEATURES
ARE ACTIVATED IN THE
SEND TEST EMAIL
Not so good (using “Send Test
Email”)
Much better (using “Schedule
Mailing”)
Gist Qualtrics: Using the UNT online survey system
AUTHENTICATION: AVOID BALLOT BOX STUFFING; ASSURE CORRECT AUDIENCE ONLY IS
TAKING YOUR SURVEY. AUTHENTICATION USUALLY OCCURS BEFORE A PARTICIPANT IS
ALLOWED TO TAKE THE SURVEY, AND MAY RELY ON EMBEDDED DATA TYPICALLY INCLUDED IN A
PANEL. THE PANEL IS OFTEN ASSOCIATED WITH A CSV FILE THAT INCLUDES A PASSWORD, EUID,
OR OTHER UNIQUE EMBEDDED DATA. IT CAN BE THE SAME PANEL USED FOR DISTRIBUTION.
Start with the Edit Survey tab  Survey Flow  click Add a New Element Here  select Authenticator
click Move and drag Authenticate Using Panel to the top of the Survey Flow.
Place blocks and other element beneath the Authentication box.
Using the drop-down menus, select the fields used for authentication process.
Optional labels can be typed in if you want the field names to be different from those used in the Panel.
The minus and plus icons are used to add or remove fields from the authenticator.
Options bottom right corner = set parameters and text messages for authentication.
Click OK and save changes.
Go to http://www.qualtrics.com/university/researchsuie/advanced-building/survey-flow select
Authenticator for a 2.5 minute video describing the authentication procedures.
Gist Qualtrics: Using the UNT online survey system
QUALTRICS TRAINING SERIES
PART III
Downloading and Analyzing Results
OVERALL TIPS: ANALYZING DATA








In general, look for trends in the responses
Evaluate blanks as a possible clue that the respondent
didn’t understand the question
Be aware of “Christmas Treeing” where respondents
answer all the same way throughout every item – consider
deleting
Look at your response rates and compute a confidence
interval – go to: http://www.surveysystem.com/sscalc.htm
for a great resource
Share data internally and externally to drive changes in
practice
Look at your demographics, do they match your overall
population?
Are you missing certain members of your community? How
does this non-response affect your findings?
When in doubt, contact IR&E for a consultation on your
data and findings.
Gist Qualtrics: Using the UNT online survey system
VIEW RESULTS
Click here first!
Gist Qualtrics: Using the UNT online survey system
RECORDED RESPONSES: INDIVIDUAL
Gist Qualtrics: Using the UNT online survey system
DOWNLOAD DATA: TO CSV, SPSS (COMMONLY USED)
Gist Qualtrics: Using the UNT online survey system
CSV OUTPUT
Answers as Values
Answers as Labels
Gist Qualtrics: Using the UNT online survey system
SPSS DATA DOWNLOAD
Gist Qualtrics: Using the UNT online survey system
THANK YOU FOR YOUR ATTENDANCE TODAY.
THIS POWERPOINT IS AVAILABLE ON THE IR&E WEB
PAGE AT: HTTP://INSTITUTIONALRESEARCH.UNT.EDU/INSTITUTIONAL-ASSESSMENT
GIST QUALTRICS: SURVEY BUILDER TRAINING MODULE
Gist Qualtrics: Using the UNT online survey system
CONTACT INFORMATION:
Dr. Russell Ruffu
Office of Institutional Research and Effectiveness
Hurley 121E
Phone:
940.369.7894
Email:
Russell.ruffu@unt.edu
Web:
http://institutionalresearch.unt.edu/institutional-assessment
Gist Qualtrics: Using the UNT online survey system