Questionnaire design
A survey is only as good as the
questions it asks
Emotional Intelligence

http://quiz.ivillage.co.uk/uk_work/test
s/eqtest.htm
What should you ask?


The questions asked are a function of
previous decisions
The questions asked are a function of
future decisions (such as statistical
analysis)
Key criteria

Questionnaire relevancy
– No unnecessary information is collected
and only information needed to solve the
problem is obtained. Be specific about
your data needs; tie each question to an
objective

Questionnaire accuracy
– Information is both reliable and valid
Phrasing Questions

Open ended response versus fixed
alternative questions
“?”

Decision criteria: type of research;
time; method of delivery; budget;
concerns regarding researcher bias
Avoid







Leading questions
Overly complex questions
Use of jargon
Loaded questions (can use a counterbiasing
statement)
Ambiguity
Double barreled questions
Making assumptions
Order?

Order bias results from an alternative
answer’s position in a set of answers
or from the sequencing of questions
– Funneling technique: general to specific
helps understand the frame of reference
first

Anchoring effect: the first concept
measured tends to become a
comparison point from which
subsequent evaluations are made
Decisions





Ranking, sorting, rating or choice?
How many categories or response
positions?
Balanced or unbalanced?
Forced choice or nonforced choice?
Single measure or index?
Types of questions
Types of fixed alternative questions…
• Single dichotomy or dichotomous-alternative
questions
“Are you currently registered in a course at the
University of Lethbridge?
Yes____ No____”
• Respondent chooses one of two alternatives
(yes/no; male/female)
• What scale would this data create?
Types of fixed alternative
questions…

Multi-choice alternative
– Respondent chooses from several
alternatives
– Many types…
Multi-choice alternative
questions…

Determinant choice
– Choose only one from several possible responses
“Which faculty are you currently registered in at
the University of Lethbridge?
Management ___
Education ____
Arts/Science____
Health sciences____
Combined degree____

What type of scale would these data create?

Frequency determination
– Asks for an answer about frequency
of occurrence
In a typical week, how often do you
purchase chocolate chip cookies?
__never
__ once
__ 2 or more times
What type of scale would these data create?

Check list
– Provide multiple answers to a single question
– Should be mutually exclusive and exhaustive
“What brands of chocolate chip cookies have you, to
the best of your memory, purchased in the past
month (check all that apply?)”
__ Dare
__ Chips A’hoy
__ Presidents Choice Decadent etc. etc.

What type of scale would these data create?

Attitude rating scales
Attitude:
An enduring disposition to consistently
respond to various aspect of the
world, including persons, events and
objects
Typically seen as having three
components:
– Cognitive
– Affective
– Behavioural
Affective
The feelings or emotions toward
an object
Cognitive

Knowledge and beliefs
Behavioral



Predisposition to action
Intentions
Behavioral expectations
Attitude Scales: Scaling Defined
The term scaling refers to procedures for
attempting to determine quantitative measures of
subjective and sometimes abstract concepts. It is
defined as a procedure for the assignment of
numbers to a property of objects in order to
impart some of the characteristics of numbers to
the properties in question.
Unidimensional
Scaling
Multidimensional
Scaling
Procedures
designed to
measure only one
attribute of a
respondent or
object
Procedures
designed to measure
several dimensions
of a respondent or
object
Attitude measuring
process




Ranking
Rating
Sorting
Choice
Types of attitude scales



Simple attitude scales
Most basic form – respondent responds to a
single question
Do not allow for fine distinctions or
placement on continua
– You are at a company party and are feeling
nervous, but you are obligated to be there. Do
you:
__ find someone you know to buddy up with
__ take it as an opportunity to meet new people
What type of scale would these data create?

Category scales
– More sensitive; provides more information
– Overall, how satisfied are you with the high speed
performance of your Mercedes:
__ very satisfied
__ somewhat satisfied
__ neither satisfied nor dissatisfied
__ somewhat dissatisfied
__ very dissatisfied
If you could choose, how long would each term be?
___26 weeks __ 13 weeks
__ 6 weeks ___4
weeks
What type of scale would these data create?

Summated rating scales – the Likert
scale
– Respondents indicate their attitudes by
checking how strongly they agree or
disagree with statements
– Chocolate chip cookies are my preferred
variety of cookie
Strongly disagree Disagree Uncertain Agree
(1)
(2)
(3)
Strongly Agree
(4)
(5)
What type of scale would these data create?

Semantic Differential Rating scale
– An attitude measure consisting of a series
of seven-point bipolar rating scales
allowing response to a “concept”
Think of your favorite type of cookie. Rate it on each
of the following continua:
Hard-----------------------------------------------------Soft
Lots of chips---------------------------------------Fewer
chips
Crispy--------------------------------------------------chewy
What type of scale would these data create?

Numerical Rating scale
– Similar to a semantic differential except that it
uses numbers as response options to identify
response positions instead of verbal descriptions
Think of your favorite type of cookie. Rate it on each of the
following continua:
Hard-----------------------------------------------------------------------Soft
8
7
6
5
4
3
2
1
This scale is called an 8 point numerical scale, why?
What type of scale would these data create?

Constant Sum Scales
– Attributes based on their importance to the
person. Respondents are asked to divide a
constant sum to indicate the relative importance
of attributes
Example: Suppose the photocopy budget per
professor was $100 per month. How much should
be allocated to the following. Divide the $100
according to your preference:
____ photocopying for student needs;
____ photocopying for research needs;
____ photocopying for committee needs.
====
$100 TOTAL

Stapel Scales
– An attitude measure that places a single adjective
in the center of an even-number range of
numerical values
Example:
Research Methodology
+3
+2
+1
Exciting
-1
-2
-3

Graphic Rating Scales
– An attitude measure consisting of a graphic
continuum that allows respondents to rate
an object by choosing any point on the
continuum

Rank-Order Scales
– Scales in which the respondent compares
one item with another or a group of items
against each other and ranks them.
Example: handout
Most important skills











Adaptability to change
Problem identification
Listening skills
Written communication
Leadership
Informal Oral communication
Analytical thinking/problem solving
Time management
Coping with stress/job pressures
Interpersonal relations
Formal oral presentations
Most important skills
Adaptability to change
Problem identification
Listening skills
Written communication
Leadership
Informal Oral communication
Analytical thinking/problem solving
Time management
Coping with stress/job pressures
Interpersonal relations
Formal oral presentations
8
6
1
2
4
3
5
7
11
9
10
9
6
1
4
2
3
5
10
7
8
11

Paired Comparison Scales
– Respondent is presented with two objects
and is asked to pick the preferred.
Example: Which type of cookie do you prefer
__ chocolate chip
__ oatmeal
__ I do not have a preference between these two

Sorting
– Respondent indicates their attitudes or
beliefs by arranging items.
Example: Please sort the following cards with
pictures of cookies into the following categories
Like
Dislike
Neither like nor dislike