Collecting Primary Data:
Questionnaires
Objectives
After this session you will be able to:
• Plan and design valid and reliable
questionnaires.
• Describe the processes involved in collecting
primary data, including piloting.
• Demonstrate the skills for writing appropriate
individual questions and designing
questionnaires.
• Write appropriate documentation to accompany
questionnaires.
Types of questionnaire
Questionnaire
Self-administered
Postal
questionnaire
Online
questionnaire
Delivery &
collection
questionnaire
Interview-administered
Focus
group
Structured
interview
Telephone
interview
Why questionnaires?
• Low costs (time and
money).
• Quick inflow of data.
• Can reach many
people.
• Quick coding of
closed questions.
• Reliability of data.
Questionnaires – disadvantages
• Low response rates
(threat to reliability).
• No opportunity for
probing questions.
• No opportunity for
respondent to seek clarity
(threat to validity).
• Socially desirable
responses (SDRs) (threat
to validity).
Face validity
• Do the questions make sense?
• Are all concepts in the questions
understandable?
• Are the questions interesting and
motivating to answer?
• Are the instructions for completing the
questionnaire clear?
• Is the questionnaire of the right length?
Achieving face validity
• Get your questionnaire reviewed by 15 of
your peers.
• Get them to write down the problems
they find on the questionnaire.
• Talk to them.
• Note which elements need improving and
comment on how you did this in your
Methodology.
Design: construct validity
Construct validity: an instrument measures
the constructs it was intended to measure.
Theory
Concept
Constructs
Question sets
Marketing
Relationship
Marketing
- Loyalty
- Internal
customers
- Supply
chain
Question sets
- Loyalty
- Internal
customers
- Supply
chain
Measuring attitudes: reliability
Reliability: the consistency between two
measures – sufficient numbers of
questions for each construct.
Loyalty:
• To me, brand loyalty is important.
• I tend to buy the same brand all the time.
• If I like a brand I always buy it in preference to others.
• Brand loyalty is unimportant in my buying decisions.
Reliability of responses
Loyalty:
• To me, brand loyalty is
important.
• I tend to buy the same
brand all the time.
• If I like a brand I
always buy it in
preference to others.
• Brand loyalty is
unimportant in my
buying decisions.
Note: these questions will be randomized
– scattered throughout the questionnaire
Measurement of reliability
• Cronback alpha: at least 0.7.
• If below 0.7 you need to eliminate or make
changes to low reliability questions.
Questionnaire design:
closed questions
• Seek a predefined
response: Yes/No;
True/False, etc.
• Easy to code and
analyse.
• Make group comparisons
easy.
• Low cost, but…
• May restrict richness of
responses.
Questionnaire design: open
questions
• How, why, what …?
• Good for eliciting rich
responses, but…
• Harder to analyse.
• Example: You have
indicated a loyalty to
Krispy Crisps – please
explain why -----------------------------------------
Questions to avoid (threats
to validity)
• Leading: Why do you like Krispy Crisps?
• Double: Do you like Krispy Crisps or
Walkers Crisps? YES/NO
• Assumptive: How often do you eat Krispy
Crisps?
• Hypothetical: Suppose you were asked to
eat 20 packets of Krispy Crisps…?
• Knowledge based: Recalling the recent
Krispy Crisp advertising campaign…
Good (valid) questions are…
•
•
•
•
•
Simple and easily understood.
As short as possible.
Unambiguous.
Not likely to offend.
Provided with categories such as ‘Don’t
know’ and ‘Not applicable’.
What’s wrong here?
•
•
What’s your view on crisps?
State your age
Under 20 20 - 25
•
•
•
25 - 30
Over 30
Without effective ‘best practice’ crisp making
cannot prosper.
How many packets of crisps have you stolen?
COMPLETE AND RETURN THIS FORM
Response categories (1)
List question
What do you think is the most important influence on the success of the
organization in the next two years?
Please as many responses as you think accurate.
Changes in government policy
affecting the legal regulation of
the market

The entry of new competitors to
the market

The impact of the company’s
current reorganization strategy

Foreign exchange rates

Response categories (2)
Category question
How often in an average week do you use our e-banking facilities?
Please  one response.
Never
Once
2–3 times
4–5 times
6 times or more





Response categories (3)
Ranking question
Please indicate in the boxes provided which features you believe are
the most important when visiting our superstore
(1 indicating the most important, 2 the next most important, etc.) .
Please leave blank those features that have no importance at all.
Ease of car parking
Low prices
Friendly staff
Store loyalty card
Variety of goods





General Instructions
Title of survey
This survey is being undertaken by [your name] as
part of [name of course] at the University of Surrey
to [state purpose].
It will take approximately x minutes to complete.
All information provided is completely confidential.
Thank you for providing your comments.
Please return to:………………by………
Specific instructions: go to
12 Do you eat Krispy Crisps?
No

Go to 18
Yes
13 How many packets per week?
Pilot the questionnaire
Through piloting you may decide to:
• Change the wording of individual
questions.
• Add/eliminate some questions.
• Amend the instructions.
• Decide not to use the questionnaire!
Summary
• Designing individual questions involves a rigorous process of
analysis to avoid ambiguity, leading questions, double questions
and simply misleading questions.
• Questions must be clearly linked to the purpose of the research
• Questionnaires should start with questions that are easy to answer,
interesting and transparently linked to the purpose of the research.
• Questionnaire layout and the use of typography can make a
questionnaire easier to complete and more appealing to
respondents, enhancing the response rate.
• Clear, well set out instructions on completing the questionnaire can
also boost the response rate.
• Web and email questionnaires offer a new and potentially powerful
tool, but also require additional design skills.
• All questionnaires, whether paper-based, email or Web-based,
require thorough piloting which will include evaluation of
accompanying documentation, instructions, individual questions,
types of question, question sequencing, the use of scales and skip
instructions – basically, everything!