Design of questionnaire
Holger Diessel
University of Jena
holger.diessel@uni-jena.de
http://www.holger-diessel.de/

Interview

Questionnaire

Experiment
Steps
1.
Establish the goals of the project - What you want to find out
2.
Determine your sample - Whom you will ask
3.
Choose the medium – How will you ask
4.
Create your questionnaire - What you will ask
5.
Pre-test the questionnaire - Test the questions
6.
Run the study and enter data - Ask questions and categorize
answer.
7.
Analyze the data - Produce the report
The goal of your study
State a precise research question (or hypothesis)
The sample

Who do you want to ask?

How many subjects do you want to ask?

How do you select your subjects?
Sampling




Convenient sample
Simple random sample
Stratified random sample
Cluster sample
Features to stratify
•
•
•
•
•
•
•
Age
Gender
Educational background
Profession
Regional background
Native language
Special interests
The medium
•
•
•
•
Reading questions aloud
Written questionnaire in face-to-face situation
Written questionnaire by mail or email
Internet survey
Form
• Keep your questionnaire short
• Keep it simple
Introduction
• Welcome message
• Brief explanation as to why you conduct the study
• Warming-up
Question types
(1)
(2)
What do you
like?
Do you like fish?
Open-ended vs. close-ended
(1)
What kind of music do you like?
(2)
What kind of music do you like?
(a) classical, (b) pop, (c) rock, (d) country
Rating scales
How would you rate this
product?
1. Excellent
2. Good
3. Fair
4. Poor
Rating scales
On a scale where 10 means you have a great
amount of interest in a subject and 1 means
you have none at all, how would you rate
your interest in each of the following topics?
1. Domestic politics
2. Foreign affairs
3. Science and health
4. Business
Rating scales
How much do you agree with each of the following statements?
Strongly
agree
The teacher provides constructive
criticism
The teacher follows the course
schedule
I would prefer to study more at
home
Agree
Disagree Strongly
disagree
Rating scales
Jena is a beautiful city.
Strongly
agree
Strongly
disagree
Rating scales
How often do you speak dialect?
Always
Frequently
Sometimes
Rarely
Never
Rating scales
Compared to other students, how would you rate this
student’s performance?
Lower 50%
Upper 50%
Upper 25%
Upper 10%
Upper 5%
Rating scales
Rating scales are also often used to elicit frequency
information. How often do you call your Mom?
__ Less than once a month
__ About once a month
__ About twice a month
__ Every week
__ Twice a week
__ Every other day
Rating scales
Always present <agree> before <disagree> and
>positive> before <negative>.
Rating scales
How many scores should your rating scale include?
Rating scales
Jena is a beautiful city.
Strongly
agree
Strongly
agree
Strongly
disagree
1
2
3
4
5
Strongly
disagree
Rating scales
What do you think about product X?
It's the best on the market
It's about average
It's the worst on the market
Direct vs. indirect questions
Direct:
‘What is the best instance of X?’
‘What is the second best instance of X?’
Indirect:
‘What is more similar to X?’
Don’t know / not applicable
For some questions it may be appropriate to allow
for a “Don't Know” or “Not Applicable” response.
Ordering of questions
• Motivation
• Influence
Ordering of questions: Motivation
Ideally, the early questions in a questionnaire
should be easy and pleasant to answer.
Ordering of questions: Influence
• Priming
• Habituation
Wording of questions
Make your questionnaire simple.
Wording of questions
Avoid ‘double-barreled’ questions’.
The subject in German is the first element of the
clause and sometimes appears in dative case.
Wording of questions
Avoid emotionally charged words or leading questions
that point towards a certain answer.
1. What do you think of the XYZ proposal?
2. What do you think of the Republican XYZ proposal?
Wording of questions
Make sure your questions accept all the possible
answers.
Is the NP the subject or object?
Coding
If you ask open questions you have to develop a
coding scheme .
Layout
Always consider the layout of your questionnaire. You
want to make it attractive, easy to understand and
easy to complete. Pictures and answering cells can
serve this purpose.
Pre-test
Run a small pre-test before you conduct the study.