Questionnaire Design
EPIET Introductory Course
Lazareto, Menorca
6 October 2011
K Alpers
C Campese, P McKeown, V Bremer, V Prikazsky
1
What is a questionnaire?
• A tool for data collection
• A series of written questions
in a fixed, rational order
2
A well designed questionnaire
• Good appearance (easy for the eye)
• Short and simple
• Relevant and logical
⇒ High response
⇒ Easier
to collect
to summarize
to analyse
• Minimises potential sources of bias
3
Bias
Systematic difference in the response measurement
• Recall bias
– Cases more likely to remember than controls
• Observer bias
– Different interviewers – different interpretations
– Different interpretation of similar questions
• Non-response bias
– telephone interviews: more females, elderly
4
How to reduce bias
•
•
•
•
Structured questionnaire
Ensure high response rate
Random choice of interview partners
Training of interviewers
5
Advantages of questionnaires
• Can reach a large number of people
• Relatively easy and economic
• Relate directly to study question
• Provide quantifiable answers
• Relatively easy to analyse
6
Disadvantages of questionnaires
• Provide only limited insight into a problem
– the range of possible responses is limited
– the question maybe misleading
• Varying response
– Unclear question can lead to
• misunderstanding
• misinterpretation
• Do not allow for mistakes
– must be right from the beginning
– missing data hard to chase
7
Types of Questionnaires
• Interviewer-administered
– face to face
8
Types of Questionnaires
• Interviewer-administered
– face to face
– telephone
9
Types of Questionnaires
• Interviewer-administered
– face to face
– telephone
• Self-administered
– by post
10
Types of Questionnaires
• Interviewer-administered
– face to face
– telephone
• Self-administered
– by post
– Email
– Internet
– Social Networks
11
Self-administered questionnaire
• Advantages
• cheap and easy to administer
– preserves confidentiality
– completed at respondent's convenience
– not influenced by interviewer
12
Self-administered questionnaire
• Advantages
–
–
–
–
cheap and easy to administer
preserves confidentiality
completed at respondent's convenience
not influenced by interviewer
• Disadvantages
–
–
–
–
–
–
low response
questions can be misunderstood
no control by interviewer
only literate persons
time delay (post)
e-mail/internet: need computer access/software
13
Interviewer-administered questionnaire
• Advantages
– participation of illiterate people
– clarification of ambiguities
– quick answers
14
Interviewer-administered questionnaire
• Advantages
– participation of illiterate people
– clarification of ambiguities
– quick answers
• Disadvantages
– interviewer bias
– needs more staff resources
– only short questionnaires possible
• especially on telephone
– difficult for sensitive issues
15
Before starting to design a questionnaire
Write a study
protocol!
• define objectives
• define study questions
Risk factors for being
stung by jellyfish at
Lazareto (Menorca),
October 2011
Study protocol
Prepared by cohort 17
Menorca, 6 October 2011
16
Study protocol and questionnaire
Study protocol
questionnaire
Definition of
objectives
Design
questions
questions must relate to
objectives
List of variables to
collect
Analysis plan
collect appropriate
information for analyses
Control and
reading
Pilot test
17
Use existing questionnaires
• As an inspiration
• Don´t need to re-invent the wheel
• Have been tested
• BUT: adapt to present situation!
18
Questionnaires for outbreak
investigations
• Exploratory
–
–
–
–
–
Generate hypotheses
Detailed
Only some cases
Based on existing questionnaires
Open questions
• Analytical study
– Testing hypothesis
– Focus on possible vehicles
19
Questionnaire needs to be adapted
to study population
• Know the respondents
– language
– education
– occupation
– ethnic group
– sensitive issues
20
Structure of a questionnaire
1
Identification
2
Interview introduction
3
Instructions on how to answer
4
Questions
5
Conclusion
21
1 - Identification
• On first page
– Return address
– Study title
• On all pages
– Identifyer
– Page numbers
• Data protection:
Identifiers kept
separately from
names
ARZTFRAGEBOGEN
Aktenzeichen:____________ Name, Vorname: _____________________ Geb.: ___.___._____
Masern-Ausbruch Nordfriesland Juni 2001
Dr. Gerard Krause
Abt. Infektionsepidemiologie
Robert Koch-Institut
Seestrasse 10
13353 Berlin
Vom Befrager einzutragen:
- Datum der Befragung:
- Initialen des Befragers (Vor-+ Nachname):
- Name des Arztes:
___.___._____
___ ___
________________________________
Guten Tag, mein Name ist ___ (bitte Namen nennen)___.
Ich bin Mitarbeiter im
Kreisgesundheitsamt in Husum
oder
Als Mitarbeiter des Robert Koch-Institutes
unterstütze ich das Kreisgesundheitsamt Husum.
Wir verzeichnen derzeit im Landkreis Nordfriesland seit Anfang September 2005 eine ungewöhnliche
Häufung von Masernerkrankungen. Wir möchten uns zunächst herzlich dafür bedanken, dass Sie uns
Ihren Patienten ___(Name des Falles wie in Kopfzeile)___, geboren am __(Datum wie in Kopfzeile)__,
als Masernfall gemeldet haben. Wir führen derzeit eine epidemiologische Untersuchung durch um diese
Masernhäufung eindämmen zu können. Das Erfassen von Masernkomplikationen ist hier besonders
wichtig. Um den von Ihnen gemeldeten Fall korrekt einordnen zu können, möchten wir noch kurz einzelne
Informationen abklären.
1. Hatte Ihr Patient eines der mehrere der folgenden Symptome?
Generalisierter Hautausschlag > 3 Tage:
(1)  Ja*
(2)  Nein
(3)  Unbekannt
* Wenn ja: Wann begann der Hautauschlag?
(4)
(5)
 Datum genau bekannt: ___.___.2005
 Datum nicht genau bekannt: Auschlag begann zwischen ___.___.2005 und ___.___.2005.
2. Welche der folgenden Labornachweise liegen Ihnen zu diesem Patienten vor?
(Mehrfachnennungen möglich)
 Virusisolierung
 Nukleinsäurenachweis (PCR)
(3)  IgM
(4)  IgG mit 4-fachem Titeranstieg
(5)  Antikörpernachweis mit 4-fachem Titeranstieg
(1)
(2)
3. War Ihr Patient gegen Masern geimpft?
Erste Masernimpfung:
 Nein
(2)  Ja* (nicht dokumentiert)
(3)  Ja* (Impfausweis)
(1)
(bitte eine Angabe pro empfohlene Impfung)
Zweite Masernimpfung:
 Nein
(2)  Ja* (nicht dokumentiert)
(3)  Ja* (Impfausweis)
(1)
Dritte Masernimpfung:
 Nein
 Ja* (nicht dokumentiert)
(3)  Ja* (Impfausweis)
(1)
(2)
22
2 - Introduction
• Covering letter/ interview introduction
– Who are you / you work for
– Why are you investigating
– Where did you obtain the respondent’s name
– How and where can you be contacted
– Guarantee of confidentiality
– Length of interview (be honest)
⇒ Usefulness of study should be clear
to all respondents
23
Example of introduction
• Good morning ,
My name is ..... , I work for ……..
You may have been already informed that a survey on risk factors for
being stung by a jellyfish will be done this week in Mahon. This study
has been approved by the national ethical committee. Only anonymized
data will be analysed. You have been randomly selected to participate
in this study. Your participation is voluntary. The interview will take
about 10 minutes.
I will start with the first question .....
24
3 - Instructions
• Minimise potential sources of bias
• Guide for Interviewers
• Guide for Respondents in self-administered questionnaires
– Which questions can be skipped => where to jump to
– Selection of multiple answers possible
• Use different fonts (e.g. bold or italics)
Example: I will ask you a few questions about your
contacts with jellyfish.
You will answer by yes or no
If no, go to question 27
25
4 - Questions
1. Content
2. Order
3. Format
4. Coding
5. Pitfalls
26
4.1 - Content of questions
• Clear focus on research question
– avoid sidetracking
– avoid unnecessary information
27
4.2 - Question order
• Decide on the order of items/questions
– easy
 difficult
– general  particular
– factual  abstract
• Group questions by topic
• Be aware of ordering effects
• Don’t put the most important items last
• Where to place sensitive questions?
28
Starting questions
– “Door-opener”
– Simple
– Closed format
– Relevant to main subject
– Non-offending
– Neither demographic nor personal questions
29
4.3 - Format of questions
• Adjust to responding audience
– professionals vs. public
– middle class vs. prisoners
• Keep sentences simple and short
• Define key words (“fully vaccinated”)
• Remember options
– “don’t know”
– “don’t want to answer”
30
Ask one thing at a time
7 - Do you like to go swimming and do you mind to be
stung by jellyfish?
 Yes
 No
• Ask for one information at a time
• Answer options
– Mutually exclusive and exhaustive
– Vertical order of answer options
31
Be accurate
8 – What is the jellyfish situation?
 Good
 Bad
Versus
8 - How often did you see jellyfish during the
last week?
 Once
 Twice
 Three times or more
 Never
 Don´t know
32
Be objective
10 - Do you agree that jellyfish are annoying?
 Yes
 No
versus
10 - According to you, jellyfish in Lazareto are…
 not at all a public health problem
 not a major public health problem
 a potential public health problem
 an important public health problem
 no opinion/don´t know
33
Be simple
11- Did you see more than an average of 33 jellyfish/m2 salt
water surface on more than 3 occasions that you went
swimming in the morning last week?
 Yes
 No
versus
11- Have you seen jellyfish on more than 3 mornings last
week?
 Yes
 No
 Don´t know
34
Two main question formats
• Closed format  forced choice
Yes
No
Don’t know



Always
Sometimes
Never



• Open format  free text
12 – What did you do to avoid being stung by jellyfish?
Please describe :
__________________________________________
________________________________________
35
Advantages of open questions
• not directive
• detailed and unexpected answers possible
– > allow exploration of issues to generate
hypotheses
• useful for exploring knowledge and attitudes
• qualitative research
• focus groups
• trawling questionnaires
36
Disadvantages of open questions
• interviewer bias
• time-consuming
• coding problems
• difficult to analyse
• difficult to compare groups
37
Advantages of closed questions
•
•
•
•
Straightforward response
Simple
Quick
Less discrimination against less verbally
expressive people
• Easy to code, record, analyse
• Easy to compare
• Easy to report
38
Disadvantages of closed questions
• restricted number of possible answers
• Possible loss of additional information
Compromise
– insert field
“others,
if yes specify : __________”
39
Examples for closed questions
2 - What is your age in years?
___ years
3 - How long have you stayed at Lazareto? ___ days
4 - What is your sex (gender)?
 Male
 Female
5 - Did you go swimming on 4 October 2010?
 Yes
 No
 Don’t know
40
Closed questions: Checklist
14 - Which of the following beaches have you visited during
your stay in Menorca?
 Lazareto beach
 Cala’n Porter
 Rafalet
 Macarella
 Sa Mesquida
41
Checklist
14 - Which of the following beaches have you visited during
your stay in Menorca?
Lazareto beach
 Yes
 No  Don´t know
Cala’n Porter
 Yes  No  Don´t know
Rafalet
 Yes  No  Don´t know
Macarella
 Yes  No  Don´t know
Sa Mesquida
 Yes  No  Don´t know
42
Rating scale
15 –
How often did you see jellyfish
during the past weeks?
Always Sometimes Seldom
Mornings



Lunchtime



Evenings



Never



43
Rating scale
Numerical
16 - How severe was your pain after you were stung?
(please circle)
1 2 3 4 5 6 7
Not painful at all
Very painful
Analogue
17- How severe is your pain (put the tick on the line)
0
10
44
Likert Scale
•
•
•
Rensis Likert, 1903-1981
Psychometric scale
Five (or more) ordered response levels
18 – Jellyfish also have the right to swim in the
Mediterranean sea
 I strongly disagree
 I disagree
 I neither agree or disagree
 I agree
 I strongly agree
45
4.4 - Coding
• Answers can be pre-coded
• Quicker and easier data entry
• Examples:
Male
Female
Don’t know
Lazareto beach
Harbour
Es Castel
Mahón
1
0
9
Ill
Not ill
Don’t know
1
0
9
1
2
3
4
46
4.5 - Avoid pitfalls
• jargon/abbreviations/slang
21- Should jellyfish victims receive a PEP?
(post exposure prophylaxis)
• not mutually exclusive options
22 - What is your age ?
 16-20
 20-25
 25-30
 35-40
47
Avoid….
• Leading questions
23 - Do you think we should forbid EPIET-fellows to go
swimming in the mornings?
• Making the questionnaire too long
• Typographical / spelling errors
48
5 - Conclusion
• Don’t forget to thank the interviewed persons
• Tell them when the results will be available
and where
49
Example of conclusion
This is the end of the interview. Thanks for answering this
questionnaire. The result of this study will be available on
the ECDC website in six weeks.
Do you have any others questions you wish to ask ?
In some outbreak investigations you might want to add:
We are still in a very early phase of our investigation. Would
you agree to be contacted again in case further questions
arise?
50
Presentation and layout
• Clear and consistent
• adequate space to answer
– large font size
– appropriate page breaks
• avoid
• messy layouts
• too many and fancy logos
• printing questionnaire on coloured paper may
help
51
Pilot your questionnaire!
• Pilot with a group of people
-> similar to your target subjects
• Highlight problems before starting
– Misunderstandings
– Look for alternative wording
– Final polishing
52
Summary
A well designed questionnaire:
• collects appropriate data
-> to answer your research question
• minimises potential sources of bias
-> increases the validity of the replies
• Will much more likely be completed
53