Guidelines for creating Questionnaires
Introduction
It is virtually impossible to give a set of rigid rules for setting up a questionnaire. This
document therefore provides only general guidelines.
Planning prior to setting up a questionnaire









The purpose of the investigation or research project must be established before
the questionnaire is drawn up. The purpose dictates what type of information
should be obtained with the questionnaire. At this stage the researcher must do a
thorough literature study on the subject under investigation.
Decide whether a questionnaire is the most suitable method to capture the desired
information.
The purpose of the investigation will determine the target group, that is the
respondents to whom the questionnaire is directed.
The target group will determine the manner in which the questionnaire is to be
completed. In other words whether the questionnaires will be posted to the
respondents, completed in a group under supervision, be completed during
personal interviews, etc.
Consider the conditions under which the questionnaire is to be completed. The
conditions in a hut in the informal sector differ considerably from conditions in an
air-conditioned hall with comfortable desks.
Discuss the size of the sample, the definition of the population and the actual
drawing of the sample with a statistician. The final draft of the questionnaire must
also be discussed with the statistician and the research consultant. The
hypotheses to be tested must preferably be formulated beforehand to ensure that
all relevant information is included in the questionnaire.
Ensure that all information which may later be necessary for evaluation of the
realisation of the sample is included in the questionnaire.
The question of anonymity must be addressed. It must be clearly explained to the
respondent that all information will be considered confidential and that no personal
information will be published. No information that can identify the respondent
should be on the questionnaire. For example, the questionnaires should not be
numbered prior to sending it to the respondent. An exception must be made in the
case of studies where information must be obtained before and after certain events
or actions and the two sets of data must be linked. In this case information such as
a student number must be used and the situation must be explained to the
respondent.
Do a pilot study to test the questionnaire. If this is not possible the questionnaire
must be tested on a few persons similar to persons in the target group. This will
help to eliminate possible inherent problems.
1
General remarks regarding the contents







The target group determines the level of the questions. The contents of the
questions as well as the language and the terminology should not overestimate nor
underestimate the abilities of the respondent.
The length of the questionnaire is determined by the size of the sample as well as
by the target group. A very long questionnaire frustrates and exhausts the
respondent. If the sample size is not sufficiently large, questions with too many
categories as possible answers can lead to frequencies that are so low and
"sparse" that the information can either not be used at all or the categories must be
grouped together to be of any use at all.
The language must be correct and unambiguous. It is important that the possible
answers can be linked to the question. Questions must be restricted to one topic
only.
Psychometric questionnaires, that is questionnaires measuring aptitude, IQ etc are
not discussed here. Certain psychometric concepts are, however, relevant to the
development of questionnaires.
Construct validity indicates whether relevant constructs or concepts are being
investigated. For example, in a questionnaire about agricultural problems, valid
constructs would be droughts, maintenance of implements, etc
Content validity concerns the contents of the questions measuring the constructs. If
the construct is, for example, maintenance of implements then "quantity of
lubricants bought" is valid, but "amount spent on dipping fluid" is not.
Reliability indicates the extent to which the questionnaire will give the same results
repeatedly. Say, for example, the question is "How much did it rain?" The possible
answers "Plenty" and "Not much" will not have the same meaning in wet years as
in dry years. On the other hand "More than 40mm", "30-40mm" and "Less than
30mm" will have a constant meaning.
A few concepts


Variables e.g. Gender, home language, denomination, distance
Associated with each variable is one of the following:
 Categories e.g. Male, Female, Afrikaans, English, Tswana
 Values e.g. 10, 15, 20km or 5, 10, 15 years
 Scale values e.g. Excellent, Good, Moderate, Poor
 Scale types
 Nominal e.g. Male, Female
 Ordinal e.g. Excellent, Good, Moderate, Poor or the five classes of a
Likert Scale
 Interval/Ratio e.g. Temperature, Mass, Distance, Age in year
intervals
2
Classification of question types
Questions can be classified according to their function, which in turn is determined by
the purpose of the questionnaire. The classification in the following table is a guideline.
It is, however, important to remember that it seldom happens that only one kind of
question appears in a questionnaire.
Purpose
Type of question
Description
Biographic
Explaining
Behaviour and knowledge
Planning
Knowledge
Predicting
Attitude
Discussion of different types of questions
Biographical questions
Only the necessary information must be collected.
Questions requiring sensitive information from the respondent must be carefully
worded.
Age must preferably be posed as an open question, and if necessary, categorised at a
later stage. This enables one to use a larger variety of statistical methods.
The statistical techniques that will be used must be kept in mind during the compilation
of the questions. If the researcher plans to use techniques such as regression analysis
and analysis of variance then the information must, where possible, be collected in an
uncategorised form.
Questions must, as said before, be unambiguous and not open to misunderstanding.
Poor example:
Do you travel by bus in the mornings or in the evenings?
Better option:
Do you travel by bus in the mornings?
Do you travel by bus in the evenings?
Yes/No
Yes/No
3
Yes/No
The possible answers provided for a question must be logical and relevant for the
target group. Once again, the researcher must be sure that the required information is
collected.
For example, if the target group is children between 13 and 16 years of age and the
question is about highest educational qualification, the following options will not give
much information
None / Grade 3-7 / Grade 8-12 / B-degree / Post-graduate
Instead, the following options would be more relevant
None / Grade 3-6 / Grade 7 / Grade 8 / Grade 9 / Grade 10 / Grade 11 / Grade 12
Questions designed to determine frequency of events can be problematic. Once
again, the solution is a consistent clear formulation.
Poor example:
How often do you consult a doctor?
Regularly, frequently, occasionally, never
The concept frequently will definitely not have the same meaning for everybody
Better example:
How many times did you consult a doctor between January, 1 2003 and December,31
2003?
Once, two or three times, four or five times, more then five times, never
In the last example the period is clearly defined, the frequency of visits is stated
unambiguously and provision is made for respondents who did not visit the doctor.
Sifting questions
In the previous example a sifting question can be used and followed by an open
question.
Example:
Did you consult a doctor during the period.........?
Yes/No
If yes, how many times?
........
The advantage, again, is that more statistical techniques can be used.
Open questions
Open questions have no formulated categories for possible answers. This type of
questions are usually used when insufficient knowledge regarding the particular
subject exists and the researcher is uncertain whether predefined categories will
cover all possibilities. Open questions are, therefore, particularly useful for exploratory
studies. It can also be used to obtain more information regarding previous closed
questions or to test the intensity of the response. As the coding of this type of question
is time consuming it should only be used when absolutely necessary.
4
"Answer each one" or "Choose only one"
The use of these two types of answers depends on the question and the purpose of
the study. "Answer each one" is used when there is more than one possible answer to
the question. It also forces the respondent to take each given category into
consideration. The following method is recommended:
Why did you start smoking?
Wanted to know what it felt like
Yes
No
I was dared
Yes
No
I wanted to prove myself
Yes
No
I did not want to be different from my friends
Yes
No
Boredom
Yes
No
I heard that it was relaxing
Yes
No
If the respondent was asked to choose all relevant reasons, as in the next example, it
cannot be assumed that a reason that has not been chosen yields "No" as in the
previous example.
Why did you start smoking? Choose all relevant reasons.
Wanted to know what it felt like
I was dared
I wanted to prove myself
I did not want to be different from my friends
Boredom
I heard that it was relaxing
"Choose only one" is used in cases where the question indicates that the most
important reason or aspect must be chosen.
The target group must be considered, as it is sometimes easier for respondents to
answer Yes/No to each possibility than to indicate the most important reason.
Ranking
Questions that expect the respondent to rank a number of reasons, causes or
anything else must only be used when the researcher is sure that the respondents will
be able to manage this. Even then the number of possibilities to be ranked must be
kept to a minimum. It must also be clearly indicated that the respondent must rank
every possibility.
5
Direct against sympathetic approach
Example:
Are you overweight?
Against:
Do you think that you are overweight?
Again there can be no fixed rule. The researcher must be guided by the target group,
the nature of the study and the specific question. Sometimes a more sympathetic
approach or a question that is worded in a conversational style would be more
effective in motivating a respondent to answer a sensitive question. The length of the
question must, however, be restricted as much as possible.
Time dimension of events
Do not overestimate the respondent's memory. Questions must be restricted to
important or recent events.
Unreasonable demands must not be made on the respondent. The relevant time
period must be clearly defined.
Example:
Did you buy a new car during the past year?
What is regarded as the past year? 2003? April 2003 to March 2004?
It is sometimes possible, given the subject and the size of the sample, to make
deductions over time by restricting the question to a specific time.
Example:
How many times did you go to church during August?
(Where August is the month preceding the survey)
If the sample size is sufficient and one assumes that the church attendance pattern is
relatively constant, it is reasonable to expect that exceptional behaviour will be
cancelled out and that the monthly church attendance of respondents will be reflected.
Reliability
It is difficult to check the answers to biographical questions and these questions must
be formulated in such a way that the respondent can give honest answers.
Knowledge questions
Most questionnaires include such questions not to test the respondent’s knowledge as
such but because knowledge is the foundation of certain behaviour or because it helps
to form attitudes.
Sifting questions are valuable in this case, if a respondent indicates that he/she has
never heard of a chiropractor, then it does not make any sense to ask this specific
respondent any questions regarding the work of a chiropractor.
The respondent's frequency of contact with the subject must also be taken into
account. The knowledge about bus routes of a person who travels by bus every day
will be more "reliable" than the knowledge of a person who only travels by bus
occasionally.
6
For the same reasons as discussed previously with the Yes/No answers, the preferred
type is True/False. In some cases "Don't know" or "Uncertain" can be added as a
category. The two possibilities are not equivalent and must be handled carefully.
Questions regarding opinions and attitudes
With regard to attitudes, there is a difference between determining the nature of the
attitude and determining the intensity of the attitude.
In the first case the answer to each question is important and must be reported
separately. In the second case a number of questions that theoretically measure the
construct must be included. Techniques like factor analysis must then be used to
ascertain that this is proved in practice. The group of questions that clearly measures
a construct is then used as a entity and values or totals can be reported for the
constructs.
The different scales that can be used will now be discussed.

Dichotomous scale
Example:
The bus service satisfies my needs
Yes
No
Buses depart on time
Yes
No
Bus tariffs are too expensive
Yes
No
In this case the answer to each question is analysed on its own.

Likert scales
This type of scale can be used to measure intensity of feelings. As mentioned
before, a specific theoretical construct is supposed and with the help of statistical
techniques it must be determined whether the data confirms the theory.
Example:
Strongly
Agree
Agree
Neutral
Disagree
Strongly
Disagree
Travelling by bus is
time consuming
1
2
3
4
5
Buses are always
late
1
2
3
4
5
Bus drivers drive
recklessly
1
2
3
4
5
7

Diagram scale
This scale determines the respondents' tendency to both sides of a central point.
Example:
Punctuality
::
::
::
::

::
::
::
::
0
::
::
::
::
+
Semantic differential scale
This is an extension of the diagram scale as the concepts that are measured in
words to the scale.
Example:
BUS SERVICE
Excellent :_:_:_:_:_:_:_: Poor
Clean :_:_:_:_:_:_:_: Dirty
Quick :_:_:_:_:_:_:_: Slow
Whenever this scale is used it is important to give a proper example.

General remarks regarding scales
When deciding which statistical techniques to use it must be remembered that the
distances between the possible answers do not invariably have the same value.
The number of categories that are used must be restricted to the minimum.
A problem, that may arise with certain target groups, is that respondents choose all
the neutral answers or answers that are considered to be "desirable". This
problem can, largely be avoided by giving proper instructions, asking relevant
questions and, while coding the questionnaires, being mindful of problem cases.
The difference between knowledge, opinions and attitudes
The difference is not always very clear and the literature sometimes differentiates on a
continuum between cognitive and evaluative attitudes. In general, a question with
"Don't know" as a possible answer indicates measurement of knowledge, whereas a
question with "Unsure" as a possibility in turn indicates measurement of attitude.
The phrasing of questions
This is the most important aspect of the questionnaire and must be properly attended
to.
Example:
Speeches against democracy should not be allowed.
will elicit a different response to:
Speeches against democracy should be prohibited.
8
"should not be allowed" is more neutral than "should be prohibited"
Adjectives indicating intensity should be used with care.
Hypothetical questions rarely yield much information. The respondent must be familiar
with the subject or object regarding which his attitude is being determined.
Where examples on how to answer questions are given, these must be clearly
separated from the rest of the question and identified as an example. Examples must
merely show how to answer questions and not be indicative as to the type of answer
that is expected.
Sequence of questions
As a general guideline similar questions and questions regarding a common subject
should as far as possible be grouped together to give the questionnaire a logical flow.
In some cases it might be better to put sensitive questions, that the respondent may
possibly refuse to answer, at the end of the questionnaire. In this way the maximum
amount of the information will be gained from the respondent.
Source
Adapted and translated by Rina Owen from:
CS Steenekamp: "Praktiese riglyne vir vraelyskonstruksie"
9