Chapter 7
Collecting quantitative data
Suggested solutions to questions and exercises
1. Describe the role of the interviewer in the data collection process.
The role of the interviewer is twofold: he or she must contact people who match the
recruitment criteria of the survey, encourage them to take part in an interview and
administer the interview. This is a skilled job. It requires a high level of interpersonal
skill, a sound understanding of the data collection and the research processes, including
responsibilities under data protection legislation and professional codes of practice.
2. Why are interviewer training and interviewer briefing important in the collection of
good quality data?
Interviewers must be familiar and comfortable with the questionnaire and the
interviewing process in order to collect good quality data. Interviewer training and
interviewer briefing are two ways in which the quality of the process can be enhanced.
Much research has been done on the effect an interviewer has on response rates and on
the quality of data collected. There is evidence to show that appearance, age, gender,
social grade, ethnic background, religion and attitude or personality have an effect on the
interviewing process and on the outcome of the interview. This is not confined to faceto-face interviews. Research shows that respondents in telephone interviews make
judgements about an interviewer's characteristics on the basis of his or her voice. To
minimise the effects of interviewer variance, interviewers are trained to carry out
interviews according to the instructions provided and to do so in a professional, courteous
and objective way.
Uniformity or consistency of approach is a key aim in structured and standardised
quantitative research – data must be collected in the same way across the sample and any
possible bias or error in asking questions or recording responses must kept to a minimum.
It is important therefore that each respondent is asked the questions on the questionnaire
in exactly the same way. Training and briefing are key to achieving this. Interviewers
are trained to read out instructions and ask the questions exactly as they appear on the
questionnaire, and in the way that they were briefed to do (a change of emphasis on a
word can change the meaning). For some questions, such as those with an 'other' code in
the list of pre-coded responses, or where the respondent says 'Don't know' or 'Not sure',
the interviewer is trained and briefed to probe in the appropriate way. Where there are
open-ended questions, questions that require the respondent to answer in his or her own
words, the interviewer is trained to record the answers verbatim. If probing is needed at
open questions, the interviewer is briefed to follow the specified probing/prompting
procedure set out on the questionnaire or specified in his or her training and in the
briefing for the particular study.
1
3. How do you decide on the most appropriate method or methods of data collection for
a particular research project?
In deciding on the most suitable method or methods of data collection, it is important to
consider:
 the study and its objectives
 the topic or issues under investigation
 reaching the right sample
 achieving the right numbers
 the time and budget available.
If you have a subject of a very sensitive nature, for example, telephone data collection
may be the best option as it offers the respondent a degree of anonymity and distance that
a face-to-face interview does not. If you have a sample that is hard to reach in person – a
sample of business executives, for example, the telephone or a postal or email survey
may be the only way of contacting them. If you need to show respondents stimulus
material, for example, an advertisement, or get them to try a product, a face-to-face
approach may be the only feasible one. If you need to achieve a particular sample size,
you may not choose a postal survey or an email survey unless you are fairly sure that the
return or completion rate (which can sometimes be hard to predict) will give you the
numbers (and the sample) you need. If you are working to a tight budget, you might
consider a postal survey – with no interviewer costs it can be cheaper than a telephone or
face-to-face survey. If you are working to a tight deadline, a postal survey may not be
appropriate – turnaround times are often relatively long – therefore a telephone or email
survey might be considered.
4. What are the advantages and disadvantages of face-to-face data collection? Give
examples of the types of study in which face-to-face data collection is appropriate.
Face-to-face data collection has a number of advantages. The interviewer has the chance
to build rapport with the respondent, which can help achieve and maintain co-operation
and increase the quality of the data. Response rates can be relatively high in comparison
to other methods. Face-to-face methods allow for a relatively high degree of flexibility in
the interviewing process – the interview can last up to an hour; stimulus material can be
used; complex questions explained and administered; and probing and prompting carried
out. In central location or hall tests, the environment of the interview can be controlled.
There are some disadvantages to face-to-face data collection, particularly in relation to
in-home interviews. It is relatively expensive and time-consuming. Finding respondents
at home (or at work) at a suitable time (and willing to take part) can be difficult. To
overcome this, it is often necessary (and almost essential for business-to-business
interviews) to make an appointment with the respondent, either by telephone or in person
to set a suitable time. Interruptions from other members of the household or workforce or
the presence of someone else in the room during the interview can impact on the quality
of the data collected. It is difficult to cover remote or rural locations. Cluster sampling
methods, which serve to reduce travel time and costs, risk introducing sample bias.
Representativeness of the sample can be affected in other ways: interviewers may be
reluctant to interview in some (socially deprived) neighbourhoods; they may find it
2
difficult to obtain interviews in higher income areas; potential respondents in any
neighbourhood may be unwilling to open the door to a stranger. There is a greater
tendency in face-to-face methods for the respondent to give socially desirable responses –
showing him – or herself in the best possible light. With quality control procedures at
more of a distance than in telephone interviewing (where interviewers’ work can be
continuously monitored), there is greater scope for interviewer bias or cheating.
If you need a quota sample of shoppers, the interview is about product preferences and is
likely to last no more than about 10 minutes, face-to-face interviews recruited and
conducted in the street or in a central location such as a shopping centre may be
appropriate. If you are conducting a random sample survey on household spending that
lasts up to 35 minutes, a face-to-face, in-home interview may be appropriate. If you have
stimulus material to show, for example, in an advertising pre-test, a face-to-face
interview may be the only option.
5. What are the advantages and disadvantages of data collection by telephone? For what
types of research is the method particularly suitable?
Telephone interviewing, and especially CATI, has a number of advantages over face-toface methods. A geographically dispersed sample (including those in remote and rural
areas) can be obtained more easily. It may be the only way of reaching some populations
– for example, the business community. It is possible to use a pure random sampling
approach rather than the cluster sampling approach that is common in face-to-face
methods, thus reducing the chance of sampling error, and all at a greatly reduced cost
because interviewer travel time is not an issue. A telephone survey may also make it
easier to reach a wider spectrum of respondents – for example, in socially deprived areas
where interviewers may be reluctant to work and in higher income areas where access to
homes may be difficult. It is possible to digitally record answers to open-ended questions
in full. Greater quality control is possible (and so cheating is minimised), with
interviewers being monitored ‘live’ rather than after the event, via back-checking (and
clients and research executives can listen in, enabling them to get a feel for the findings).
It is relatively easy to monitor interview length and the time taken for individual
questions – this can facilitate a dynamic or rolling pilot study and questions can be
altered if necessary. It is also relatively easy to determine the strike rate and refusal rate
and so monitor the sample and control quotas. Call-backs are easily managed so that bias
towards those more often at home is reduced. Centralised, face-to-face briefings are more
common than face-to-face projects and supervisors and executives may be on-hand to
answer queries during fieldwork. Multi-country studies can be conducted from a central
telephone unit enabling greater control over administration and increased consistency.
Telephone interviews offer respondents a greater degree of perceived anonymity than do
face-to-face interviews. As a result, it is a useful method for collecting data on sensitive
or intimate subjects, and it is useful in reducing social desirability bias. Telephone
interviewing is faster than face-to-face – more questions can be asked in a shorter period
of time and project turnaround times are faster. It can therefore be more cost-effective
than face-to-face interviewing.
3
There are some disadvantages with this method of data collection. Although fixed line
telephone ownership in the UK is almost universal this is not the case in other parts of the
world – even among EU Member States. Those who do not have a fixed line telephone
are different from those who do – they are more likely to be from lower income
households and they tend to be older. This has implications for obtaining a truly random
sample of these populations – not everyone has the same chance of being selected and not
all of them are in the sampling frame or universe. This problem – of sampling error and
bias – is exacerbated if the telephone directory is used as a sampling frame. Not all
telephone numbers (and so individuals and households or businesses) are listed. In
addition, the problem is further exacerbated by the fact that many people (particularly in
younger age groups) have mobile rather than fixed line phones. The incidence of
telephone answering machines, call screening, lines being used to access the Internet and
more than one line per household add to the problems of access and sampling. The rise
in telephone sales means that response rates can make people suspicious of bona fide
telephone research.
In a telephone interview some of the benefits of social interaction and the chance to build
rapport with a respondent are lost. It can be easier for the respondent to refuse an
interview or end it early and harder for the interviewer to encourage the respondent to
take part. It is difficult to include stimulus material, although this can be overcome by
sending material out to respondents in advance of the interview. Long and complex
questions are best avoided.
6. Describe what is meant by the term ‘computer-aided interviewing’ and its advantages
and disadvantages over the traditional pen and paper survey methods.
Computer-aided interviewing means that data are captured and recorded electronically on
a PC or laptop computer or handheld tablet or personal digital assistant (PDA). There are
several methods of computer-aided interviewing:
 CAPI – Computer Aided Personal Interviewing
 CATI – Computer Aided Telephone Interviewing
 CASI – Computer Aided Self Interviewing
 CAWI – Computer Aided Wireless Interviewing.
The questionnaire (and in some cases the stimulus material), data entry and data editing
and verification procedures are programmed into the computer. To start a new interview
the interviewer calls a new questionnaire up on screen and enters responses into the
computer using the keyboard or touch screen. The program is designed so that it
automatically brings up the next question relevant to that respondent as soon as the
response to the previous question is entered. All completed questionnaires are stored in
the computer’s memory and can be downloaded via modem or wireless connection to a
central computer at the agency’s data processing centre.
There are many advantages in using computer-aided methods:
Fieldwork management
 No printing of questionnaires required
 Review of quotas and sample
4



Monitoring of interviewer performance and strike rates
Data on interview length
Electronic access to all questionnaires
Use of stimulus material


High quality images embedded in the questionnaire
Multi-media capabilities – can embed and play high quality video and audio material
Data processing




Checking and editing done automatically
No manual data entry – data downloaded via modem or wireless connection to central
processing
Interim results easily obtainable
Fieldwork to tabulation time greatly reduced
Data quality




Smooth flow of interview
Digital recording of verbatim responses
Automatic routing reduces errors
No separate data entry eliminates keying errors
The main disadvantage is the capital investment required to buy and maintain the
equipment and the software (although this is becoming less of an issue as devices become
smaller and cheaper) and to train and support interviewers and other staff to use it.
7. For what types of study is data collection via the web or email particularly suitable?
Give reasons.
Web and email data collection methods are particularly useful in business-to-business
(B2B) and employee research – Internet access is almost universal among medium and
large organisations in the US and Europe. They are also effective methods in popular
online business-to-consumer markets such as financial services, retailing (especially
food, books and music) and travel services. For research among the general public,
where sampling is more difficult and response rates poorer, online panels are the most
popular approach.
They are useful if the target population is widely dispersed (and where different
languages may be used), and in situations where the population may not be amenable to
research by other methods. They are an effective way of asking questions that need time
for consideration or involve the respondent in checking or consulting documents. They
are also effective in collecting data on sensitive topics and for reducing the risk of social
desirability bias as they offer respondents a high degree of perceived anonymity.
Large-scale surveys are relatively easy to handle using web and email methods.
Response rates can be monitored easily – for example, there is automatic notification if
an email is undelivered. Turnaround times from end of fieldwork to production of tables
5
are fast compared to postal surveys: data (including verbatim responses) are captured
directly, which also reduces data processing errors. Multimedia stimulus material
(pictures, audio and video clips) can be used.
8. What are the advantages and disadvantages in using a postal survey?
There are a number of advantages in using a postal survey:
 It is relatively easy to set up and manage
 It can be cost-effective as there are no interviewers to pay
 With no interviewer there is no risk of interviewer bias or error
 A widely dispersed sample can be reached
 Questions can be asked that need considered answers
 Questions on sensitive topics can be asked
There are also a number of disadvantages:
 Response rates can be poor, which means there may be problems with
representativeness
 You need a sampling frame that is accurate, up-to-date and easy to use
 There is a lack of control over data capture
 It is not good for collecting data via open-ended or complicated questions
 It is not always the most cost effective route
 There may be concerns about data quality, for example:
 the respondent can consult with others before answering the questions
 respondents may not answer all the questions they were supposed to or in the way
required
 you may get little detail at open-ended questions
 there is no opportunity to probe or clarify answers – you must accept the response
written in by the respondent
 questions requiring spontaneous answers do not work well
 the respondent can skip ahead or indeed read the whole questionnaire before
filling it in so any ‘funnelling’ of questions and topics does not work
 there is no opportunity to observe, for example, or to read body language or hear
tone of voice.
9. What steps would you take to ensure that you get a reasonable response rate from:
(a) a postal survey?
Ensure that:
 the nature of the research and the topic is suited to this method of delivery
 the topic is relevant and of interest to the target population
 the method is a suitable way of reaching and achieving a response from the target
population
 the questionnaire is well-designed – clear and easy to follow – and presented in a
professional manner.
Notifying the respondent in advance of the survey, ensuring that all correspondence with
the respondent is personalised, including the covering letter that accompanies the
questionnaire, can encourage response. Including details of the client or funder on
6
material sent to the respondent – in effect, sponsorship of the survey – can help, as can
reminders to respond, provision of a return envelope and an incentive (monetary or
otherwise).
(b) an email survey?
The steps taken to ensure a reasonable response rate to a postal survey apply here too.
Pre-notification is important – email questionnaires sent to respondents who have not
agreed to take part may be rejected, and the sender stopped from sending others. The
sample may be notified about the survey by email or by telephone or post. Use of a
covering letter or email is also good practice. Interview length should be kept to about 15
minutes, and, as with all self-completion methods, the questionnaire should look good
and be easy to fill in.
10. Describe what happens in a mystery shopping exercise.
A mystery shopping exercise involves a trained observer posing as an everyday shopper.
He or she goes through the shopping experience, asking the sort of questions a real
customer might ask. As soon as the mystery shop is complete, the mystery shopper fills
in the details of his or her experience on a questionnaire. For a personal visit the
information recorded might include, for example, length of time in the queue; number of
service points or tills open out of the total available; details of the greeting and exchange
with the member of staff; handling of questions; information or advice offered and so.
For a telephone mystery shopping exercise the information recorded might include
number of rings before the phone was answered, length of time on hold as well as
information on the exchange between shopper and staff member.
7