Independent Studies Resource 3:
Data Collection Methods
Dr Jill Hanson
N509
J.Hanson@derby.ac.uk
Resource Content





Secondary data
Observation
Interviews
Focus groups
Questionnaires/tests
Learning Outcomes


The student should be able to describe and
evaluate the main approaches to collecting
data
The student should be able to decide which
collection method will work best for a given
research objective
Secondary Data


Although your project must involve some
primary data collection, you can also make
use of secondary data.
What is secondary data?
Documentary data
 Survey-based data
 Multiple source data
(Saunders et al)

Secondary Data
Locating secondary data?

1.
2.
Availability (e.g. check references to sources of
data in articles/books, indexes and catalogues)
Actually finding it (Published data – track
through reference and then find library that
holds it using the ASLIB directory, for in-house
data contact information/data manager, can try
internet but be mindful of reliability of source)
Secondary Data: Advantages and
Disadvantages?

Advantages?

Disadvantages?
Evaluating Secondary Data
Assess overall suitability
1.


Assess precise suitability for analyses required
to meet each objective
2.


3.
Measurement validity
Coverage (including unmeasured variables)
Validity and reliability
Measurement bias
Cost/benefit analysis (in comparison to
gathering primary data)
Observation

“The systematic observation, recording,
description, analysis and interpretation of
people’s behaviour.” (Saunders et al, 1997, p.
186)


Participant observation (qualitative: the meaning
of actions)
Structured observation (quantitative: the
frequency of actions)
Observer roles

1.
2.
3.
4.

Gill & Johnson (1991)
Complete participant
Complete Observer
Observer as participant
Participant as observer
Ethical considerations?
Data Collection

Can take the form of
 Diaries
 Informal interview scenarios
 Could you record these
observations?
Threats to validity and reliability





Good ecological validity but that means less control
over confounding variables
Observer bias (can be combated using informant
verification)
Subject error (subjects not truly representative of the
norm)
Time error (behaviour at time data collected must
also be typical)
Observer error (subject behaves differently because
they are being observed)
Types of Interview
Structured v Semi-structured v
unstructured
 Standardised v non-standardised
 Respondent v Informant

Threats to Reliability and Validity

Reliability of non-standardised interviews
(would an interview on another day/with
another person yield the same data?)


Interviewer bias
Response bias
Interviewing Skills






Opening the interview and developing trust and rapport (don’t
start with highly personal/sensitive questions)
Asking the right questions (closed v open ended questions,
probes)
Asking the questions in the right way (to avoid bias – use neutral
tone and phrase them clearly)
Control your verbal and non-verbal behaviour to reduce bias
Active listening
Recording the data properly
Other Issues To Be Aware Of
 TIME!
Your time (to conduct and
transcribe – this takes AGES)
 The interviewees time (interviews
can run into hours of their time –
are they likely to be prepared to
do this?)

Group Interviews (Focus Groups)







In a group interview the researcher acts to facilitate and manage
the discussion AND the group and the group processes
Interview is likely to be unstructured
Group dynamic can reveal new themes
But can also inhibit shy people
Need to manage dominant people and allow everyone to
contribute
Consider location and setting (promote relaxation so not the
managers office)
Often useful to have help – 1 facilitates and manages, the other
takes notes
Questionnaires



Applicable with the experiment, quasi-experiments,
surveys, case studies and descriptive or explanatory
work
Efficient way to collect data from a LARGE sample
BUT you must ensure it will actually enable you to
gather the exact data you actually need before you
send it out…(check your data requirements table
carefully and make sure you are clear how to
analyse it)
How To Administer?

Face to face

Over the telephone

Via email

Post
Questionnaire Design

You should not design your own
questionnaire unless you absolutely have to!



The questionnaire you do use must be shown to
be valid and reliable
If you write your own you will have to do this
BEFORE you collect any data with it (pilot)
Therefore usually easier and better to adopt or
adapt an existing questionnaire
Where would I find such a questionnaire?

The questionnaire you use should be based on the theory/model
you are using as your theoretical framework.
 e.g. You are examining the psychological contract and its use as
a performance management vehicle.
 You are using Guest & Conway’s model of the psychological
contract
 Therefore you use Guest and Conway’s questionnaire
 Look at journal articles to find the printed questionnaire
 Or maybe contact authors direct
 Some of these questionnaires are copyrighted/owned by test
publishers so you need to pay for them
Research Design Requirements
Plan carefully

What variables you need to measure (independent,
dependent and extraneous)
What relationships you think might exist
What form you want the data to take for each of these
variables
1.
2.
3.

Ordinal, nominal or interval?
Exactly what analyses you will conduct.
4.



Influenced by form of data
Start with descriptive statistics (and I do not mean
pages of pie charts)
At Masters you will almost certainly need to conduct
inferential statistics as well (but using SPSS makes this
quite easy)
Return To The Literature

Before you put your questionnaire together
refer back to the literature

What theory are you using as a framework?


Usually a theory has a standardised questionnaire which
you can use to assess the relevant variables
Or what instruments have similar published studies
used? Use the same instrument so your findings can be
compared to theirs
Closed Questions
The questionnaire lends itself best to closed
questions:

1.
2.
3.
4.
5.
Lists (where respondent is offered a list of items, any of
which can be selected – but think how you will analyse
this kind of data)
Categorical (respondent selects one category – reduces
accuracy of data)
Ranking (respondent ranks choices – again think how
you will analyse)
Scale (respondent indicates preference on scale e.g.
Likert scale – easiest to analyse)
Quantity (respondent gives a number)
Response Style and Data Forms
1.
2.
3.
4.
5.


Lists (nominal data)
Categorical (nominal data)
Ranking (ordinal data)
Scale (interval data)
Quantity (interval data)
The form your data takes directly controls what
kind of analyses you can do. Scale wins every
time.
Always better to ask participants to rate every
category rather than ask them to rank or select
Yes/No
Response Styles And Analyses




Lists and categorical response styles give the least
informative data and are most difficult to work with
Ranked data also limits the tests you can use but to
a lesser extent
Scale and quantity data is much richer in nature and
easier to work with
Recommend using likert response scales (1 to 5 or
1 to 7) rather than check boxes so you can run
correlations and regression analyses.
Other considerations





Font, size and spacing
Order and flow of questions (lead them in
gradually, warm them up, don’t jump straight to
difficult or personal questions)
Don’t ask for information if you don’t actually need
it (e.g. gender/age – this goes to the ethical
considerations of anonymity and confidentiality)
Provide definitions of terms so that the respondent
interprets your questions in the way you want them
to
Explicit advice on how they should respond
Avoiding common problems in question
wording











Does your question collect detail at the right level? (check data
requirements table)
Will respondents have the knowledge required to answer it?
Will respondents be familiar with the terminology and interpret it in
the way you want them to?
Are there any words which might cause offence or will the question
cause embarrassment?
Can your question be shortened?
Are you asking more than question at a time? (don’t)
Does your question include a negative or double negative?
Is your question unambiguous? (be sure to make it clear what exact
response is required)
Does your question imply that a certain response is correct?
Does your question prevent certain answers from being given?
Have you incorporated advice on how to answer each question (e.g.
circle a number, tick ONE box etc)
Covering Letter

A requirement on ethical grounds. Must provide:
 Who you are
 Why you are doing the research/who you are representing/why it
is important
 Why respondent is important and how long it will take to complete
 Promises of confidentiality and anonymity
 How long/where data will be kept, who will have access to it and
what will be done with findings
 Whom to contact in the event of any queries
 Right to withdraw at any time/refuse to answer questions
 Thank you to the respondent
 Your signature followed by name and title
Introducing The Questionnaire



Explain clearly and succinctly why you want
the respondent to complete the questionnaire
Include a clear title and possibly a subtitle
Include return name and address at end of
questionnaire
Pilot Testing

Once you have written your questionnaire



Ask a small sample of people to complete it
Examine their responses to see if they interpreted it in
the way you wanted them to and if they answered it
correctly
Analyse scale data using SPSS to check that your scales
measure what they are supposed to



Internal consistency
Construct validity
See why it is easier to use someone else’s?
Summary

The two most likely data collection methods you will
use are





Questionnaire
Interview (with individuals or groups)
Neither are easy to construct
You must check them for reliability and validity
before you use them with your sample
You must ensure that ALL the data you need to
collect can be obtained using the method you have
devised (refer back to detailed data requirements
table)