Qualitative Data Collection Instruments
Research Rescue Lab
Dr. Kelley Wood
Business & Graduate Studies, Trinity Washington University
A Holistic View of the
Research Paper
Topic of
Inquiry
Literature
Survey
Research
Questions
Abstract
Introduction
Literature
Review
Research
Methods
Dr. Kelley D. Wood, Business & Graduate Studies
Results /
Findings
Discussion
2
Data Collection Instruments
• Literature Review
– This chapter builds the argument, through
logic, that is the basis of the study
• The result of this is the Theoretical Framework ,or
Theoretical Construct, or Conceptual Map
• Research Methods
– Determine the strategy for collecting data
– Develop the Data Collection Instrument
• Derived from the Quantitative Theoretical
Framework, or the Qualitative Theoretical
Construct, Conceptual Framework, or Literature
Map
Dr. Kelley D. Wood, Business & Graduate Studies
3
A Holistic View of the
Research Paper
• Literature Review
– Sections of the chapter
• Preamble
– A brief argument, moving from general to specific, that prepares
the reader for engaging in the topic areas you have determined
must be covered to support the Theoretical Framework or
Theoretical Construct
• Sections and subsections as necessary to the argument
– Case studies should include the subject of the case
• Review of other research methods
– Used in investigating similar topics
• Theoretical Framework or Theoretical Construct
– Describe and define the elements as they will be used in your
study
– Including a figure or model as a visual representation
Dr. Kelley D. Wood, Business & Graduate Studies
4
Data Collection Instruments
• Research Methods
– Preamble
– Setting
– Population/ Data Source
• Your potential participants
– Research Strategy/ Measurements
– Intervention Protocol
– Data Collection Instrument Protocol
• Observation, Interview, Questionnaire, Survey
• Secondary data, content or document analysis - How data
sources will be selected
– Data Analysis Strategy
– Summary
Dr. Kelley D. Wood, Business & Graduate Studies
5
Variable
or Factor
Variable or
Factor
Theory
Data Collection Instruments
Concept
Concept
Concept
Becomes the…
Concept
Concept
Concept
Dr. Kelley D. Wood, Business & Graduate Studies
6
Data Collection Instruments
Becomes the…
Question
Question
Question
Factor
Question
Dr. Kelley D. Wood, Business & Graduate Studies
Data Collection
Instrument
Question
Factor
Question
7
Variable
or Factor
Variable or
Factor
Concept
Question
Concept
Question
Concept
Question
Concept
Question
Concept
Question
Dr. Kelley D. Wood, Business & Graduate Studies
Data Collection
Instrument
Question
Factor
Concept
Factor
Theory
Data Collection Instruments
8
Data Collection Instruments
• Purpose of data collection instrument
– To test the hypotheses
– Clarify relationships between variables
(concepts)
– To test propositions
– To discover more about the phenomena of
interest
• Which is really to say we are testing the
theoretical framework, construct, or model
Dr. Kelley D. Wood, Business & Graduate Studies
9
Data Collection Instruments
• Interviews, Questionnaires, and Surveys
– Schensul, Schensul, & LeCompte (2009,
chapters 6, 7, & 8)
• Ethnography
– Process is predicated on the understanding
that nothing is really known of a new culture
to be studied
– Uses scientific methodology to develop a
greater understanding
Dr. Kelley D. Wood, Business & Graduate Studies
10
Data Collection Instruments
• Ethnography
– Process is predicated on the understanding that
nothing is really known of a new culture to be studied
– Moves from
•
•
•
•
Open-ended Observation
Open-ended Interviews
Semi-structured Interviews
Structured (directed) Surveys
– Follows the Mixed Methods pattern
• Qualitative to Quantitative
• Development of theory
• Testing of theory
Dr. Kelley D. Wood, Business & Graduate Studies
11
Data Collection Instruments
• Ethnography
– Process is predicated on the understanding that
nothing is really known of a new culture to be studied
– Follows the sequential mixed methods pattern
• Qualitative
– Development and testing of propositions
– Propositions = local or low level theories
• Testing
– Development of theory
• Quantitative
– Development of hypotheses
– Testing of hypothesis
Dr. Kelley D. Wood, Business & Graduate Studies
12
Data Collection Instruments
• Developing the instrument
– Follow the continuum of abstraction
• Observation
– Description of domains, macro-structural
features, cultural continuities
• Interviews
– Description of factors, sub-factors, identification
of cultural variation
• Questionnaire
– Refined description of factors, sub-factors,
identification of cultural variation
• Survey
– Identification of variables, testing of
associations and predictive models
Dr. Kelley D. Wood, Business & Graduate Studies
13
Data Collection Instruments
• Constructing the instrument
– Variable (concept) development
• Domain to Factors to Variables (concepts)
• Example
– Domain - Work
» Factor - Satisfaction with job
» Variables (concepts)
» Compensation, autonomy,
motivation, relationships –
coworkers and supervisors or
management
» Position, advancement, relevance,
decision making power, support
• Schensul, Schensul, & LeCompte (2009, pp.
174-176)
Dr. Kelley D. Wood, Business & Graduate Studies
14
Data Collection Instruments
Interviews
Dr. Kelley D. Wood, Business & Graduate Studies
15
Developing the Interview
• Open-ended interviews
– Schensul, Schensul, & LeCompte (1999, chapter 6)
• Purpose of the open-ended interview
–
–
–
–
Explore undefined domains
Identify new domains
Break known domains into factors and subfactors
Obtain contextual and historical information to orient the
research
– Build understanding and positive relationships with the study
population
– To understand the individual lived experiences of participants
– To understand participant’s meaning schemas
Dr. Kelley D. Wood, Business & Graduate Studies
16
Developing the Interview
• Think of the interview in the continuum of
ethnographic methodology
– Qualitative
• Open-ended observation
• Structured observation
– Observe for specific constructs, interactions, behaviors
• Open-ended interview
– Open-ended questions
• Semi-structured questionnaire
– Semi-directed questions
– Quantitative
• Survey
– Closed-ended questions
Dr. Kelley D. Wood, Business & Graduate Studies
17
Developing the Interview
• And the continuum of the data developed
– Qualitative
• Open-ended observation
– Determine initial constructs and domains
• Structured observation
– Refine domains and constructs
• Open-ended interview
– Deeper understanding of domains and constructs
– Determine factors that comprise the domains and constructs
– The factors are your Dependent and Independent Variables
• Semi-structured questionnaire
– Specific understanding of factors (variables)
– Determine subfactors that comprise the factors
– Quantitative
• Survey
– Measuring the subfactors
Dr. Kelley D. Wood, Business & Graduate Studies
18
Developing the Interview
• Interview questions relate to a factor
– Do not go as deep as the variable (concept)
– Develop data that relates to your research questions
• Purpose of the questions
– Open-ended questions
• Inspire a narrative account of the participant’s experience
• Inspire the participant to relate naturally
• Not to meet an expectation
• Applied to a limited population
Dr. Kelley D. Wood, Business & Graduate Studies
19
Developing the Interview
• Open-ended questions
– Represent the factors in the domain
– Inclusive of a factor or multiple factors
– Do not lead the response or express bias
•
Prepare probing and clarifying questions
–
–
–
–
–
–
Neutral agreement
Repeat what is said as a question
Ask for more information
Ask for their opinion
Ask for clarification of discrepancies
Ask for clarification of terms and acronyms
Dr. Kelley D. Wood, Business & Graduate Studies
20
EXERCISE 1
In the handout
Dr. Kelley D. Wood, Business & Graduate Studies
21
Data Collection Instruments
Questionnaires
Dr. Kelley D. Wood, Business & Graduate Studies
22
Developing the Questionnaire
• Open-ended and semi-structured interviews
– Schensul, Schensul, & LeCompte (1999, chapter 7)
• Purpose of the semi-structured interview
–
–
–
–
Clarify central domains and factors
Operationalize factors into variables (concepts)
Develop a preliminary hypothesis
Develop the qualitative base for a quantitative survey
Dr. Kelley D. Wood, Business & Graduate Studies
23
Developing the Questionnaire
• Think of the questionnaire in the continuum
of ethnographic methodology
– Qualitative
• Open-ended observation
• Structured observation
– Observe for specific constructs, interactions, behaviors
• Open-ended interview
– Open-ended questions
• Semi-structured questionnaire
– Semi-directed questions
– Quantitative
• Survey
– Closed-ended questions
Dr. Kelley D. Wood, Business & Graduate Studies
24
Developing the Questionnaire
•
And the continuum of the data developed
– Qualitative
• Open-ended observation
– Determine initial constructs and domains
• Structured observation
– Refine domains and constructs
• Open-ended interview
– Deeper understanding of domains and constructs
– Determine factors that comprise the domains and
constructs
– The factors are your Dependent and Independent Variables
• Semi-structured questionnaire
– Specific understanding of factors (variables)
– Determine subfactors that comprise the factors
– Quantitative
• Survey
– Measuring the subfactors
Dr. Kelley D. Wood, Business & Graduate Studies
25
Developing the Questionnaire
• Questions link back directly to a
variable (concept)
– Develop data that relates to your research questions
• Purpose of the questions
– Semi-structured questions
• Semi-directed
• Provide focus
• Not leading
– Question does not contain the expected answer
• Not closed
• Applied to a larger population with a limited interpretation
Dr. Kelley D. Wood, Business & Graduate Studies
26
Developing the Questionnaire
• Developing semi-structured questions
– Use terms and phrases that the participant
• Will understand
• Are appropriate to their context
– Keep the length minimal
– Avoid leading questions
– Avoid ‘two for one’ questions
Dr. Kelley D. Wood, Business & Graduate Studies
27
Developing the Questionnaire
• Developing semi-structured questions
– Avoid negatively phrased questions
– Avoid rank ordering
• Save for the survey
– Avoid questions that require directions or tasks to complete
– Avoid yes or no questions
– Be sensitive to cultural context or social meanings in the questions
Dr. Kelley D. Wood, Business & Graduate Studies
28
Developing the Questionnaire
• Developing semi-structured questions
– Ordering the semi-structured questions
• Temporally – use an arc of time
– Earliest to latest
• Complexity – use an arc of complexity
– Least to most complex
• Group by topic or domain
– Topically similar questions together
• Abstraction – use an arc of abstraction
– Most concrete to most abstract
• Threat level
– Least sensitive to most threatening
Dr. Kelley D. Wood, Business & Graduate Studies
29
Using both Open-ended and SemiStructured Questions
• Lead with open-ended questions
– Opens the participant up
– Refreshes their sense of the experience
• Transition
– Briefly review what you have heard to clarify
– Allow them to correct or modify
– Discuss purpose of the next section without giving too many
specifics
– The next section may ask the participant to repeat or clarify
• Move to semi-structured questions
• Transition
– Briefly review what you have heard to clarify
– Allow them to correct or modify
• Closing comments from participant
– Ask them if there is anything that they wished you asked
– Or that occurred to them that they would like to share
Dr. Kelley D. Wood, Business & Graduate Studies
30
Data Collection Instruments
• Research article showing a coding key
– Jehn, K. A. (1997) A qualitative analysis
of conflict types and dimensions in
organizational groups. Administrative
Quarterly 42(3) 530-557 retrieved from
ProQuest Social Sciences Journals
– While you are developing your
instruments
• Interview, Questionnaire, or Survey
• Develop a coding key
Dr. Kelley D. Wood, Business & Graduate Studies
31
EXERCISE 2
In the handout
Dr. Kelley D. Wood, Business & Graduate Studies
32
Data Collection Instruments
Surveys
Dr. Kelley D. Wood, Business & Graduate Studies
33
Developing the Survey
• Structured (directed) Surveys
– Schensul, Schensul, & LeCompte (1999, chapter 8)
• Purpose of the directed question
survey
– Clarify relationships between factors and
variables (concepts)
– Operationalize factors into variables (concepts)
– Test hypotheses
– Develop detailed , granular understanding of
factors
Dr. Kelley D. Wood, Business & Graduate Studies
34
Developing the Survey
• Think of the questionnaire in the continuum
of ethnographic methodology
– Qualitative
• Open-ended observation
• Structured observation
– Observe for specific constructs, interactions, behaviors
• Open-ended interview
– Open-ended questions
• Semi-structured questionnaire
– Semi-directed questions
– Quantitative
• Survey
– Closed-ended questions
Dr. Kelley D. Wood, Business & Graduate Studies
35
Developing the Survey
• And the continuum of the data developed
– Qualitative
• Open-ended observation
– Determine initial constructs and domains
• Structured observation
– Refine domains and constructs
• Open-ended interview
– Deeper understanding of domains and constructs
– Determine factors that comprise the domains and constructs
– The factors are your Dependent and Independent Variables
• Semi-structured questionnaire
– Specific understanding of factors (variables)
– Determine subfactors that comprise the factors
– Quantitative
• Survey
– Measuring the subfactors
Dr. Kelley D. Wood, Business & Graduate Studies
36
Developing the Survey
• Questions link back directly to a variable
(concept)
– Develop data that relates to your research questions
• Purpose of the questions
– Directed questions
• Provide measurement of behaviors at specific points in time
• Detailed examination of relationships between dependent and
independent domains (variables)
• Not leading
– Question does not contain the expected answer
• Closed response
• Larger random and generalizable population
Dr. Kelley D. Wood, Business & Graduate Studies
37
Developing the Survey
• Constructing the instrument
• Considerations
– Exhaustive – no surprise responses
– Mutually exclusive – response fits only one
option
– Avoid two or more nouns or verbs – you want
to have no confusion of subject or action
Dr. Kelley D. Wood, Business & Graduate Studies
38
Developing the Survey
• Constructing the instrument
– Variables (concepts) to questions
• Question or statement format
• Avoid open-ended questions
• Each items is stated clearly
• Mutually exclusive
• Avoid more than one noun or verb
Dr. Kelley D. Wood, Business & Graduate Studies
39
Developing the Survey
• Constructing the instrument
– Variables (concepts) to questions
• Can respondents answer the question
• Are questions relevant to the topic
• Questions are as short as possible
• Avoid alternating positive and negative questions
in the same scale
• Avoid items and terms that have hidden or
unclear meanings
Dr. Kelley D. Wood, Business & Graduate Studies
40
Developing the Survey
• Constructing the instrument
– Level of Measurement
• Nominal – number is assigned to represent
something as numbers are easier in statistical
programs
– Vary by quality but not quantity
– Country, gender, etcetera
• Ordinal – values convey order or rank only
– Ordered by ranking, not absolute
– May chose only one answer
• Interval – values have no true zero and are
equidistant
– Distance between ranked intervals, no zero start
point
– 100 to 120 = 90 to 110 (interval of 20)
• Ratio – has a true zero and values are equidistant
– Based on a zero input
Kelley D.
Wood, Business
& Graduate
Studies
– Age,Dr.
income,
numbers
of a variable
(units)
41
Developing the Survey
• Constructing the instrument
– Types of questions
• Fill in – correct answer
• Multiple choice – choose correct answer from
options
• Scaled – such as Likert, continuous (on a
continuum with decimals), or discrete (whole
numbers or units not on a continuum)
• Semantic Differential
– Choose from opposites
• Ranking
– Order (<=) 10 items against each other
• Rating
– Rate(<=) 10 items from 1 to 10
• Category
– Often a yes/no choice of distinct options
Dr. Kelley D. Wood, Business & Graduate Studies
42
Developing the Survey
• Scales – an aside
– Scales can easily convey bias or preferred
response
– Choosing a scale of measurement
• When using Median or Mean
• Order – does a larger number indicate a greater
value than a smaller number
– Ordinal, interval, ratio
• Differences – does subtracting two numbers
indicate a meaningful value
– Interval, ratio
• Ratio – Does dividing two numbers indicate a
meaningful value
– Ratio
• Use the same pattern for consistency –
parallelism
Dr. Kelley D. Wood, Business & Graduate Studies
43
Developing the Survey
• Scales – an aside
– Likert is not the only game in town
• The point of Likert is to cause data to be
easier to summarize around a central
tendency
– Median, Mode, Mean
• Using a 5 point scale should encourage the
response to be either positive or negative,
unless it is central
– Often encourages the selection of the central
value
– 12345
• Some use a 6 point Likert
– Middle two values are the central tendency
Dr. Kelley D. Wood, Business & Graduate Studies
– 123456
44
Developing the Survey
• Scales – an aside
– When is Central Tendency important, or
when to use Likert?
• Ordinal
– One sample sign test, Mann-Whitney U test,
Wilcoxon sign-ranks T test, Kruskal-Wallis H
test, Friedman test
• Interval/Ratio
– One-independent sample t-test, Twoindependent sample t-test, Related samples ttest, One-way between-subjects ANOVA, Oneway within-subjects ANOVA, Two-way
between-subjects ANOVA
Dr. Kelley D. Wood, Business & Graduate Studies
45
EXERCISE 3
In the handout
Dr. Kelley D. Wood, Business & Graduate Studies
46