Discussion Questions 3
Analyzing Qualitative Data
How do you analyze qualitative
data?

Data reduction
Data display
Drawing and verifying conclusions

Give examples from Griggs Article.


Data Reduction:
Organizing data to make it





more meaningful
Summarizing and paraphrasing
Selecting some themes and exluding
others
Putting specific instances into larger
patterns
Quantification into numbers and ranks
(coding into categories)
Look for differences across groups
Data Display Forms





Narrative text and quotations
Tables, matrices and graphs
Checklists
Always arrange in a systematic way
Lets look at:
• Example 1: Tables 1 and 2 (p. 24-25)
• Example 2: Table 3 (p.26)
• Example 3: Figure 2 (p.28)
Drawing Conclusions




Remember the research question and
note for regularities and patterns
Derive explanations
Draw causal flows
Come up with propositions
Verifying Conclusions

Verifying is checking results by:
• Triangulation for validity
• Use different measures for the same sample
• Use different samples and look for common results
• Get feedback from respondents
• Compare two set of respondents known to
•
•
differ
Do not ignore the outliers
Replicate findings ina second study
How do you evaluate validity and
reliability with qualitative data?

Validity:
Meaningfulness of
results
•
•
•
•
•
•
Apparent
Internal
Instrumental
Theoretical
Construct
Consultative
Skyes Article

Reliability:
Consistency of
results
•
•
•
•
•
•
Two different
researchers?
The same researcher?
Transferability
Confirmability
Dependability
Interrater agreement
methods
Content Analysis



What is it?
What is the unit of analysis?
How is it done? Procedure?
• Coders?
• Judges?
How to assess reliability in content
analysis?
Zimmer and Golden Article

Content Analysis Methodology


Formulate the research question
Select the unit of analysis (content):
•
•



Single word, theme, symbol
Fiction, movie, drama, TV program
Entire article, book, etc.
Code the content
Quantify the coded data
Check reliability by:
•
•
Ability to form categories
Extent of agreement between/among coders

Analyze and interpret the findings

Holsti, O.R. (1969), Content analysis for Social Sciences and Humanities,
Wesley Publishing Co.
Content Analysis Methodology



Sample: 894 respondents out of 1600
people selected as representative
population
Research question:
•
Please describe your image of ___________?
Stages:
•
•
•
•
Stage 1: Content Analysis of Themes
Stage 2: Aggregation of Themes
Stage 3: Selection of Final Categories
Stage 4: Reliability Analysis
Methodology: Stages in Content Analysis
Reliability
Reliability Analysis 2:
Researchers reduced 47 dimensions to 10. Inter-rater reliabilities of
all 3 pairs of judges were again calculated and found to be significant
Judges
# of
matching
(Out of 220)
Percent
Match
A&B
A&C
B&C
132
137
161
59.73 %
61.99 %
72.85 %
Reliability
Stage 4:
Reliability Analysis 1:
Three judges independently coded 220 themes into 47 dimensions and
the probability that any two judges will assign any one theme to the
same category by chance alone is found to be extremely small using
binomial probability distribution
Judges
# of matching
(Out of 220)
Percent Match
A&B
90
40.72 %
A&C
107
48.42 %
B&C
119
53.85 %
Testing the significance of Judge Agreements:
Interjudge reliability

Binomial Test of proportions (z-Test)

Cohen’s Kappa

Several Others
• Appendix of Zimmer and Golden Article
• Reading will be provided
• Wiil be discussed as we proceed in the
course
Cohen’s Kappa Measure
J1
A
B
C
D
E
Total
A
8
0
0
0
0
8
B
2
21
1
0
0
24
C
1
0
18
2
1
22
D
0
0
0
24
6
30
E
1
0
0
1
14
16
Total
12
21
19
27
21
100
How to calculate Cohen’s Kappa





Probability of Observed Agreements
Probability of Expected Agreements
Take the difference
Divide by expected agreements
Formula:
• Kappa: PO-PE
PE