Data Analysis
Analyzing Qualitative Data:
A Technical Note For Lead Impact Assessment Researchers
OVERVIEW
This technical note is aimed at lead researchers and their teams. The purpose of the note
is to serve as a reminder or refresher regarding some basics of good qualitative data
analysis. The note is neither toolbox, nor comprehensive introduction to data analysis,
nor training mechanism for novice researchers.
DEFINING ‘ANALYSIS’
More than likely, we all actually have different mental models of what “analysis” is and,
therefore, how it should be done. Two definitions are complementary and helpful. The
first is very general, applying to just about any thought process:
Analysis is the search for patterns in data and for ideas that help explain why those
patterns are there in the first place (Bernard 2002, 429).
The second ties this generic definition more tightly to research:
Data analysis consists of examining, categorizing, tabulating, testing, or otherwise
recombining…evidence to address the initial propositions of a study (Yin 2003,
109).
So, data analysis is about the identification of patterns, patterns that connect to the
questions being studied.
ANALYTIC STRATEGIES
The goal of an analytic strategy is “to treat the evidence fairly, produce compelling
analytic conclusions, and rule out alternative interpretations” (Yin 2003, 111). Much
impact assessment qualitative research will require “analytical induction.” This is a very
fancy phrase. Simply put, analytical induction is
a formal, qualitative method for building up causal explanations of phenomena from a
close examination of cases (Bernard 2002, 512).
Generic Analytical Strategies
There are three generic analytical strategies for you to chose. They are:
 Test a Theory
 Investigate rival explanations
 Develop your own theory or causal model
The table below summarizes them.
Approaches
Within any particular analytical strategy, there are three approaches that help you
identify and make sense of patterns in the data
 Causal Webs
 Time-Series Analysis
 Logic Models
They are summarized in the table below.
Analytic Strategies and General Approaches (adapted from Yin (2003, Chapter 5)
Strategy
Test a theory
Investigate rival
explanations
Develop your
own theory or
causal model
Strategic Intent
Follow your theory/hypotheses and use techniques that permit you to investigate the causal relationships you have proposed. Ideally,
your study design has been linked to a wider theory base, to existing questions in the literature, and so forth.
Establish rival explanations and interrogate your data with them. This strategy can be combined with the first. “Rival explanations”
can include ones embedded in the research process itself (validity threats, researcher bias) but can also look explicitly at the
possibility that other interventions can account for results observed (direct rival), that the researched intervention works in tandem
with other variables (commingled rival), an alternative theory can account for the same observations (rival theory), or that other
contextual factors produce the outcomes observed (social trends, for example).
Particularly when conducting open-ended, exploratory research, a guiding analytic strategy can be to actually let the empirical data
speak to you with the goal of developing a theory or model that explains observations. This is particularly common in studies that
rely on grounded theory techniques (Strauss & Corbin 1998)
Common Approaches
All three of the approaches below are forms of “pattern matching” or comparing an empirically observed pattern against some predicted or expected one.
All require looking at internal threats to validity; all require the identification, first, of patterns in the data…for which task you have scores of options of
specific techniques.
Causal Webs
Time-Series Analysis
Logic Models
Specify a linked web of multiple dependent
Analyzing the change over time between causes
Deliberately simulates a complex series of
variables that all should go together due to an
and effects against a theoretically derived
relationships over time. In essence, logic models
intervention or interventions. Rigor and
historical relationship specified prior to the onset
combine causal webs and time-series analysis.
persuasiveness is increased if your theory would
of the study. This goes far beyond simple
Frequently, logic models get produced in a
predict different outcomes in different contexts, you chronologies of events as the focus is
deeply iterative model between data gathering,
investigate those contexts, and you find this indeed demonstrating or refuting a hypothesized set of
analysis, returning to theory and hypotheses,
to be the case.
relationships
back to data gathering for testing, etc.
SPECIFIC TECHNIQUES
Data analysis is intended to be a participative process involving CARE staff, researchers,
and to the best of our ability women and men in the communities in which we are
working and doing impact assessment. We need, therefore, to pay attention to the
analysis techniques we use in terms of the skills, experience, and competencies that they
require. We need to pay attention to both:
a) Cross-cutting methodological principles and,
b) Specific analytic techniques
For more detail, see the full technical note. The note also contains guidance on achieving
rigor in qualitative analysis through:
1.
2.
3.
4.
Confirmability (objectivity);
Dependability/Auditability (reliability);
Credibility/Authenticity (internal validity);
Transferability/Fittingness (external validity).