Carol J. Pierce Colfer
Center for International Forestry Research
&
Cornell International Institute for Food, Agriculture and
Development
 Why do qualitative M&E?
 Three Examples of M&E in action:
1.
Quilcene, WA, educational research
2.
Global comparative research on assessment of
sustainable forest management
3.
Global comparative action research with local
communities
 Some Dangers and Conclusions
 Need for holistic understanding of a situation & how it
has changed.
 An unexpected finding emerged that hadn’t been
measured initially.
 An external, post-facto analysis is demanded, on issues
not initially assessed.
 The team has qualitative, not quantitative skills.
[Typically] involves
 Participant observation
 Long term residence in the research
context
 A holistic, inductive, open-ended
orientation
 Varying degrees of independence from
project being evaluated
 10 US rural schools, given grants to experiment with
their local schools
 External, long term M&E, both qualitative (fieldwork)
& quantitative (cross-site)
---------------------------------------------------------------Field researchers lived in communities for ~3 years,
documenting what happened and helping with crosssite studies – teasing out what went right, what went
wrong.
A central idea has been that C&I can be used to monitor,
assess, & even define a subject of interest.
Hierarchy of Principles, Criteria, Indicators, Verifiers
Ideal ones are SMART (Specific, Measurable,
Achievable, Realistic, Timely)
BUT some topics are difficult (impossible?) to
quantify:
1.
2.
3.
4.
Greater self-confidence among women & other
marginalized groups
Improved knowledge of regulations among
groups previously uninvolved
Involvement in enforcing sanctions, by a broader
spectrum of stakeholders
Closer links between communities & outsiders
(government officials, industry, projects,
academics)
Aim was to develop widely agreed-upon C&I to
define SFM, & for use in monitoring & assessing it
(initially, in certification of timber).
Series of 1-month, interdisciplinary, international
field visits to compare & hone existing sets of C&I
that would work in each country studied, using a
series of filtering steps (described in CIFOR
Toolbox No. 1).
A long term, learning-based approach involving
(facilitated) community groups
 identifying shared, future goals
 Analyzing, planning, & implementing what
is needed to reach those goals
 Monitoring progress & revising plans
accordingly
 Linking productively with relevant
external actors
Facilitator/researchers worked with
communities in 11+ countries (vertically,
horizontally & iteratively)
Teams assessed community progress in ways
that worked in their contexts:
 C&I that local people developed
 repeated ‘reflection’ meetings
 use of ethnographic observations
 Ongoing M&E by community members – are we
reaching our community/group goals?
 M&E by researcher/facilitators---to what degree is
ACM actually empowering, enriching people, &/or
enhancing their well being or environments?
 Result: A complicated life for
researcher/facilitators; a qualitative, cross-site
assessment, examining/comparing site
experiences (7 dimension framework)
 Getting too complex - e.g., Landscape Mosaic
project’s 4 levels of monitoring
 Producing something so holistic, ‘deep’ and long
that no one will ever read it (cf. quantitative baseline
surveys so long & complex that data never get entered,
let alone analyzed).
 [as with any method] Being swayed by your own
ideological biases, or someone’s (donors’,
employers’) desire for evidence of success
 Conducting qualitative M&E can be an uphill battle –
donors, policymakers, & many researchers prefer
quantitative assessments
BUT
 Qualitative M&E can often provide valuable insights,
unexpected findings, not available with conventional
quantitative approaches.
IDEALLY: Combine the two!