Ripple Effects Mapping:
A Participatory Strategy for Measuring
Program Impacts
Presented at the 21st Annual MESI Spring Training
Conference
Scott Chazdon, Ph.D., Evaluation and Research Specialist
Extension Center for Community Vitality
© 2011 Regents of the University of Minnesota. All rights reserved.
Session Overview
 Background – 20 minutes
 Demonstration – 40 minutes
Want to try mind mapping? Download from www.xmind.net
 Discussion – 45 minutes
© 2011 Regents of the University of Minnesota. All rights reserved.
Session Learning Objectives
 Describe the elements of the Ripple Effects Mapping
process.
 Identify the skills needed to facilitate a Ripple Effects
Mapping session.
 Weigh the benefits and limitations of Ripple Effects
Mapping as an evaluation tool.
 Assess the usefulness of Ripple Effects Mapping as a
tool to evaluate programs in your organization.
© 2011 Regents of the University of Minnesota. All rights reserved.
Ripple Effect Mapping
 Purpose – to better understand intended
and unintended results of a program,
intervention or collaborative for individuals,
groups, sectors or communities.
 Can be post-program (more summative) or
mid-program (more developmental)
© 2011 Regents of the University of Minnesota. All rights reserved.
Direct vs. Indirect Impacts
 Collaboratives and high engagement
programs often build social capital, but
don’t take credit for it.
 People do not act in isolation -strengthened social capital is a necessary
pre-condition for other impacts
 Other impacts may occur that were not
foreseen in program theory
© 2011 Regents of the University of Minnesota. All rights reserved.
The Core Ingredients of REM
 Appreciative Inquiry
 Participatory Approach
 Interactive Group Interviewing
and Reflection
 Radiant Thinking
(aka Mind Mapping)
© 2011 Regents of the University of Minnesota. All rights reserved.
Mind Mapping – Radiant Thinking
Pictorial Method
– Note taking
– Brainstorming
– Organizing
– Problem
solving
– Evaluation
For more on mind mapping, see Buzan, T. (2003). The mind map book. London: BBC Books.
Image: Mindmap, Graham Burnett,
http://en.wikipedia.org/wiki/File:Mindmap.gif
Ripple effect mapping:
related approaches
 Outcome Mapping (Outcome Mapping Learning Community, 2011)
 Participatory Impact Pathway Analysis
(Douthwaite et al, 2008)
 Outcome Harvesting (Wilson Grau & Britt, 2012)
 Most Significant Change (Davies 2005)
 Appreciative Inquiry
© 2011 Regents of the University of Minnesota. All rights reserved.
(Preskill & Catsambas, 2006)
How Does it Work?
 Identify the intervention
 Schedule the event and invite
participants
 Group mapping session held
 Follow-up interviews
 Cleaning, Coding, Analysis
© 2011 Regents of the University of Minnesota. All rights reserved.
RIPPLE EFFECT MAPPING
Method
– Identify the intervention
 High engagement program or position
 Cross-sector initiative
 Collaboration
– Invite stakeholder group
 Participants
 Non-participant stakeholders
 12 to 20 participants
 Two moderators
© 2011 Regents of the University of Minnesota. All rights reserved.
RIPPLE EFFECT MAPPING
Method
– Appreciative Inquiry interview
 Conducted among pairs of participants
 Examples of questions:
– Tell me a story about how you have used the information from the
program?
– Is there anything that resulting from the program that you are
proud to share?
– List an achievement or a success you had based on what you
learned.
© 2011 Regents of the University of Minnesota. All rights reserved.
RIPPLE EFFECT MAPPING
Method
– Starting the Map
 On wall or using Mind Mapping software with data
projector
 Floating topics generated from Appreciative Inquiry
– Different approaches
© 2011 Regents of the University of Minnesota. All rights reserved.
THEMING AND RIPPLING
- Building the Map
 Theming
- Group identifies which items are closely related
- Group generates initial theme names
- Floating topics moved and organized
 Rippling
- Group cross-validation
- Potential for probing using the Community Capitals
Framework
© 2011 Regents of the University of Minnesota. All rights reserved.
THREE APPROACHES
Web mapping
Use of Community
Capitals
Core focus of group
time
In-depth rippling
Theming and
rippling
Used in the mapping process Used to analyze data after the Used to analyze data after the
group session
group session
Develop a deep and rich
visualization of the causal
chains that led to impacts, with
an understanding of how
impacts in one capital
influence changes in the other
capitals.
Develop a deep and rich
visualization of the causal
chains of events that led to
impacts.
Develop a broad
understanding of core impact
themes as well as some
visualization of causal chains
of events that led to impacts.
Flow of session
1. Appreciative Inquiry
2. Reporting and mapping onto
capitals framework of short
term , medium term, and long
term outcomes
1. Appreciative Inquiry
2. Reporting and rippling
discussion
1. Appreciative Inquiry
2. Reporting
3. Theming
4. Ripple discussion
5. Negative effects discussion
Data analysis
Quantify the number of assets
developed by capital.
Use of technology
1. Coding of data based on
CCF
2. Quantify the number of
assets developed by capital
Ripple map is captured on
Ripple map is captured on
butcher paper and later typed butcher paper and later typed
into mind mapping software
into mind mapping software
1. Coding of data based on
CCF
2. Quantify the number of
assets developed by capital
Ripple map is typed directly
into mind mapping software
and project on screen
Emery, M., Higgins, L., Chazdon, S., and Hansen, D. (2015). Using Ripple Effect Mapping to evaluate program impact:
Choosing or combining the methods that work best for you. Journal of Extension 53(2). Available for download at
http://www.joe.org/joe/2015april/tt1.php
RIPPLE EFFECT MAPPING
Examples of Contexts where REM
makes sense
 Community gardening
 Community leadership development
 Coalition efforts to improve community health
outcomes
 Child care systems change
© 2011 Regents of the University of Minnesota. All rights reserved.
RIPPLE EFFECT MAPPING
Demonstration of Mapping Process
 Think back to your first program evaluation
experience.
– Is there anything that you gained from this experience that you
are proud to share?
– List an achievement or a success you had based on what you
learned or who you met.
– Did the experience lead to other important developments in your
personal or professional life?
 Floating topics
 Beginning to categorize
© 2011 Regents of the University of Minnesota. All rights reserved.
Example: Community Gardening in Frogtown & Rondo
neighborhoods
Example: Community Gardening in Frogtown & Rondo
neighborhoods
Example: Hugo, MN Business Retention and
Expansion program
© 2011 Regents of the University of Minnesota. All rights reserved.
Example: CYFC Scholars Program
RIPPLE EFFECT MAPPING
Cleaning, Coding, Analysis
– Organize map to better identify pathways or
combine pathways
– Download data to Excel for coding
– Code using relevant thematic framework and
type of outcome



KASA = something learned
Behavior change = action taken
Impact = change in system
– Follow-up interviews if more clarity is needed
© 2011 Regents of the University of Minnesota. All rights reserved.
The Community Capitals Framework
(Emery and Flora, 2008)
RIPPLE EFFECT MAPPING
Coding Demonstration
© 2011 Regents of the University of Minnesota. All rights reserved.
Coding Example
First
order Second
(core
order
outputs) ripples
Third
order
Fourth
order
Human capital
effects
(knowledge
and behavior Social capital
change)
effects
Civic effects
Financial
effects
X
X
Built capital
effects
Health, Food
and Nutrition
Effects
Cultural
effects
Natural
environment
effects
Market the City of Hugo
City identity workshop -
How to attract residents
and biz (coninuing work)
Have identified key attributes about
the City
X
X
X
X
X
Create, Coordinate, and Encourage Events
New position at City for park & rec.
planning
X
X
~10 new recreation
programs
X
X
Hanifl Fields attracted
over 20,000 kids
X
X
Entrepreneurial Bootcamp
X
Businesses have used City resources
Provide promotion opps. for biz
Coupons at football
tourney
X
X
X
?
X
X
X
?
Reporting Example (Hugo BR&E program)
Category of
Community
Program Effects
Human capital
effects (knowledge
and behavior
change)
Social capital
effects
Civic effects (aka
Political)
Financial effects
Built capital effects
Count of
reported effects Percent of
(out of a total of 41 reported
Definitions of Categories effects reported)
effects
Changes in knowledge,
attitudes, or skills among
25
61.0%
community members.
Strengthened or expanded
connections among people,
groups and organizations.
Increased ability of
communities to access and
mobilize public resources.
Increased private and public
wealth that is invested in the
well-being of communities.
Improvement of structures
and infrastructures that
contribute to the well-being of
communities.
12
29.3%
20
48.8%
11
26.8%
7
17.1%
Examples
1. Increased awareness of how many and
what kind of businesses are in the
community 2. City staffers got to know
businesses
1. Hugo Business Association is gathering
more often with a purpose. 2. City and
business groups are interacting more.
1. Created and filled a park & recreation
planning position 2. City is still using a biz
resource guide that was created in the
BR&E program to assist businesses.
1. Xcel Energy creating a training center
with 10 jobs and lots of visiting trainees. 2.
Provide promotion opps. for biz 3. City
actively recruiting events to be held in Hugo
Removed blighted buildings using 10 year,
interest free loans with businesses.
Reporting Example (Tourism Assessment Program)
Community A
Capital
# of
outcomes
% of
reported
outcomes
Cultural
22
Financial
Community B
Community C
# of
outcomes
% of
reported
outcomes
# of
outcomes
% of
reported
outcomes
31.0%
9
11.3%
11
12.2%
21
29.6%
27
33.8%
26
28.9%
Social
18
25.4%
22
27.5%
27
30.0%
Political
16
22.5%
9
11.3%
11
12.2%
Built
15
21.1%
14
17.5%
17
18.9%
Human
13
18.3%
18
22.5%
8
8.9%
Health
6
8.5%
6
7.5%
7
7.8%
Natural
0
0.0%
4
5.0%
7
7.8%
Total number of
Outcomes (unduplicated)
71
80
90
Reporting Example
Reporting Example (Wilder Research)
Skill Set Needed
 Facilitation/group process skills
 Interviewing and probing
 Technology and Typing (if you’re gonna do
live capture)
 Qualitative analysis
 Commitment to participatory evaluation
© 2011 Regents of the University of Minnesota. All rights reserved.
Benefits
 Simple and cheap tool
 Captures impacts of complex or evolving
work
 Captures intended and unintended
impacts
 Participatory and appreciative approach
that engages stakeholders
 Group validation of results
© 2011 Regents of the University of Minnesota. All rights reserved.
Limitations
 Risk of bias in participant selection and
data collection
 Participants may not have complete
information about a program or program
outcomes
 Potential for inconsistency in
implementation
© 2011 Regents of the University of Minnesota. All rights reserved.
Suggestions
 Use same facilitator, recorder and “mapper”
 Develop a facilitator guide with ideas for probes.
 Make decision prior to mapping whether to use a
pre-existing framework as probes during group
interviews
 May need to recognize that one organization
isn’t trying to take all credit for all change
 It is important to probe for negatives
© 2011 Regents of the University of Minnesota. All rights reserved.
Lessons Learned Thus Far
 Find the right balance between breadth and
depth
 Schedule the event along with another activity
 Put much effort into recruitment and explaining
the process
 Choose a good setting – not too informal
 Use external facilitators, not program staff
© 2011 Regents of the University of Minnesota. All rights reserved.
Q&A and Discussion
 How might your organization use this?
 What types of interventions would be
most amenable to this type of
evaluation?
 What concerns do you have about this
method?
© 2011 Regents of the University of Minnesota. All rights reserved.
Thank you!
Scott Chazdon, Ph.D.
Evaluation and Research Specialist
Center for Community Vitality
612-624-0982
schazdon@umn.edu
© 2011 Regents of the University of Minnesota. All rights reserved.
The University of Minnesota is an equal opportunity educator and employer. This PowerPoint is available in alternative formats
upon request. Direct requests to 612-625-8233.