AEA Think tank 2015
Developmental Evaluation approach to
Evaluating INGO Organizational Change
Skyway 265 -- Sat, Nov 14, 2015 (09:45 AM - 10:30 AM)
Session description
Several large INGOs are currently undergoing significant organizational change processes. These are
emergent and complex in nature, take place in a highly dynamic environment, and involve many
interdependent factors as well as feedback loops.
Developmental Evaluation (DE) documents and interprets dynamic developments, interactions and
interdependencies that happen as the intervention occurs. Thus, DE approaches naturally come to mind
as a way to assess and support organizational change processes. But what is their utility? This session is
an open space to explore this question.
The Question:
Can DE help organizations respond to the evaluation questions and need for learning that INGO leaders
and staff express? Can it help evaluate both change planning and management approaches as a process,
as well as their emergent outcomes? How to apply systems and complexity thinking without it becoming
overly ‘theoretical’ to busy practitioners?
The Invitation:
What are session participants’ experiences in encouraging organizational learning about change
processes? What has worked and what has not? What are the contexts in which DE can be useful, and
what are its limitations? How to develop collaborative, evaluative ways of sense- and meaning- making
and pattern spotting in INGOs, while being realistic about the amount of time and effort that can be
invested in this?
These questions will be the focus of this exploratory, ‘open space’ Think Tank, which is facilitated by
people who do not claim to be DE specialists at all – we merely are interested in offering this space and
are curious about the answers participants will come up with.
Background
Large INGOs such as Save the Children, Oxfam, CARE, Amnesty, WWF, ActionAid, CAFOD and several
others are undergoing or have undergone large organizational change processes. These often have
common drivers for change though they may take different directions in response. They are also intent
on learning from their change processes both in terms of how effectively they are being managed and
led, as well as their outcomes as compared to what was intended.
What’s Developmental Evaluation?
DE is designed to support social innovation and adaptive management and therefore seems very
suitable to support transformative processes (Patton, 2011). DE, by asking evaluative questions, can
apply evaluative logic and gather real time data to inform ongoing decision making and adaptation. The
evaluator is part of change management teams whose members collaborate to conceptualize, design
and test interim change directions, objectives and approaches in a process of continuous development,
adaptation and experimentation toward a long term change objective. In DE, evaluators are sensitive to
unintended consequences and side effects and can infuse the change management/championship team
with evaluative questions, thinking and data and identify relevant organization system-level data that
can inform strategic reflection on progress.
Practically, DE can be described the following ways:






the evaluator supports the implementing team
their task is to bring evaluative thinking and data to bear as implementing team conceptualizes
& tries out new approaches
the evaluator supports periodic sense-making processes and action-oriented decisions in
response to changing internal and external conditions (including among others organizational
culture change trends), lessons learned and emerging needs of participants as the change
agenda evolves in focus and clarity
the evaluator tailors the methods to track emergent and changing realities and change agendas,
illuminating reality and feeding back meaningful findings in real time to support innovation
DE can draw on multiple data collection and evaluation techniques such as staff engagement or
satisfaction surveys (QUANT) and QUAL methods such as Most Significant Change, Outcome
Mapping, Rich Pictures, Appreciative Inquiry, Timelines, Key informant interviews, etc.
the evaluator has to balance themselves on the insider-outsider continuum (‘critical friend’).
Why might DE be relevant for Organizational change processes?
Developmental Evaluation (as introduced by Michael Patton) may be a suitable approach to evaluation
and learning as an organizational change process unfolds. Organizational change processes are often
driven by networks of change champions, and as processes, they “cycle through stages, with transitions,
frequent need for adaptation, and they certainly are undertaken under conditions of complexity and
require innovations to guide adaptation to emergent and dynamic realities” (Patton, 2011).
Organizational change processes are also “characterized by large numbers of interacting and
interdependent elements in which there often is no central control” (Patton, 2011), given the highly
consultative culture of many INGOs. They involve situations in which “key stakeholders are in conflict
about how to proceed and reflect a situation of high uncertainty in terms of its outcomes where what
may emerge is unpredictable and uncontrollable” (Patton, 2011).
This think tank invites participants to explore two questions :
Q1: What examples do participants have of learning/evaluation processes that have supported
organizational change processes? To what extent do they either support or refute the proposition that
DE is particularly well suited to organizational change processes? (eg by either making the process more
effective or by not being particularly helpful or maybe even harmful/a waste of time & effort)?
Q2: What are the (pre)conditions for using a DE approach to organizational change process evaluation?
(For example, sufficient levels of mutual trust & confidence between leadership & staff, critical mass of
the organization embraces the change process, sufficiently common vision for the destination of the
change process, willingness of leadership to act on “findings” from emergent, participatory evaluation
processes, willingness to discuss and address power dynamics, etc.)