Third Sector Evaluation:
Challenges and Opportunities
Presentation to the Public Legal Education in Canada
National Conference on “Making an Impact”
26 October 2012
Michelynn Lafleche
Director of Research, Public Policy & Evaluation, United Way Toronto
Evaluation in a changing context
•
•
Evaluation is a vital component of our work in the third
sector – supporting organizational planning, decisionmaking, and learning
Evaluation has always played an important role in the
sector, but the changing context of the third sector –
the current funding environment and economic climate
– has increased expectations regarding evaluation,
with a particular focus on outcomes and accountability.
Funders and funded organizations need to
work through 6 issues together
1. Understanding the context in which our work is situated
2. Acknowledging the limits of performance measurement and
monitoring
3. Recognizing the complexity of the interaction of individual,
family, community and structural factors that influence
outcomes
4. Addressing the question of attribution
5. Building evaluation capacity across the sector
6. Planning for evaluation as part of program implementation
Challenges of evaluation focused on
outcomes and accountability (1)
•
•
•
Results in data-driven exercises that do not always
contribute to a meaningful assessment of impact or to
program improvement
Generates vast quantities of data, some of questionable
value
Shifts the focus among service providers from service
delivery to measurement with no clear benefit to the
programs
Challenges of evaluation focused on
outcomes and accountability (2)
•
•
•
Common outcome setting processes, often driven by funders,
are becoming more common-place, but outcome reporting is
most useful when outcomes are aligned to the program
theory of an intervention and the mission of an organization
Outcome measurement alone can only demonstrate if
something is changing, not why it is changing or if the
intervention or initiatives is causing the change
Linking outcomes to funding decision may intensify these
challenges
Opportunities to advance evaluation
•
•
•
•
Moving beyond outcome evaluation to include not just a
focus on what worked, but why it worked, with whom and in
what context – i.e. community-based approaches such as
developmental and realist evaluation or participatory
approaches using qualitative methods
Promoting evaluation for learning to explore how to
understand and solve problems more effectively
Fostering opportunities for knowledge generation and
exchange about what does and what doesn’t work
Recognizing that evaluation is not a solution all on its own
Characteristics associated with evaluation
leaders
1.
2.
3.
4.
5.
6.
7.
8.
A long-term commitment to evaluation
An appreciation that evaluation is an ongoing and evolving activity
A willingness to share mistakes and learn from failure
An engaged and committed team of internal staff, outside
evaluation expertise, and key stakeholders
Clear leadership commitment to sharing and using evaluation
findings to support learning and improvement
A commitment and process to translate findings into action,
knowledge exchange, and advocacy
An ability to tailor evaluation to context
A focus on building the capacity of all partners
Contact details
Michelynn Lafleche
Director, Research, Public Policy & Evaluation
United Way Toronto
mlafleche@uwgt.org
416 777 1444 x220