Outcome Harvesting
Ricardo Wilson-Grau
9 February 2012
Beirut, Lebanon
The next two hours
Purpose: Introduce you to the principles of Outcome
Harvesting as a monitoring and evaluation tool for
Outcome Mapping.
Intended results: You will understand the essential
principles of Outcome Harvesting and be able to
decide when and if it could be useful in your Outcome
Mapping work.
Ricardo.Wilson-Grau@inter.nl.net
Outcome Harvesting
A tool for monitoring and evaluating the results of
development interventions
Developed since 2003 by me and my colleagues Claudia
Fontes, Fe Briones Garcia, Gabriela Sánchez, Goele
Scheers, Heather Britt, Jennifer Vincent, Julie
Lafreniere, Juliette Majot, Marcie Mersky, Martha
Nuñez, Mary Jane Real, and Wolfgang Richert, and
currently Barbara Klugman and Natalia Ortiz.
Ricardo.Wilson-Grau@inter.nl.net
1. International social change networks
Ricardo.Wilson-Grau@inter.nl.net
2. Funding agencies
Outcome Harvesting
0. Focus on utilisation
1. Design the harvest
2. Review documentation and
draft outcomes
3. Engage with informants
4. Substantiate
5. Analyse, interpret
6. Support use of findings
Outcome Harvesting to meet needs
Monitoring
Evaluation
“I can honestly say that not
a day goes by when we don’t
use those evaluations in one
way or another.”
Outcome Harvesting
0.
1.
2.
3.
4.
5.
6.
Focus on utilisation
Design the harvest
Review documentation and draft outcomes
Engage with informants
Substantiate
Analyse, interpret
Support use of findings
Ricardo.Wilson-Grau@inter.nl.net
0. Focus on usefulness
Primary intended users
Principal intended uses
Useful M&E questions
Ricardo.Wilson-Grau@inter.nl.net
Examples
Management team requires information about
programme effectiveness in order to make funding
decisions for the next three years.
Ricardo.Wilson-Grau@inter.nl.net
Examples
Management team requires information about
programme effectiveness in order to make funding
decisions for the next three years.
To what extent did the outcomes we influenced in
2009-2011 represent patterns of progress towards our
strategic objectives?
Was our investment in the activities and outputs that
contributed to our 2009-2011 outcomes cost-effective?
What? So what?
Ricardo.Wilson-Grau@inter.nl.net
Focus on usefulness compared to OM
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
STEP 12: Evaluation Plan
My comparison
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
STEP 12: Evaluation Plan
Outcome Harvesting
0. Focus on utilisation
1. Design the harvest
Ricardo.Wilson-Grau@inter.nl.net
The M&E balancing act
The OM concept of
“outcome” as changes
in social actors
Useful Evaluation Question
To what extent did the outcomes we influenced in
2009-2011 represent patterns of progress towards
our strategic objectives?
Data required to answer
In the light of monitoring and evaluation questions,
what data is required and how and from whom will it
be obtained?
Ricardo.Wilson-Grau@inter.nl.net
To be
harvested
Behaviour
Relationships
CHANGE
Individual
Group or
community
SOCIAL
ACTOR
Institution
Organisation
Policies
and
practices
Actions,
activities
Outcome defined
A. An observable and significant change in a social
actor’s behaviour, relationships, activities, actions,
policies or practice that has been achieved. ..
B. … and that has been influenced by the change
agent.
Ricardo.Wilson-Grau@inter.nl.net
Outcome example
A. On May 16, 2007, World Bank President Paul
Wolfowitz resigns in a wake of scandals.
B. The Government Accountability Project got the
incriminating information to the media, leading to
widespread international coverage, and in turn,
inescapable pressure for Mr. Wolfowitz to resign.
Source: Evaluation of the Ford Foundation’s Global Governance Programme (2009)
Ricardo.Wilson-Grau@inter.nl.net
Design of Outcome Harvest
compared to OM
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
My comparison
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
Outcome Harvesting
0. Focus on utilisation
1. Design the harvest
2. Review documentation and draft outcomes
Ricardo.Wilson-Grau@inter.nl.net
Exercise
Please take five minutes to read this short, one-page
case study from AWID, the Association for Women’s
Rights in Development. Then, with the person seated
next to you, identify one outcome:
A. Who changed what, when and where?
Ricardo.Wilson-Grau@inter.nl.net
MGD3 Fund
Description: During 2011, the Dutch government
decided to allocate an additional €12 million to the
MGD3 Fund and to launch a new phase of support for
the MGD3 with €70 million for the Fund for Leadership
Opportunities for Women (FLOW) to strengthen the
rights and opportunities for women and girls.
The formulation of an outcome is as specific, verifiable
and as detailed as makes sense for the primary
intended users and their principal uses.
Ricardo.Wilson-Grau@inter.nl.net
Exercise
Now please identify how AWID contributed to it.
A. Who changed what, when and where?
B. What activities and outputs plausibly contributed
to the change in the social actor, however partially,
indirectly and even unintentionally?
Ricardo.Wilson-Grau@inter.nl.net
MGD3 Fund
Contribution of AWID: AWID compiled the document
"Mid-term Summary of the MDG3 Fund Value Added
and Outcomes", which was used by the Dutch Ministry
in its internal advocacy for the replenishment of the
MDG3 Fund. This material along with AWID’s
coordination efforts among MDG3 grantees also
supported the Dutch gender lobby platform with a
strong advocacy tool that they used to lobby the Dutch
government to commit these additional resources for
women’s rights organizations around the world.
Ricardo.Wilson-Grau@inter.nl.net
Other information?







Significance of the outcome
Collaboration with other social actors
Contribution of other actors and factors
History
Context
Evidence of impact on people’s lives
And so forth – it all depends on the information
required to answer the M&E questions.
Ricardo.Wilson-Grau@inter.nl.net
For example
Outcome’s significance: In the light of the impact to
which you wish to contribute, why is this outcome
important?
Ricardo.Wilson-Grau@inter.nl.net
Outcome example
A. On May 16, 2007, World Bank President Paul
Wolfowitz resigns in a wake of scandals.
B. The Government Accountability Project got the
incriminating information to the media, leading to
widespread international coverage, and in turn,
inescapable pressure for Mr. Wolfowitz to resign.
C. The Wolfowitz episode dramatically exposed the
weak governance structure of the World Bank itself
and it is unlikely that governance at the Bank can
revert to the minimal scrutiny and oversight carried
out before the scandal.
Ricardo.Wilson-Grau@inter.nl.net
Reviewing documentation and
drafting outcomes compared to OM
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
My comparison
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
My comparison
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
Outcome Harvesting
0.
1.
2.
3.
Focus on utilisation
Design the harvest
Review documentation and draft outcomes
Engage with informants
Ricardo.Wilson-Grau@inter.nl.net
3. Engage with informants
 Email
 Skype or telephone
 In person
MGD3 Fund
Description: During 2011, the Dutch government
decided to allocate an additional €12 million to the
MGD3 Fund and to launch a new phase of support
for the MGD3 with €70 million for the Fund for
Leadership Opportunities for Women (FLOW) to
strengthen the rights and opportunities for women
and girls.
Comment [RW-G1]: Can you be
more specific? Who in
government? Which agency?
Ricardo.Wilson-Grau@inter.nl.net
Rita Fund
Description: The Rita Fund is created in the
United States. The Rita Fund is a woman’s fund
which strives to respond to the “funding gap”
between donors’ interest and their actual
funding by creating a reliable non restrictive
funding source for women’s funds operating
worldwide.
Contribution of AWID: AWID’s report Where is
the Money for Women’s Rights?, published in
2008, was the source of information and
inspiration from which came the idea to create
the Rita Fund.
Comment [RW-G1]: Who created
this fund?
When was it created?
Specifically where was it created?
Comment [RW-G2]: Is this an
appropriate characterisation of
“funding gap”?
Comment [RW-G3]: Did AWID do
something more active to influence
the creation of the Fund?
Ricardo.Wilson-Grau@inter.nl.net
Outcome Harvesting to date
Client
Hivos (Environment) – Central and South America
CIFCA-Central America
AFAD-Asia
Forum-Asia
Hivos (Art & Culture) - Central America
IDRC (PAN Asia)
Ford Foundation- Global
PIDHDD - Central and South America
GPPAC - Global
IDRC (Acacia) – Africa
Oxfam Novib – Global
PSO - Honduras and Ghana
Open Society Institute – South America
ECOSAD – Lima, Perú
UNTF – Global
ActionAid – Ghana
ActionAid – Tanzania
BioNET - Global
Total
Client
Date of harvest report
Hivos (Environment) – Central and South America
January 2005
CIFCA-Central America
April 2005
AFAD-Asia
September 2005
Forum-Asia
June 2007
Hivos (Art & Culture) - Central America
May 2008
IDRC (PAN Asia)
July 2008
Ford Foundation- Global
PIDHDD - Central and South America
September 2008
July 2009
GPPAC - Global
September 2009
IDRC (Acacia) – Africa
December 2009
Oxfam Novib – Global
March 2010
PSO - Honduras and Ghana
September 2010
Open Society Institute – South America
October 2010
ECOSAD – Lima, Perú
October 2010
UNTF – Global
August 2011
ActionAid – Ghana
ActionAid – Tanzania
BioNET - Global
Total
November 2011
August 2011
February 2011
2004-2011
Client
Date of harvest report
# informants*
outcomes
Hivos (Environment) – Central and South America
January 2005
13
April 2005
4
AFAD-Asia
September 2005
8
Forum-Asia
June 2007
14
Hivos (Art & Culture) - Central America
May 2008
21
IDRC (PAN Asia)
July 2008
4
September 2008
24
July 2009
8
GPPAC - Global
September 2009
19
IDRC (Acacia) – Africa
December 2009
13
Oxfam Novib – Global
March 2010
38
September 2010
2
Open Society Institute – South America
October 2010
5
ECOSAD – Lima, Perú
October 2010
1
UNTF – Global
August 2011
61
November 2011
6
August 2011
8
February 2011
12
CIFCA-Central America
Ford Foundation- Global
PIDHDD - Central and South America
PSO - Honduras and Ghana
ActionAid – Ghana
ActionAid – Tanzania
BioNET - Global
Total
2004-2011
261
* Networks, NGOs, research centres, think tanks, university departments, government agencies, communitybased organisations
Client
Date of harvest report
# informants*
outcomes
# outcomes
harvested
Hivos (Environment) – Central and South America
January 2005
13
300
April 2005
4
33
AFAD-Asia
September 2005
8
32
Forum-Asia
June 2007
14
35
Hivos (Art & Culture) - Central America
May 2008
21
87
IDRC (PAN Asia)
July 2008
4
23
September 2008
24
104
July 2009
8
43
GPPAC - Global
September 2009
19
68
IDRC (Acacia) – Africa
December 2009
13
85
Oxfam Novib – Global
March 2010
38
196
September 2010
2
19
Open Society Institute – South America
October 2010
5
28
ECOSAD – Lima, Perú
October 2010
1
18
UNTF – Global
August 2011
61
653
November 2011
6
46
August 2011
8
27
February 2011
12
187
CIFCA-Central America
Ford Foundation- Global
PIDHDD - Central and South America
PSO - Honduras and Ghana
ActionAid – Ghana
ActionAid – Tanzania
BioNET - Global
Total
2004-2011
261
1984
* Networks, NGOs, research centres, think tanks, university departments, government agencies, communitybased organisations
How much or little detail?
Depends on the users, uses and M&E questions.
Can be one sentence as in the example or a page or
more….
Ricardo.Wilson-Grau@inter.nl.net
Outcome example
A. On May 16, 2007, World Bank President Paul
Wolfowitz resigns in a wake of scandals.
B. The Wolfowitz episode dramatically exposed the
weak governance structure of the World Bank itself
and it is unlikely that governance at the Bank can
revert to the minimal scrutiny and oversight carried
out before the scandal.
C. The Government Accountability Project got the
incriminating information to the media, leading to
widespread international coverage, and in turn,
inescapable pressure for Mr. Wolfowitz to resign.
Ricardo.Wilson-Grau@inter.nl.net
Title: On May 16, 2007, World Bank President Paul Wolfowitz resigns in a wake of scandals.
Outcome: On May 16, 2007, World Bank President Paul Wolfowitz resigns in a wake of wide-ranging scandals. Among
them: Wolfowitz’s companion, Shaha Riza, received salary raises far in excess of those allowable under Bank rules; Riza
received a questionable consulting position with a U.S. defense contractor in 2003 at his direction that resulted in State
and Defense Department inquiries; Juan Jose Daboub, Bank Managing Director and Wolfowitz-hire, attempted to
remove reference to “family planning” from the Bank’s new health sector strategy; Wolfowitz’s office was responsible
for weakening a “climate change” strategy document; Bank Senior Management delayed reporting to Bank staff that a
fellow staffer had been seriously wounded in a shooting in Iraq; Bank lending to Africa was deficient and delayed and
could only meet lending targets if new loans were brought to the Board for approval without the proper formulation
procedures; and that Wolfowitz was trying to broaden the Bank’s portfolio in Iraq over Board opposition, among other
disclosures. Within a year, four controversial Wolfowitz appointees had left the Bank, including special advisors Robin
Cleveland and Kevin Kellems, General Counsel Ana Palacio and Suzanne Folsom, the director of the Bank’s Department
of Institutional Integrity (INT).
Contribution of the grantee: The Government Accountability Project was the recipient of anonymously sent internal
World Bank memos and tips relevant to many of the disclosures leading to the Wolfowitz resignation, among them,
information that Wolfowitz was negotiating a contract with the resident Iraq Country director, in violation of the Bank’s
Articles of Agreement; an email about the shooting of a Bank employee in Iraq and the Bank’s failure to follow its
protocol for informing staff about these injuries; payroll records of Shaha Riza; internal documents showing that
Daboub, the Wolfowitz-appointed Managing Director, had instructed a team of Bank specialists to delete all references
to “family planning” from the proposed Country Assistance Strategy for Madagascar; internal documents indicating that
Daboub had ordered the toning down of references to “climate change” in Bank environmental strategy papers; etc.
GAP was directly responsible for getting this information to the media – through press releases, leaking documents or
corroborating information – leading to widespread international coverage, and in turn, inescapable pressure for Mr.
Wolfowitz to resign. As The Economist wrote on April 12, 2007: “As he prepares to welcome ministers to the bank's
spring meetings in Washington, DC, on April 14th and 15th, Mr. Wolfowitz's own toes … may feel a bit toasty. For that,
he can thank the Government Accountability Project (GAP), a vibrant participant in America's civil society.”
Significance: The Wolfowitz episode dramatically exposed the weak governance structure of the World Bank itself, even
as its senior officials demand good governance to borrowing countries. In the wake of the scandal, it is unlikely that
governance at the Bank can revert to the minimal scrutiny and oversight carried out before the scandal.
A story
On May 16, 2007, World Bank President Paul Wolfowitz resigns in a wake of wide-ranging scandals. As The Economist wrote on April 12,
2007: “As he prepares to welcome ministers to the bank's spring meetings in Washington, DC, on April 14th and 15th, Mr. Wolfowitz's own
toes … may feel a bit toasty. For that, he can thank the Government Accountability Project (GAP), a vibrant participant in America's civil
society.”
Wolfowitz’s companion, Shaha Riza, received salary raises far in excess of those allowable under Bank rules; Riza received a questionable
consulting position with a U.S. defense contractor in 2003 at his direction that resulted in State and Defense Department inquiries; Juan Jose
Daboub, Bank Managing Director and Wolfowitz-hire, attempted to remove reference to “family planning” from the Bank’s new health sector
strategy; Wolfowitz’s office was responsible for weakening a “climate change” strategy document; Bank Senior Management delayed
reporting to Bank staff that a fellow staffer had been seriously wounded in a shooting in Iraq; Bank lending to Africa was deficient and
delayed and could only meet lending targets if new loans were brought to the Board for approval without the proper formulation procedures;
and that Wolfowitz was trying to broaden the Bank’s portfolio in Iraq over Board opposition, among other disclosures.
The Government Accountability Project was the recipient of anonymously sent internal World Bank memos and tips relevant to many of the
disclosures leading to the Wolfowitz resignation, among them, information that Wolfowitz was negotiating a contract with the resident Iraq
Country director, in violation of the Bank’s Articles of Agreement; an email about the shooting of a Bank employee in Iraq and the Bank’s
failure to follow its protocol for informing staff about these injuries; payroll records of Shaha Riza; internal documents showing that Daboub,
the Wolfowitz-appointed Managing Director, had instructed a team of Bank specialists to delete all references to “family planning” from the
proposed Country Assistance Strategy for Madagascar; internal documents indicating that Daboub had ordered the toning down of references
to “climate change” in Bank environmental strategy papers; etc. GAP was directly responsible for getting this information to the media –
through press releases, leaking documents or corroborating information – leading to widespread international coverage, and in turn,
inescapable pressure for Mr. Wolfowitz to resign.
The Wolfowitz episode dramatically exposed the weak governance structure of the World Bank itself, even as its senior officials demand good
governance to borrowing countries. In the wake of the scandal, it is unlikely that governance at the Bank can revert to the minimal scrutiny
and oversight carried out before the scandal.
Within a year, four controversial Wolfowitz appointees had left the Bank, including special advisors Robin Cleveland and Kevin Kellems,
General Counsel Ana Palacio and Suzanne Folsom, the director of the Bank’s Department of Institutional Integrity (INT).
Ricardo.Wilson-Grau@inter.nl.net
One liners
1.
2.
3.
4.
5.
6.
In December 2005, the United Nations issues a groundbreaking
whistleblower protection policy.
In August, 2006, GAP, in tandem with the World Bank Staff Association,
stopped an attempt by then Bank President Paul Wolfowitz’s office to
hurry through a flawed draft whistleblower protection policy.
In January, 2007, the Board of Executive Directors of the African
Development Bank (AFDB) approves a whistleblower protection policy.
On May 16, 2007, World Bank President Paul Wolfowitz resigns in a
wake of scandals.
The World Bank adopts recommendations for improving the Bank’s
investigative unit INT and the Director of INT resigns.
In June 2008, the World Bank passes an improved whistleblower
protection policy.
Ricardo.Wilson-Grau@inter.nl.net
Engaging with informants
compared to OM
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
My comparison
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
Outcome Harvesting
0.
1.
2.
3.
4.
Focus on utilisation
Design the harvest
Review documentation and draft outcomes
Engage with informants
Substantiate
Ricardo.Wilson-Grau@inter.nl.net
Present the final outcome formulation to one or more credible (independent,
knowledgeable) person(s) and ask them to go on record with their opinion:
1. To what degree you are in agreement with the description of the Dutch government’s decision
to allocate an additional €12 million to the MGD3 Fund and to launch a new phase of support for
the MGD3 with €70 million for the Fund for Leadership Opportunities for Women (FLOW)
[ ] Fully agree
[ ] Partially agree
[ ] Disagree
Comments if you like:
2. How much do you agree with the description of how AWID influenced the Dutch government’s
decision?
[ ] Fully agree
[ ] Partially agree
[ ] Disagree
Comments if you like:
Case studies
To enhance the
understanding and
credibility of the findings.
Substantiation compared to OM
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
My comparison
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan??
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
Outcome Harvesting
0.
1.
2.
3.
4.
5.
Focus on utilisation
Design the harvest
Review documentation and draft outcomes
Engage with informants
Substantiate
Analyse, interpret
Ricardo.Wilson-Grau@inter.nl.net
Organise the outcomes so that they can be interpreted
in ways that will enable you to answer the Outcome
Harvesting questions.
To what extent did the outcomes we influenced in
2009-2011 represent patterns of progress towards our
strategic objectives?
Classify the outcomes according to strategic objectives,
country or region, year…
Ricardo.Wilson-Grau@inter.nl.net
Excel Database
Access
Accessdatabase
Database
Seeking processes, patterns, trends
+
Women’s
group
launches
campaign
President
declares
women's land
inheritance a
right
Agriculture
committee
discusses draft
law
+
Legislature
passes
women’s right
to land
inheritance
law
+
Court decides
against widow
Widow takes
late husband’s For the first
family to court time, press
runs story on
women’s land
inheritance
+
Women’s
group initiates
legal aid for
widows
+
Over the past
5 years, in
90% of the
cases the
Land
Commission
has given to
widows the
titles of their
Late husbands’
late
families
husbands’
murder three
land
widows
Court decides
landmark case
in favour of
widow
+
Policy
change
+
Practice
change
+
Finance
committee
discusses draft
law
Ten widows
take late
husbands’
family to court
+
+
Land
Commission
approves
regulations for
women’s
inheritance of
deceased
husband’s land
+
Legislator
drafts law
Time
+
+
Religious
leader
declares
women's land
inheritance a
right
Agricultural
union admits
widows as
members
Theory of change
MISSION
IMPACT
ASSUMPTIONS
RESULTS
STRATEGIES
SOCIAL ACTORS
Systems approach
Ricardo.Wilson-Grau@inter.nl.net
Analyse, interpret compared to OM
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
My comparison
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
My comparison
INTENTIONAL DESIGN
STEP 1: Vision
STEP 2: Mission
STEP 3: Boundary Partners
STEP 4: Outcome Challenges
STEP 5: Progress Markers
STEP 6: Strategy Maps
STEP 7: Organizational Practices
EVALUATION PLANNING
STEP 12: Evaluation Plan
OUTCOME &
PERFORMANCE MONITORING
STEP 8:
Monitoring Priorities
STEP 9:
Outcome Journals
STEP 10: Strategy Journal
STEP 11: Performance Journal
Outcome Harvesting
0.
1.
2.
3.
4.
5.
6.
Focus on utilisation
Design the harvest
Review documentation and draft outcomes
Engage with informants
Substantiate
Analyse, interpret
Support use of findings
Ricardo.Wilson-Grau@inter.nl.net
6. Support for use of findings
My summary comparison between
Outcome Harvesting
and
Outcome Mapping
Outcome Harvesting
0. Focus on utilisation
1. Design the harvest
STEP 8:
STEP 9:
Monitoring Priorities
Outcome Journals
2. Review documentation and
draft outcomes
STEP 9: Outcome Journals
STEP 10: Strategy Journal?
STEP 11: Performance
Journal?
3. Engage with informants
4. Substantiate
5. Analyse, interpret
6. Support use of findings
STEP 9: Outcome Journals
STEP 9: Outcome Journals?
INTENTIONAL DESIGN, EVALUATION
Many thanks!
Do you have questions?
ricardo wilson-grau consulting
Rua Marechal Marques Porto 2/402, Tijuca, Rio de Janeiro, CEP 20270-260,
Brasil
Telephone: 55 21 2284 6889; Skype: ricardowilsongrau
Direct via VOIP, dialing locally from:
The Netherlands (or Europe): 31 71 302 0429
USA or Canada: 1 347 404 5379
S.M.A.R.T. outcomes
Specific
Each outcome is formulated in
sufficient detail so that a
primary intended user without
specialised subject or
contextual knowledge will be
able to understand and
appreciate what changed.
Who changed what, when and
where?
Example
In 2008-2009, the Norwegian
Ministry of Foreign Affairs
replenished its Budget Line on
Gender Equality to fund the
implementation of their
Women’s Rights Program,
which included significant
funding for women’s
organizations.
SMART outcomes
Measurable
The description of the
outcome provides objective,
verifiable quantitative and
qualitative information,
independent of who is
collecting data.
How much? How many? When
and where did the change
happen?
Example
In 2008-2009, the Norwegian
Ministry of Foreign Affairs
replenished its Budget Line on
Gender Equality to fund the
implementation of their
Women’s Rights Program,
which included significant
funding for women’s
organizations.
SMART outcomes
Achieved
By AWID while perhaps not
fully attributable to AWID. A
plausible relationship, a logical
link between the outcome and
what AWID did that
contributed to it.
What did AWID do, when and
where that contributed —
wholly but probably partially,
indirectly or indirectly,
intentionally or unexpectedly?
Example
WITM had a session on
funding at CSW that generated
important discussions and
sparked conversations with
various partners in the
movement as well as bilaterals and provided evidence
for the steady or expanding
funding support for women’s
organizations .
SMART outcomes
Relevant
The outcome represents
noteworthy progress towards
the impact you desire.
Example
Shifts in donor and
development policy and
practices for a greater financial
and political commitment to
gender equality and women’s
rights, including women’s
organizations.
SMART outcomes
Timely
The outcome occurred within
2006-2011, although AWID’s
contribution may have been
months or even years before.
Example
In 2008-2009, the Norwegian
Ministry of Foreign Affairs
replenished its Budget Line on
Gender Equality…
WITM had a session on
funding… and provided
evidence for the steady or
expanding funding support for
women’s organizations .
Outcome substantiation
One or more people independent of AWID and
knowledgeable about a mutually agreed percentage of
the outcomes, or of a representative group of
outcomes, agrees (or not) with the formulation of the
outcome and AWID’s contribution.
The credible “authority” goes on public record with her
or his opinion about the outcome and AWID’s
contribution.
Ricardo.Wilson-Grau@inter.nl.net