STRATEGY, MEASUREMENT & EVALUATION
QUALITY OUTCOMES & RESULTS FRAMEWORKS
Quality Outcomes and Results Frameworks
Guidance Note for External Partners (v1, 8 August 2014)
Contents
Introduction .................................................................................................................................................. 1
Terms & Definitions ...................................................................................................................................... 2
How to Craft an Outcome Statement ........................................................................................................... 2
Characteristics of a Quality Results Framework ........................................................................................... 3
Distinct, logically-connected results ......................................................................................................... 4
Clear, specific and measureable results.................................................................................................... 6
Ambitious, yet attainable results .............................................................................................................. 6
Checklist of Review Questions ...................................................................................................................... 8
Finding a Common Language to Discuss Results Frameworks ................................................................... 10
Introduction
A central part of the foundation’s business model, Outcome Investing, involves designing and managing
investments that drive toward measureable outcomes. The approach has three defining characteristics:
outcomes at the outset, logical design, and managing toward results.



Outcomes at the outset – Program Officers and grantees align early with our grantees on the
few, most important results that define what success looks like for the investment
Logical design - Define a set of logically connected results that are on the pathway toward
achieving the investment outcomes
Managing toward results – Program Officers and grantees identify the most relevant indicators
to monitor progress over the course of the investment, and then managing toward those results
This guidance note will help you practice Outcome Investing, with a focus on developing quality
outcomes and results frameworks. It includes practical suggestions for crafting results, illustrative
examples, and a checklist of review questions that will help you provide actionable feedback to your
grantee.
1
QUALITY OUTCOMES & RESULTS FRAMEWORKS
Terms & Definitions
Table 1 outlines the Outcome Investing terms and definitions relevant to the Results Framework, with
an associated example for each. The investment results – primary outcomes, intermediate outcomes,
and outputs – are described in the blue box.
Table 1. Terms, Definitions & Examples
Term
Strategic Goals
Primary Outcomes
Intermediate
Outcomes
Definition
The 3-5 year results of an initiative,
sub-initiative or portfolio toward
which the investment contributes
most directly (identified in the
strategy scorecard)
The overall change(s) in
technologies, systems, populations
or behaviors the investment seeks
to achieve within the context of the
investment timeframe
The changes in technologies,
systems, populations or behaviors
that need to be achieved in order to
realize the primary outcome(s)
Outputs
The goods, services, events or
deliverables produced during an
investment
Activities
The actions taken or work
performed to produce goods,
services, events or deliverables
Example
The sustainable elimination of
Visceral Leishmaniasis (VL) from
Bihar over five years
Reduced VL incidence in four
districts of Bihar to less than 1 in
10,000 by 2016
Increased coverage (to 95%) of VL affected villages in four districts of
Bihar with Indoor Residual Spraying
that meets quality standards
Block-level micro-plans developed
for all villages in four districts of
Bihar where VL patients have been
identified over the past five years
Collect data on where VL patients
have been identified over the past
five years
How to Craft an Outcome Statement
An investment outcome is the highest level result or change we expect to see from a single investment.
The foundation does not measure impacts (e.g. decreased poverty or mortality) as part of measuring
investment results, as these are long-term, sustainable changes that are more relevant to a set of
investments, an initiative or a strategy.
At its most basic level, an outcome (primary or intermediate) describes the direction and type of change
among a specific group of people or system. The following rubric can be useful when crafting or
reviewing investment outcomes.
2
QUALITY OUTCOMES & RESULTS FRAMEWORKS
Verb Indicating
Change
Examples:
• Increased
• Decreased
• Improved
• Reduced
• Adopted
• Established
• Used
• Integrated
What Changes
Who Changes
Examples:
• Coverage
• Behavior
• Knowledge
• Technologies
• Models
• Data
• Systems
• Policies
Examples:
• Individuals
• Communities
• Populations
• Governments
• Institutions
Additional Specificity,
when feasible
Examples:
• Where you expect
to see this change?
(include geography)
• By when you expect
to see the change?
(include date)
• How much change
you expect to see?
(include target)
Example investment outcomes:
The components of the following outcomes are color-coded according to the rubric above.
o Increased use of self-paced math programs among 25 school districts in California by 2018
o Improved early care-seeking practices for pneumonia and diarrhea among caretakers from
two regions in Nigeria by year 3 of the project
o Adoption of national sanitation and hygiene policies and strategies (including non-sewer basic
sanitation) in four countries classified as fragile by 2015
While we strive to include additional specificity in our outcomes during the planning stage, it is not
always possible until after the investment has begun. This is particularly true for outcome targets as it is
necessary to have baseline or other data available prior to the investment to determine “how much
change” you would reasonably expect to see over the course of the project. Such data are often not
available at the time of planning, so additional detail related to targets would have to be added after the
investment has been approved.
Characteristics of a Quality Results Framework
Once Program Officers and grantees align on the outcomes that indicate success for the investment, we
work together to build out a set of results that if achieved, will produce the primary outcome(s). The
Results Framework is the tool that shows these results and the foundation’s strategic goals toward
which they contribute.
It is important to note that the Results Framework is just a set of boxes. Developing the content that
goes into those boxes takes time, conversation, and collaboration between Program Officers and
prospective grantees, and sometimes support from other foundation stakeholders.
3
QUALITY OUTCOMES & RESULTS FRAMEWORKS
A “quality” Results Framework is one that lends itself to both alignment between the foundation and
partner, and good, useful measurement. This requires three things: (1) distinct, logically connected
results, (2) clear, specific and measureable results, and (3) ambitious, yet attainable results. Each
characteristic is described in greater detail below.
Distinct, logically-connected results
Outputs and outcomes should be distinct from activities. Activities describe what you as an
implementer will do (e.g. develop resources, hold trainings, conduct outreach), while outputs, for
example, tell us what is produced from those activities (e.g. new resources, trained staff, people
reached). Activities can be included in the Proposal Narrative, as needed, but we do not measure them
or manage the execution of them.
Activity
Educate banking customers about
the benefits of making payments
through mobile money
Output
Vs.
Banking customers enrolled for
mobile money services
Outputs and outcomes should be distinct from one another. Outputs describe what the investment will
directly produce (e.g. reports, tools, products, skilled staff), while outcomes describe the changes (e.g. in
behavior, coverage, policies) we hope will be achieved by the investment. As the implementing
organization, you will likely have more control over achieving outputs than outcomes.
Outcome
Output
Early learning educators in five
states completed new e-learning
modules
Vs.
Use of curriculum adapted based
on content in new e-learning
modules by early learning
educators in five states
When an investment has two levels of outcomes, the intermediate outcome should represent changes
that are critical steps on the pathway to achieving the primary outcome(s).
Intermediate Outcome
Improved adherence to
diagnostic guidelines by health
workers in two regions of Burkina
Faso
Primary Outcome
Vs.
Improved treatment of childhood
illnesses in two regions of Burkina
Faso
Outcomes should be distinct from strategic goals. Strategic goals are higher-level initiative, subinitiative, or portfolio level results that investments are contributing to (and that signal an investment is
4
QUALITY OUTCOMES & RESULTS FRAMEWORKS
strategically aligned). Outcomes, however, are changes considered achievable given the specific
investment context and timeframe. In most cases, it takes multiple investments to achieve a strategic
goal.
Outcome
Strategic Goal
Increased access to evidence on
agricultural interventions that
improve nutritional outcomes
Vs.
Increased evidence-based
nutrition aid to agriculture over
five years
Investment results should be logically connected, with one level of result leading to the next. A logical
set of investment results should clearly show the causal, or if/then, relationship between a related set of
results. For example, if staff are trained on maintaining and managing vaccine cold chains (output), then
it would be reasonable to expect an outcome of
Tip: When designing investment results, it is
reduced vaccine wastage to be achieved. You should
include a discussion of assumptions in the proposal
often helpful to visualize the results logic and
then translate the key information into the
narrative to describe the important events, conditions,
Results Framework. One way to do this is to
or decisions outside the control of the project that are
write all of your results on sticky notes and build
necessary for the results logic to hold.
a results chain from top-down (primary
outcomes  outputs) or bottom-up (outputs 
primary outcomes). Then ask yourself a series of
questions about what needs to happen to get
from one level of result to the next. Add new
results to fill any gaps in logic that you uncover
through this process.
Logically connected results should not have any gaps,
where the reader would need to infer additional details
to understand how the outputs will lead to the
outcomes. Imagine if the logical flow shown in Figure 1
only included the output and primary outcome. A
reader would likely identify a gap between the output
of ‘village level demonstrations held’ and the primary outcome of ‘use of the storage bags by farmers’.
There are several steps (or changes) that need to happen between these two results. Adding the
intermediate outcome of ‘increased sales of bags by merchants’ helps to tell the story of how village
level demonstrations of the new storage bags would lead to farmers using them.
Figure 1. Logical Flow of Results
Primary
Outcome
Increased use of hermetic produce storage
bags by smallholder farmers in the major
production zones of West Africa by year 5
Intermediate
Outcome
Increased sales of hermetic produce storage
bags by merchants in the major production
zones of West Africa by year 4
Output
At least 400 village level demonstrations of the
produce storage bags held in the major
production zones of West Africa by year 3
5
QUALITY OUTCOMES & RESULTS FRAMEWORKS
Some straightforward investments may end in outputs (e.g. a new rapid drug susceptibility test for TB).
When this is the case, it is important that the outputs are logically linked to other investment results
that together contribute to the achievement of a strategic goal (e.g. WHO endorsement of two new TB
molecular diagnostics by 2016). This information can be captured in the strategic fit section of the Short
Form Results Framework and Tracker, which is the tool that you will receive if you and your Program
Officer agree that the investment will end in outputs.
Clear, specific and measureable results
Investment results should be clear, specific, and measureable. Results language should be easy to
understand by any reader, not just a few insiders that know the project well. The Results Framework
does not ask about measurability, as this information is included in the Results Tracker. However, The
more precise the results language, the less time the Program Officer will spend asking clarifying
questions to reach alignment with you, and the easier it will be to identify indicators for those results in
the Results Tracker.
A few tips:
 Avoid ambiguous, vague terms, and do not assume familiarity with the content.

Poor Example
Improved Example
Increased support from South
East Asian governments to
reduce tobacco use by 2014
Increased taxation level on
cigarettes in five South East
Asian countries by 2014
Make sure every result reflects a single change, not multiple changes together in one statement.
Poor Example
Improved Example
Increased access to and use of
books in Ugandan public
libraries by year 2 of the
project
Increased use of books in
Ugandan public libraries by
year 2 of the project
Ambitious, yet attainable results
Investment results should be ambitious, yet attainable. The foundation supports ambitious ideas and
innovative projects, but at the same time expects the results from our investments to be feasible given
your organization’s capacity, the timeframe of the work, and the financial resources available.
6
QUALITY OUTCOMES & RESULTS FRAMEWORKS
Case Study:
The Malaria team made a 21 month, $30 million investment for their grantee to lay the groundwork for
demonstrating the feasibility of malaria elimination in four medium-to-high burden sub-Saharan African
countries. The grantee originally submitted a very ambitious primary outcome of eliminating local
malaria transmission in the four countries. After much discussion, the foundation and grantee realized
that the pathway of change for the investment was not logical to lead to such high-level results. They
were able to scale-back the original primary outcome to a more reasonable result around an increased
use of evidence to inform policy and practice.
Poor Example
Improved Example
Elimination of local malaria
transmission at national and
subnational levels in four
Africa countries
Increased use of evidence by
government stakeholders in
four African countries to
inform national policy and
practice
Note that all of the “improved examples” are just that, improved over the poor example, but still not
perfect. Designing quality investment results is messy work and there is no one “right” way to approach
it. Remember that it is not about getting to perfect, but rather getting to the point where both you and
the Program Officer are aligned on the set of investment results and feel jointly responsible for their
achievement. Foundation measurement and evaluation staff are available to help you and the Program
Officer get to this point..
7
QUALITY OUTCOMES & RESULTS FRAMEWORKS
Checklist of Review Questions
The following questions are ones you might ask yourself as you review a draft of the Results
Framework. They help identify problems or common pitfalls when drafting a set of logical results. If any
statements are true, then use the associated discussion prompt to help guide your internal conversation
or a conversation with your Program Officer about how best to revise the Results Framework.
Distinct, logically connected results
 Checklist of Review Questions
 Do any of the outcomes or outputs
describe how you will do the work or
the specific tactics that you will use?
 Are you having trouble distinguishing
between what the investment will
produce (i.e. outputs) and what the
investment will change (i.e.
outcomes)?
 Do any of the outcomes identify a
change that you think multiple
investments are working towards, and
that won’t likely be achieved by the
end of a single investment?
 Are you having trouble believing that
the outputs will really lead to the
achievement of outcomes?
If so, then…
 The investment results are likely worded as
activities. Ask yourself about the end toward
which the activities will be performed, and
consider revising the results.
 The investment results are likely not distinct
enough. Try mapping out the ‘if, then’ logic of the
results, which should help clarify what needs to be
produced (i.e. outputs) in order to lead to change
(i.e. outcomes).
 These outcomes are likely worded as strategic
goals or identify the broader vision or aspiration
for the investment. Ask your Program Officer to
discuss the strategic goals with you and other
related investments to clarify where your
investment fits in and what your investment can
feasibly achieve within its context and timeframe.
You and the Program Officer may decide to scaleback the results to reflect what you’re really trying
to get out of this single investment.
 There are likely gaps in logic. Try mapping out the
‘if, then’ logic of the results, and talk to your
Program Officer about where additional outputs
or outcomes may be needed to fully tell the story.
Clear, specific and measurable results
 Checklist of Review Questions
 Do any of the results include vague or
ambiguous terms that peer reviewers
or your Program Officer might
question what they mean?
If so, then…
 The results may not be clear and/or specific
enough. In order to determine how how they
might be framed more concretely, you might ask
yourself: What does X look like? How will I know
when X has occurred? This can help you reword
the results to include more clear and/or specific
language.
8
QUALITY OUTCOMES & RESULTS FRAMEWORKS
 Do any of the results contain multiple
changes strung together in one
statement?
 Do any of the results lack detail about
when the result will be achieved,
where the result will be seen, and/or
how much change is likely to be
achieved within the investment
timeframe?
 Are you having trouble identifying
how you will know when the results
have been achieved (or whether the
investment is on track towards
achieving them)?
 The results may not be clear enough. Consider
breaking up the content of those results so each
contains a single, measurable change. Also
consider the relationship between the two
changes and determine where best in the results
framework to place each result.
 The results could be more specific. Review the
rubric on crafting an outcome statement found on
page 3 of this document, and add specificity about
timeframe, geography, and targets when it is
feasible and relevant to do so.
 The results may not be specific enough to be
measurable, or the wrong indicators are used in
the Results Tracker. Try either revising the results
language to get increased clarity on what the
specific changes are, or identify more appropriate
indicators on the Results Tracker.
Ambitious, yet attainable results
 Checklist of Review Questions
 Does the Program Officer expect you
to achieve more than what you have
documented within the investment
timeframe and resources available?
 Do any of the results seem out of
reach given the investment’s work
streams, timeframe, and planned
resources?
If so, then…
 The results may not be ambitious enough. Think
about “stretch” results for the investment, and be
sure to right-size expectations about what will
happen if those results are not met.
 The results may be unattainable for the
investment. Talk to the Program Officer about
scaling back results or scaling up funding and/or
resources to match the type of results you are
expected to achieve by the end of the investment.
9
QUALITY OUTCOMES & RESULTS FRAMEWORKS
Finding a Common Language to Discuss Results Frameworks
The Outcome Investing practice involves conversations between Progam Officers and partners to define
and align on the content in the Results Framework. However, the foundation and our grantees often use
different terminology and/or have varying definitions of the components in a Results Framework (e.g.
outputs, intermediate and primary outcomes). The nuanced differences and similarities highlighted
below is intended to help clarify the foundation’s language and expectations around the Results
Framework.
Other Organizations’ Results
Frameworks
BMGF Results Framework
Scope of
Results
Framework
Other organizations’ Results
Frameworks articulate everything
from resources/inputs to ultimate
results and impact
The foundation intentionally focuses on
outputs and outcomes (primary and
intermediate), to allow the PO to focus
on and manage to investment results
rather than investment activities
Definition of
Outcomes
Outcomes across organizations
broadly mean the same thing: the
change in behavior or systems that
the project/investment seeks to
achieve
In addition to the basic definition of
outcomes, the foundation articulates
both “primary” outcomes (i.e. overall
changes) and “intermediate” outcomes
(i.e. short- to- medium changes) in the
Results Framework
Linking
Outcomes to
Higher-level
Results
Many organizations require
articulation of the logical link between
outcomes and a higher level change or
impact; e.g. USAID’s Results
Framework links to a Country
Development Cooperation Strategy
Goal
In the foundation’s Results Framework,
the higher level change an investment is
contributing to is represented by
strategic goals at an initiative, subinitiative, or portfolio level, and is
described in the Strategic Fit section;
there is no expectation to measure
impact or the strategic goal at the
investment level
Articulating
Assumptions
Most Results Frameworks require
articulating assumptions (the general
conditions under which the Results
Framework will hold true)
The foundation’s Proposal Narrative and
Results Framework do not include a
section referred to as “assumptions”;
instead, the Proposal Narrative includes
a few optional questions on “critical
relationships” and “external factors”
that address areas commonly included
as assumptions
10
QUALITY OUTCOMES & RESULTS FRAMEWORKS
Table 2 provides a more detailed comparison of our results language to some of our partner
organizations, for reference.
Table 2. Terms and Definitions across Different Organizations
Organization
The Bill &
Melinda Gates
Foundation
USAID Results
Framework
Components of the Results Framework
Strategic Goal
Primary Outcome
The 3-5 year results
of an initiative, subinitiative or portfolio
to which the
investment
contributes most
directly (identified in
the strategy score
card)
Overall change(s) in
technologies, systems,
populations or
behaviors the
investment seeks to
achieve within the
context of the
investment timeframe
Development
Objective(s) (DO)
Intermediate Results
(IRs)
Sub Intermediate
Results (Sub IRs)
The highest level
objective that a
Mission can affect
and for which USAID
is willing to be held
accountable
Interim events,
occurrences, or
conditions that are
essential for achieving
the DO. IRs and sub-IRs
are results that
together, are sufficient
to achieve the DOs. In
most cases, IRs serve as
the basis for “projects”
that will be designed
and evaluated
Outcomes
contributing to
Intermediate
Results
*Also / Formerly
called Assistant
Objectives (AOs)
DFID Results
Chain
Intermediate
Outcome
Short- to mediumterm changes in
technologies,
systems,
populations or
behaviors that need
to be achieved in
order to realize the
primary outcome(s)
Outputs
The goods,
services, events,
or deliverables
produced during
an investment
Output
Products or
services produced
as a result of
internal activity
Impact
Outcome
Output
The higher level
situation that the
project will
contribute towards
achieving
The outcome will identify what will change and
who will benefit
Specific, direct
deliverables of the
project; outputs
provide the
necessary
conditions to
achieve the
outcome
11
QUALITY OUTCOMES & RESULTS FRAMEWORKS
World Bank’s
Results
Framework
CARE’s Project
Outcome
Model
UNICEF
The W.K.
Kellogg
Foundation
Strategic Objective
Impact
Outcome
Outputs
Higher-order
changes in systems,
communities, or
organizations
Evidence on whether
outcomes are actually
changing beneficiary
behavior or longerterm conditions of
interest
Benefits of outputs to
the target population
Particular goods
or services
provided by an
intervention
Impact
Effects
Outputs
Equitable and
durable
improvements in
human wellbeing
and social justice;
typically, “projects”
aim at levels of
impact that can be
manifested during
their lifetimes, given
their resources
“programs” aim at
“higher level,”
longer-term impact
Changes in human behavior or in the existence
and quality of systems; effects are what
beneficiaries do on their own (influenced by the
project’s outputs as well as external factors)
The goods and
services resulting
from project
activities
Strategic Result
(Goal or Intended
Impact)
Key Result
Outcomes
Outputs
The change to whose
achievement a
program has made a
major contribution;
one should be able to
track and report key
results
United Nations
Development
Assistance Framework
(UNDAF) outcomes
(results) and Country
Program outcomes
(results)
Products and
services, whose
attainment
depends on and is
mainly
attributable to the
implementing
agency
The expected change
in the lives of
children and women;
it provides direction
for the overall
program
Impact
Outcomes
The fundamental
intended or
unintended change
occurring in
organizations,
communities or
systems as a result
of program activities
within 7 to 10 years
The specific changes in program participants’
behavior, knowledge, skills, status, and level of
functioning; short-term outcomes should be
attainable within 1 to 3 years, while longerterm outcomes should be achievable within a 4
to 6 year timeframe
Outputs
Direct results of
program
activities; usually
described in terms
of the size and/or
scope of the
services and
products delivered
or produced by
the program
12