Outcomes Working Group
Background - Social Performance Task Force 2014 Annual Meeting
Plenary Session: Top Ten Lessons on Measuring Outcomes
(held June 2014 in Dakar; speaker was Amelia Greenberg)
In preparation for its 2014 annual meeting, the Social Performance Task Force (SPTF) reviewed
reports of many microfinance institutions that have attempted or are attempting to track clientlevel outcomes. It also interviewed financial services providers, investors, TA providers and
researchers who have experience with collecting and analyzing client-level outcomes data. This
brief sets out the background, and the Top Ten lessons that emerged, leading to the creation of
the Outcomes Working Group.
BACKGROUND
Some financial service providers (FSPs) pursue a double bottom line, seeking to achieve
financial success and to contribute to positive changes in clients lives. SPTF contends that by
implementing strong social performance management (SPM) practices, including defining and
tracking indicators to measure the changes in the lives of their clients (i.e., client-level
"outcomes"), FSPs better position themselves actually to achieve their social goals. This is
because without reliable data on how customers’ lives are affected by products/services,
businesses cannot know whether their products and services have helpful effects or the
reverse, much less how to improve them. On the other hand, by clearly defining and
systematically tracking specific outcome indicators, such as enterprise growth, household
poverty level, assets, health, and women's empowerment, FSPs can understand how they are
doing, and how to do better.
However, data collection can
be complicated, and FSPs have
limited resources and many
competing priorities. With
these constraints in mind, the
Social Performance Task Force
(SPTF) devoted a plenary
session at its 2014 annual
meeting to discuss some of
the practical lessons that have
emerged from measuring
outcomes to date.
These are listed alongside, and
briefly described on the next
page.
TEN LESSONS ON MEASURING
OUTCOMES
1
•Stakeholders need to understand what is realistic
2
•RCTs are not the whole answer
3
•Use multiple tools
4
•Methods: credible & relevant, but affordable & relatively simple
5
•Involve field staff for inputs
6
•Client self perception is also a valuable input
7
•Do not expect 100% improvement
8
•Analyze outcomes by client segment
9
•Use data to monitor social goals - and to improve
10 •Keep employees in the feedback loop
1. Educate external stakeholders. Many external stakeholders are unrealistic about what
changes to expect over what time frame. FSPs and others should be clear on what is
realistic in terms of the extent of positive outcomes and the time involved.
2. RCTs are not the whole answer. Randomized control trials (RCTs) are complex, require a
large sample and a limited time frame. They may have particular uses, but appear to be too
expensive for the typical FSP to undertake on its own.
3. Collect outcomes data that are credible and relevant but affordable and simple to collect.
Data quality must be reliable. However, the data collection process must not overburden
the institutional capacity. It should be affordable and relatively easy to execute.
4. Ask field agents for input. Field agents are often able to suggest good outcomes indicators
to track. And, one Freedom from Hunger study found that answers given by field agents
closely matched household survey information from clients and bank records.
5. Use multiple tools to measure changes. We can make better use of a range of available
methodologies to understand whether and how an FSP's products and services are helping
clients: e.g., a quantitative measurement tool such as the PPI or PAT, combined with other
relevant indicators, and client focus groups or case studies to understand more deeply the
process of change. You can find different tools in the SPTF Implementation Guide.
6. Self-perception can be an additional valuable source of outcomes data. Client selfperception can be a valid and inexpensive way to gain additional insights.
7. Report honestly. No institution can achieve positive outcomes for 100% of clients, and
investors and donors know this. One investor reported feeling more comfortable investing
in an FSP when it provided complete and accurate information on defaults, than previously
when the FSP had shared only positive stories.
8. Analyze outcomes by client segment. Different types of clients can be affected in very
different ways by the same product or service. Averages are not as useful as an analysis for
different types of clients (‘segmented’ by e.g., gender, geography, livelihood) to understand
which services work for whom and in what context. More on this in this CGAP Focus Note.
9. Use outcomes data to monitor achievement of social goals and to improve. Reporting
outcomes data keeps an institution accountable to its social mission and demonstrates the
extent to which the institution is achieving its social goals. Analyzing who is benefitting from
the use of a financial product and who is not, and what led to these changes, gives the FSP
insight into adjustments it might make to its products and services to improve client
outcomes.
10. Use monitoring to increase employee satisfaction. Employee feedback shows that
monitoring outcomes is important to employees because it helps them better understand
clients and it motivates them to see positive change in their clients' lives.
OVERALL
Reliable outcomes data should help providers answer the fundamental question: are clients’
lives improving? FSPs need not be afraid of finding surprising or negative results, but should
strive for an accurate understanding of client conditions. This knowledge is a powerful
motivator for operational and product changes that can lead to improved client outcomes.
SPTF launched the Outcomes Working Group in October 2014 to continue our learning in this area.
The working group is open to all stakeholders :practitioners, networks, TA providers, investors,
researchers, subject matter specialists – academics.
Contact us to share your experience and views: info@sptf.info