SPTF Annual Meeting 2015: Day Two Notes
Client Outcomes Measurement Workshop: Savings outcomes
This workshop discussed how financial institutions can measure and encourage client savings
habits. Examples from CARD bank (Philippines) and EFC Zambia illustrated different ways of
measuring savings outcomes using simple yet innovative methods, including using mobile SMS
technology, transactions data, and even an RCT.
Speakers: Alex Fiorillo, GRID Impact; Anu Valli, Bamboo Finance; Frances Sinha,
EDA Rural
Anu Valli of Bamboo Finance worked with EFC Bank in Zambia.
In 2009, EFC transformed to be able to take savings. The savings accounts had low balances
(less than $200) and about 35% of the almost 25,000 savers were women. Most of the MFI’s
credit customers were men. Bamboo Finance wanted a robust, low cost methodology to track
outcomes rather than outputs. The budget for the research was $5,000 so they decided to use a
survey delivered through mobile phone technology. The main research question for the
investor was: Do savings products create positive impact on clients? At the same time, the MFI’s
objectives for the survey were centered on how to increase savings deposits among their
customers. The survey sought to address both these research goals.
The mobile phone based survey was offered by TTC (Text to Change) and the mobile operator
offered a monetary incentive to users for their participation in the survey of $0.75. The money
was provided as a credit on the mobile phone usage of the customer. A mobile phone survey
must be short – no more than 12 questions. (For survey questions see the PPT). The survey was
delivered to both clients and non-clients of EFC and the survey took approximately 20 minutes
to complete. The survey questions were designed through a collaborative process between
Bamboo Finance and EFC Bank.
For statistical significance, EFC needed to obtain at least 400 responses and so TTC sent the
survey out to over 20,000 people and only 1002 people responded. Of those who responded
400 were clients and 602 respondents were non-clients. Both sets of samples had about 35% of
female clients. Because the survey response rate is quite low, the MIV wondered if there could
be a bias introduced or a type of mobile customer than was more likely to answer this survey
than others e.g. better off, less entrepreneurial, less rural, and these characteristics may be
different from their average customer. But given the budget constraints this was as close as
they could get and the results were useful so better than not having the data, which was the
alternative. EFC Bank was quoted a price of $90,000 for the same type of study but not done
through the mobile platform.
Challenges:
 Budget – there was only $5,000 for the survey.




Survey had to be short to accommodate mobile platform, so they were limited to 10
questions.
Synchronizing the goals of the three organizations – Bamboo Finance, EFC Bank, and TCC
TCC included lots of clients in the survey that were in areas where no EFC branches were
located. This made the results less useful to the MFI and is a lesson learned – next time
make sure they only send the survey to non clients who are potential clients because
they are located in a region where the bank operates.
Bamboo Finance was organizing the project and designing the questions but they were
not on the ground so it was hard to get close to the client and understand the real
situation.
Results:
 55% of the users most recent withdraw transaction had been used to invest in the
business.
 73% of those considered that their savings had helped to grow their business.
 Customers’ complaints centered on a lack of understanding of the terms (many did not
know their accounts accrued interest), a lack of respect, and a need for more convenient
access to ATMs.
 The MFI identified areas for implementation of improvements that included a need for
staff training for savings outreach as it differs from credit promotion.
 The MIV had to convince the MFI to apply the results of the survey to enhance their
operations.
Alex Fiorillo of GRID Impact worked with CARD Bank in the Philippines
GRID Impact is a data consulting firm that worked with Grameen Foundation and Grameen’s
affiliate CARD Bank to analyze why customer’s were not using their savings accounts. CARD had
previously participated in Grameen Foundation’s three-year Gates funded savings project to
help them develop two new savings products, including an ATM based account. However most
clients did not open savings accounts and many clients (72%) who opened accounts never used
them, not even once after opening the account. This study sought to answer why this was
happening and what could be done by the MFI to encourage usage of the accounts and more
extensive savings by their members.
Because of loyalty to CARD, most of the clients who opened saving accounts felt like to be good
members they should participate in all of CARD’s products. However since this was the clients
only reason for opening the account, they did not use the accounts. Those who used the
account even once usually continued to use the account in the future, usually with one
transaction every 2 weeks. However even those who used the accounts often did not have a
savings goal in mind and many continued to save the majority of their surplus income in
informal ways.
GRID Impact carried out a low cost version of a Random Control Trial (RCT) using the MIS data
from the MFI as a baseline to save on performing an additional baseline study. CARD’s MIS was
greatly enhanced during the Gates Foundation savings project, which made this possible. Grid
looked at both analysis variables (transaction and administrative data) and outcomes variables
(see PPT for full list of outcomes variables). In addition to the streamlined RCT, Grid Impact
performed “immersive behavioral research” - which consists of ethnography, observation, and
in-depth interviews - with about 80 staff and clients of CARD. The research methodology uses
human centered design to look at the motivations of clients in how and why they open savings
accounts for both the clients that used the account to its maximum potential and those who did
not use it at all.
The “RCT Lite” was conducted over 6 months to see if the “treatment” designed by GRID Impact
would produce the desired results. The treatment was piloted in 3 branches and for these
branches the consultants reviewed all transactional data for the previous year as well as
monthly data during the study, including end of month summaries and weekly data on new
account openings. The treatment consisted of several interventions bundled together and
included:
 establishing a savings goal with the client,
 developing a savings plan to help the client reach this goal,
 simplifying the savings account application form (from a 5 page form in English to a one
page form in the local language),
 a gift for opening the savings account e.g. CARD branded calendar, and
 SMS reminders about the savings account and savings goal for both staff and clients.
The treatment was applied for 2 months and then the results were monitored for 4 months.
The treatment group was around 1,900 people and included three groups: those without savings
accounts, those with dormant savings accounts, and a control group.
Challenges:
 CARD’s MIS is product based rather than client based, so there is no way to look up a
client by name and see all the products they have at one time. Each product has its own
database and these are not linked.
 GRID required that CARD not introduce any new products in the pilot branches during
the 6-month term of the study, however CARD did launch a new product in those
branches and it affected the data collection. To manage this problem, GRID selected a
subsample of the original sample to retain only those clients that were unaffected by
the launch of the new product. This resulted in a smaller sample size than anticipated
but due to the original over-sampling the total number of clients in the subsample was
still large enough to be statistically relevant.
 On an operational level, the weekly transfer to transactional data from CARD was often
late, which delayed the researchers’ timeline.
 There was no point person from CARD to support this research project consistently and
learn the researchers’ processes so that the work could continue internally after the
project was over.
Results:
 GRID produced a data dashboard with a summary of the key information from the MIS
data for CARD.
 Clients in the treatment group were 73% more likely to USE their savings account
(indicated by performing at least one future transaction after initially opening the
account). They were also 60% more likely to continue to use the account actively e.g.
more than one future transaction.




Clients in the treatment group made 15% larger deposits than the control group.
Of the clients in the treatment group, 37% had higher balances at the end of the pilot
than those in the control group.
Clients who already had accounts at the start of the pilot made larger deposits and
larger withdrawals, and in general used the account more actively than those in the
control group.
Of the set of interventions in the treatment group, CARD decided to implement on an
ongoing basis the new application form and the institution retrained staff to talk to
clients differently about savings, including the idea of setting a savings goal when
opening the accounts.