Impact
BRAC Research Impact Assessment
Summarized by BRAC USA
www.brac.net
Why this exercise?
Q. Why do donors contribute to an
organization?
A. Awards, word of mouth, testimonials,
charity ratings, preferred habitat
(programs, geography, culture, religion,
ethnicity), vanity, emotive photos/stories,
scale, quantifiable impact.
www.brac.net
Quantifiable impact is (becoming) the
gold standard for NGO evaluation
• More and more funders are asking for proof of
effectiveness by demonstrating a quantifiable
impact
• BRAC’s independent research department is
unrivaled in the magnitude of research produced
• Development organizations ignore quantitative
impact at their peril
• Easier said than done
www.brac.net
What is Impact?
Impact
“The positive and negative, primary and secondary long-term
effects produced by a development intervention, directly or
indirectly, intended or unintended.”
-- OECD Development Assistance Council
Impact Evaluation
“A study of the attribution of changes in the outcome to the
intervention.“
-- 3ie
www.brac.net
Impact research should be
independent
Real life example of research bias
www.brac.net
Reports Used
Program
Title
Time Period
Agriculture
Impact of BRAC’s Agriculture & Livestock Programme in Uganda
2009-2010
Education
Economic and Social Analysis of Primary Education in Bangladesh: A Study of BRAC
Interventions
2009-2010
Education
An Assessment of BRAC Pre-Primary Schools Environment
Health
Health
Health
Health
Health
Manoshi: Community Health Solutions in Bangladesh. Impact Evaluation Surveys in Dhaka
Urban Slums, 2007, 2009, & 2011
Maternal, Neonatal, and Child Health Programmes in Bangladesh: Review of Good Practices
and Lessons Learned
The Pilot Maternal, Neonatal and Child Health Project (MNCH) at Nilphamari: Profiling the
Changes During 2006-07
Sustaining Health: The Role of BRAC’s Community Health Volunteers in Bangladesh,
Afghanistan, and Uganda
Impact of Community-based Tuberculosis Programme: Ex
2012
2007-2011
2006
2006-2007
2008-2009
N/A
Microfinance
The Wider Impacts of BRAC Poverty Alleviation Programme in Bangladesh
1992-2004
Microfinance
An Impact Evaluation of BRAC’s Microfinance Program in Uganda
2008-2009
Ultra-Poor
Addressing Extreme Poverty in a Sustainable Manner: Evidence from Ultra-Poor Programme In
Bangladesh
2002-2008
Ultra-Poor
Can Basic Entrepreneurship Transform the Economic Lives of the Poor?
2007-2011
Ultra-Poor
Asset Transfer Programme for the Ultra Poor: A Randomized Control Trial Evaluation
2007-2011
Achievements of BRAC Water, Sanitation, & Hygiene Programme
2006-2011
Youth
Social Network and Financial Literacy among Rural Adolescent Girls: Qualitative Assessment of
BRAC’s SoFEA Programme
2011-2012
Youth
Empowering Adolescent Girls: Evidence from a Randomized Control Trial in Uganda
2008-2010
Water & Sanitation
www.brac.net
Impact Synopsis
Agriculture & Livestock
 Positive impact on usage of modern inputs, cultivation methods, and
vaccination services (Uganda)
 Program participants are 27 percent more likely to use improved
seeds than the comparison group (Uganda)
Education
 BRAC Pre-primary schools: Scores 7.7% higher than non-BRAC
schools
 Reduced Dropout and Repetition, Rate, Higher Pass Rate: BRAC
schools are twice as effective as government schools in terms of
lower dropout, higher pass rates and repetition rate
www.brac.net
Impact Synopsis
Health
 Increases in MNCH knowledge, including danger signs for pregnant women,
new mothers, and newborns
 Increase in uptake of services, practicing essential newborn care, and
increased access and follow-through of referral
 Improved health metrics including: ANC and PNC visits, increase in use of
Post-partum hemorrhage or PPH (the leading cause of maternal mortality)
medication, increase in percentage of institutional delivery and an increase
in percentage of women practicing exclusive breast feeding.
 Reduction of MMR (Maternal Mortality Rate) from 236 per 100,000 live
births in 2008 to 141 live births in 2010.
www.brac.net
Impact Synopsis
Microfinance
 Better child survival, nutritional status, family planning, and education in
households served by the program.
 In Uganda, a Study published in 2013 revealed positive impacts including
increase in savings, improved food consumption and increased financial and
social resources leading to improved business startup and investment
Ultra Poor
 Positive long-term impact on the lives of the ultra poor, even four years after the
end of the program’s two-year support phase, in terms of per capita income,
employment, food security and asset holding
 Among participants that entered the two-year program in 2002, 92% satisfied at
least 7 out of 10 indicators of extreme poverty in 2008.
 By 2008, six years after the baseline survey, the program’s impact on
participants’ per capita income had almost doubled compared to the impact
from 2002 to 2005.
www.brac.net
Impact Synopsis
Water & Sanitation
 Sanitary latrine use increase: Households using sanitary
latrines increased significantly from the baseline (32%) to
midline (41%) to end line (53%)
 Reduction in water related diseases: The prevalence of water
related diseases significantly reduced from 9.4% in the
baseline to 7.1% in midline and 2.3% at the end line.
www.brac.net
Impact Synopsis
Youth
 An RCT shows Uganda's ELA program having positive impacts on life
and vocational skills, health-related knowledge, risky behavior, early
childbearing, and engagement in income generating activities.
 A study of the SoFEA program in Bangladesh found that the
program was successful in increasing self-confidence and
motivation, and in strengthening the relationship between girls and
their parents and within the community.
 Initiative to encourage adolescent girl’s socialization and improve
financial literacy has been found to be “quite effective”.
 After receiving Financial Literacy Training (FLT), SoFEA members
were more financially aware and many were involved in Income
Generating Activities, such as livestock and poultry rearing.
www.brac.net
Agriculture and
Livestock
www.brac.net
Agriculture & Livestock
Impact of BRAC’s Agriculture & Livestock Programme in Uganda
Timeframe of study: 2009-2010
 Positive impact on usage of modern inputs, cultivation
methods and vaccination services
 Participants 27 percent more likely to use improved seeds
than the comparison group
 Improved cultivation methods: Participants are six percent
more likely to follow line sowing cultivation and 10 percent less
likely to follow inferior mixed-cropping
 Participants are 24 percent more likely to utilize vaccination
services
 Participants are 16 percent more likely to be ‘market-oriented’
resulting in higher revenues from sales
www.brac.net
Education
www.brac.net
BRAC Education Program Model
www.brac.net
Education
Economic and Social Analysis of Primary Education in
Bangladesh: A Study of BRAC Interventions
Timeframe of study: December 2009 – February 2010
Value for Money: Cost per student completed is 2x in formal schools.
 BRAC schools boost test scores: BRAC pre-primary students enjoyed ↑ in
final exam scores in primary school by 7.7% compared to comparison group
higher as well as ↑ secondary school enrollment (89% v. 82%).
 BRAC schools have reduced dropout rates & higher completion rates,
making them twice as effective as government schools.
 Cost-Benefit: Private returns to primary education are higher for BRAC
graduates, due to the 4 year cycle & lower cost of BRAC schools, compared
to the 5 year cycle of mainstream schools
 Internal Efficiency: BRAC schools advantage includes intensive monitoring
of teachers & teacher input.
www.brac.net
Economic and Social Analysis of Primary Education in
Bangladesh: A Study of BRAC Interventions
Positive Effect of BRAC Pre-Primary Education on the Transition to Secondary Schools
100
88.6
90
80
83.8
82.2
72.6
70
60
50
40
30
20
10
0
Did not attend pre-primary
All Pre-Primary
BRAC Pre-Primary
Mainstream Pre-Primary
%enrolled in secondary
www.brac.net
Education
An Assessment of BRAC Pre-Primary Schools Environment
Timeframe of study: July – September 2012
 Poor Overall Environment: In a 7point scale, BRAC schools received an
average score of 1.47
 On the other hand, POs considered
that the majority of schools were
rated as good or very good.
Personal care, lang. learning, interaction, parents & staff, parts of activities can be
improved with little additional monetary investment.
Do BRAC Pre-Primary Schools Have a Positive Impact?
•
•
•
Shahjamal and Nath (2008) compared performance of former pre-primary students
enrolled in formal primary schools and a comparable group who did not have a preprimary background, and found that former BRAC students did significantly better in
Grade I, but no difference was found in other grades
Ahmad and Haque (2011) also find that scores achieved in primary school
examinations are higher for students with BRAC pre-primary education
Are these results contradictory?
www.brac.net
Education - Summary from Andrew Jenkins
Reaching Primary Education at the Doorstep of the Poor: The BRAC
Experience
Timeframe of study: 2005
 Over 2.4 million children completed the primary education curriculum, due to BRAC’s educational
interventions
 6.5% of total primary school students were enrolled in BRAC schools and over 90% of BRAC students went
on to enroll in formal secondary schools
 BRAC students had a better knowledge of health & development issues than formal school students
Socioeconomic Impact of BRAC’s Non-Formal Primary Schools
Timeframe of study: 2005
 BRAC primary students enjoyed better performance compared to formal school students.
 NFPE school attendees were significantly more knowledgeable on child immunization/prevention of disease
than those who attended government primary schools or never attended school at all

A smaller proportion of adults who attended government primary schools had immunization cards (57.9%)
compared to NFPE school enrolled households (64.3%)

NFPE school enrolled couples collected birth control at a higher rate (37.4%) than those who attended
government primary schools (32.1%) and who never attended a school (26.9%)
www.brac.net
Health
www.brac.net
Health
Manoshi: Community Health Solutions in Bangladesh. Impact
Evaluation Surveys in Dhaka Urban Slums, 2007, 2009, & 2011
Timeframe of study: 2007-2011
 Increases in MNCH knowledge. Though improvement varied depending on the particular
aspect of knowledge. For example, knowledge about recommended 4+ ANC visits increased
from 45% in 2007 to 59% in 2011 in the project area v. 50 to 52% in comparison area.
 Increase in uptake of services, practicing essential newborn care and increased access and
follow-through of referral. For example, colostrum feeding increased 32 absolute percentage
points (PP) in the project area v. 18 PP for the comparison area.
 Improvement in multiple health practices: ↑ ANC & PNC visits, ↑ use of Misoprostol (drug used
for post-partum hemorrhage -leading cause of maternal mortality), ↑ in percent of institutional
delivery & % of women breast practicing exclusive breast feeding.
 For example: Women receiving PNC after delivery improved 28PP in project area v. 7 PP in
comparison area.
 ↓Maternal Mortality Rate from 236 per 100,000 live births in 2008 to 141 in 2010, as a result of
household services, timely referral, & coordinated support at referral facilities. (Source:
Manoshi: A Community-based Solution to Avert Maternal Death, 13th ASCON 2011).
www.brac.net
MMR declined by 50% in Manoshi
areas
www.brac.net
Health
Maternal, Neonatal, and Child Health Programmes in Bangladesh:
Review of Good Practices and Lessons Learned
Timeframe of study: 2006
 Saving Newborn Lives Program: A
collaborative program Funded by BMG
implemented by BRAC, et al demonstrated
changes in many areas of maternal care:
SNL Program has Positive Impact on
Post-Natal Checkup
 Post-Natal Checkup increased by 37.5
to 61% from the baseline survey
 Newborn checkup within 24 hours
increased from 14.4% in 2002 to
27.3% in 2004
 ANC increased from 22% in 2002 to
92% .
*National averages sourced from UNICEFhttp://www.unicef.org/infobycountry/bangladesh_bangladesh_statistics.html) and the World Bank
(http://data.worldbank.org/indicator/SP.DYN.CONU.ZS?page=1)
www.brac.net
The Pilot Maternal, Neonatal and Child
Health Project (MNCH) at Nilphamari:
Profiling the Changes During 2006-07
Timeframe of study: 2006-2007
 ↓ in induced abortion. However, self-treatment and treatment-seeking from informal
providers for complication of abortion remains high.
 Mean number of ANC visits ↑ with increased proportion attended by skilled providers
relative to baseline.
 Intake of iron tabs and TT vaccination increased.
 Majority of deliveries continued to take place at home. Though deliveries by trained
TBAs increased since baseline, the proportion delivered by the TBAs remained
substantial.
 For treatment of neonatal complications, mothers increasingly preferred UHCs/district
hospitals.
 Management of illnesses of the under-fives is fragmentary and the thrust is on
prevention only.
www.brac.net
Health
Sustaining Health: The Role of BRAC’s Community Health Volunteers in
Bangladesh, Afghanistan, and Uganda
Timeframe of study: 2008-2009
 Dropout rate of 12% low compared to similar programs internationally.
 Self-reported motivations for CHPs primarily related to financial incentives in all
settings except Afghanistan - women see this role as income-generating
opportunity:
• 75% of SSs reported that their income makes a big difference. In Uganda
and Afghanistan these numbers were 80% and 15%, respectively. 97% of SS
in Bangladesh reported that being an SS gave them financial independence
 BRAC is meeting the demand for medicines, but CHWs in all three countries felt
competition from pharmacies, private clinics, and other providers.
 Importance of increased Social recognition: 18% in Bangladesh, 11% in Uganda
and 16% in Afghanistan
www.brac.net
Health
Sustaining Health: The Role of BRAC’s Community Health Volunteers
in Bangladesh, Afghanistan, and Uganda
www.brac.net
Health
Impact of Community-based Tuberculosis Programme:
Experience of BRAC
Timeframe of Study: 2008-2009
Cost effectiveness: In BRAC areas, cost was $64 compared to $96 in other
areas.
 BRAC jointly expanded DOTS services to cover a population of 89.5 million,
with services mainly provided by CHWs
 In 2009, 96,427 patients were diagnosed, of them, 71,946 were new
sputum-positive
 Strong case-detection rate of ~80% and treatment success rate of 93%.
 CHPs fill critical gaps in human resources at community level; reduce delays
in diagnosis, bring doorstop DOTS service, and ↑ case-detection and cure
rates
www.brac.net
The Bangladesh Miracle –
Significant drop in mortality rates and increase in life expectancy
www.brac.net
The Bangladesh Paradox –
Source: the Lancet
The Bangladesh Paradox: Exceptional health achievement despite economic poverty
 Success may be attributed to a multi-stakeholder pluralistic health system.
 Women-focused, equity-oriented, nationally targeted programs in family planning,
immunization, oral rehydration therapy, maternal and child health, tuberculosis, and
vitamin a supplementation were critical success factors
 Bangladesh’s approach to equity and its widespread deployment of female community
health workers to bring essential health services to local households is noteworthy and a
“testament to how many services can be provided when there is a scarcity of credentialled
health professionals”
BRAC and The Bangladesh Paradox
 After the War of Liberation, the government created the space and donors provided the
financial support for NGOs like BRAC, which, as a group “have innovated to address issues
or poverty, unemployment health education, and the environment, and in many cases, the
government and NGOs have worked together to achieve a common goal.”
 The national implementation of an oral rehydration therapy program by BRAC enables
mothers to prepare homemade oral rehydration saline
 Scaling up of innovations has been strengthened by investment in research, which has
enhanced program design, monitoring & evaluation, and implementation
www.brac.net
Microfinance
www.brac.net
Microfinance
The Wider Impacts of BRAC Poverty Alleviation Programme in
Bangladesh
Timeframe of Study: 1992-1995
 ↑ child survival and nutritional status in households.
 Protein-Energy Malnutrition (PEM) prevalence declined from 23% to
12%.
 Also improvements in expenditure patterns, per capita calorie
consumption, family planning practices and children’s education.
 BRAC households % using family planning was 57% v. 49.6% for
poor-non BRAC members.
 Violence against women initially increased when credit was
introduced but later tapered after other services like training were
introduced.
www.brac.net
Child survival rate better for BRAC
members
www.brac.net
Microfinance
An Impact Evaluation of BRAC’s Microfinance Program in Uganda
Timeframe of Study: January 2008- May 2009
 The study finds that BRAC’s program “seems to confer significant
positive benefits to borrowers” including:
• Increase in total savings (1.63 log change in savings) and assets
(0.76 log change)
• Greater consumption in the form of more expensive and nutritious
food (increase of 0.3 meals per week consumed with fish)
• The resources and incentives to start a household business
(Household business ownership increased by 27%)
 Results vary considerably depending on the statistical technique used.
This coupled with data issues makes it difficult to make causal claims.
 One year between baseline and follow up, may not be adequate.
www.brac.net
Ultra Poor
www.brac.net
Ultra Poor
Addressing Extreme Poverty in a Sustainable Manner: Evidence
from Ultra-Poor Programme In Bangladesh
Timeframe of study: 2002-2008
Key Findings:
 Positive long-term impact, four years after the end of the two-year support phase, in terms of per
capita income, employment, food security and asset holding
 92% of the ultra-poor households satisfied at least 7 out of 10 indicators of extreme poverty in 2008
v. 64%.
Other Findings
Income/Employment:

By 2008, participants’ per capita income had almost
doubled. The study calls this “remarkable,” because
it “indicates that impact on per capita income was
not only sustainable but also increased over time”

Among working-aged female participants, farm selfemployment rose from 0.2% in 2002 to 37.0% in
2008, versus 0.3% to 11% in the control sample.
Positive Impact on Farm SelfEmployment
www.brac.net
Ultra Poor
Addressing Extreme Poverty in a Sustainable Manner: Evidence
from Ultra Poor Programme In Bangladesh (Cont.)
Assets:
 Financial assets increased sharply compared to the control sample, for years after the end of
the 2-year program, attributable to a “dramatic” change in savings behavior & participation in
the microcredit market (11, 24)
 The percentage of women with cash savings increased from 8% at the start of the program in
2002 to 98% in 2008, versus 21% to 34% in the control group
Food security:
 By 2005, self-perceived food insecurity “had fallen tremendously” for participants, or 47%
compared to the 2002 baseline versus only 11% drop in the control
Housing & sanitation:
 By 2008, 94% of participants had roofs made of tin and 57% of participants had a sanitary
latrine, compared to 44% and 2% in 2002
Education:
 The program “did not have significant impact on education in the short term” but “in the
long-term a modest positive impact on boys’ net primary enrollment was observed.”
www.brac.net
Ultra Poor
Addressing Extreme Poverty in a Sustainable Manner:
Evidence from Ultra-Poor Programme In Bangladesh
(Cont.)
www.brac.net
Ultra Poor
Can Basic Entrepreneurship Transform the Economic Lives of
the Poor?
Timeframe of study: 2007-2011
Cost of evaluation: ~$2 million
 Annual income increased 38% over 4 years.
 Food security improved 18pp after two years, and 8pp after four years,
corresponding to a 39% and 18% increase from the baseline.
 After 2 years, self-employed women in the treated communities increased
20pp from the baseline, compared to only 6.2pp increase in control group.
This difference further improved at 4 year point.
 After 2 years: ↑ ownership of livestock despite being free to dispose of
these assets. Chicken ownership ↑ 2.3pp, v. a .2pp increase in the control
group. Most chickens were retained at the four-year point.
 Businesses are successfully retaining and expanding livestock assets even
as BRAC has withdrawn direct assistance.
www.brac.net
Ultra-Poor - Summary from Andrew Jenkins
Change in Food and Nutrient Consumption Among the Ultra Poor: Is the CFPR/TUP
Programme Making a Difference
Timeline of Study: 2001-2004
Key Findings: A significantly higher % of TUP households consumed meat and fish regularly (>= 4
times/week), compared to control households. A relatively large number of TUP households never
consumed eggs, milk or fruit, however, which has an impact on the overall nutrient intake.
Impact Assessment of CFPR/TUP: A Descriptive Analysis based on 2002-2005 Panel Data
Timeline of Study: 2002-2005
Key Findings:
 Participants are more likely to have savings, taken a loan, & be informed of laws. Selected
households more likely to possess assets such as cows & furniture.
 Savings and asset accumulation are precursors for ownership of land and significant
improvements in housing
 “The program has contributed significantly to improving the lives of the ultra poor and helping
them help themselves”
www.brac.net
Water and
Sanitation
www.brac.net
Water & Sanitation
Achievements of BRAC Water, Sanitation, & Hygiene Programme
Timeframe of study: 2006-2011
Key Findings:
 Sanitary latrine use increase: from the baseline (32%)
to midline (41%) to end line (53%)
 Hand washing ↑: Hand washing practice with soap at
‘critical times’ significantly increased from baseline to
end line. At end line 97% of respondents had
knowledge about hand washing with soap before
eating and after defecation, and 22% and 88%
reportedly did so at that particular time.
 Reduction in water related diseases: The prevalence
of water related diseases ↓ from 9.4% in the baseline
to 7.1% in midline and 2.3% at the end line (see detail
next slide).
www.brac.net
Water & Sanitation
WASH program significant reduction in water related disease
prevalence
www.brac.net
Water & Sanitation
Achievements of BRAC Water, Sanitation, and Hygiene
Programme (Cont.”)
Other Findings:
 ↑ in hygienic practice in terms of covering water vessels during
transportation and storage. The improvement in knowledge and practice
in drinking pure water could be a result of the health education provided
by the WASH program.
 Although the prevalence of water-related diseases significantly reduced
amongst children over and under five-years-old, the prevalence of waterrelated diseases continued to be highly pronounced among children under
five.
 Evidence suggests it is likely that a substantial number of
mother/caregivers do not wash their hands with soap/ash after fecal
contact and before feeding a child.
www.brac.net
Youth
www.brac.net
Youth
Social Network and Financial Literacy among Rural Adolescent
Girls: Qualitative Assessment of BRAC’s SoFEA Programme
Timeframe of study: April 2011- September 2012
Key Findings:
•
•
•
•
↑ in self-confidence and motivation and in strengthening the relationship between girls and
their parents and within the community and improved socialization.
After receiving Financial Literacy Training (FLT), SoFEA members were more financially aware
and many were involved in Income Generating Activities, such as livestock and poultry rearing
The fact that many of the girls are now supporting themselves financially has empowered them
immensely, not only within their own families but also within the community as a whole.
Concerns:
– Time constraints for active participation—girls stopped attending regularly during exam
periods and harvest season.
– Early Marriage The fact that rural Bangladeshi women tend to marry quite young poses a
socio-cultural challenge to the SoFEA program; after they marry they tend to drop out of
school to focus on post wedding responsibilities and no longer maintain a wider network
of friends.
– Father opposition: biggest opponents to club participation were the fathers. Girls
mentioned during sessions that even some educated fathers forbid daughters from going.
www.brac.net
Youth
Empowering Adolescent Girls: Evidence from a Randomized
Control Trial in Uganda
Timeframe of Study: 2008-2010
Cost Effectiveness of program: Cost of $17.9 v income gains of $26.7
Life skills:
 Condom usage ↑ 50% points among sexually active.
 Fertility rates were 29% lower compared to a control sample.
 Participants’ reports of having sex unwillingly decreased by 83% from the baseline – a
clear indicator of success in empowering youth girls.
 Girls in treated communities who agree that “females should earn money for the family”
increased by 18 pps from the baseline level of 37%.
Vocational Skills:
 The likelihood of being engaged in income generating activity ↑ 35% over baseline levels.
 Personal consumption expenditures ↑ 33% from the baseline.
 No adverse effect on schooling: The ELA program does not appear to have an adverse
effect on schooling investments made by participants.
www.brac.net
www.brac.net