Impact Evaluation Concept Note
South Africa
Matching grant scheme
Gabriel Makhonxa
Global Workshop on
Development Impact Evaluation
in Finance and Private Sector
Rio de Janeiro, June 6-10, 2011
Intervention Summary


Black Business Supplier Development Programme (BBSDP)
is matching grant scheme offered to black-owned small to
medium enterprises in South Africa.
The objective of BBSDP is to assist business to improve their
competitiveness with the intention to bid successfully for
tender opportunities created by both the private and the
public sectors. The BBSDP to be evaluated is an upgraded
version of a prior programme with the same name.
Intervention Summary

Eligible Enterprises
(1) Black-owned businesses (50+1%)
(2)Management team predominately black
(3) Registered companies
(4)Trading for at least a year
(5)Turnover of R250 000 to R35 million ($37000 to $5
million)
Intervention Summary

Eligible Expenditure
(1)Total amount you can apply for is R1 million
($143,000)
(2) Cost-sharing of 50:50 for equipment/technology
(3) Cost-sharing of 80:20 for Enterprise Support
Services (training, software, ISO, business planning,
etc)
(4) Enterprises can apply for only 25% of its previous
turnover
Evaluation Questions
How effective is BBSDP in enhancing the
performance (productivity, sales, profits,
employment) of black-owned small enterprises in
South Africa?
 What is the return on investment of the program?
 Are the effects of BBSDP different across different
groups (sector, gender of the owner, region)?
 Are the effects different between applying costsharing and full payment? (IE design later)

Story so far
Started engaging with World Bank about the impact
evaluation of this matching grant from mid-2010.
 Program was soft-launched in late 2010 under more
restrictive selection criteria. The submission for the
final terms of the criteria of eligibility have been
recently been approved by the Minister.

Initial challenges: take-up
POPULATION OF ALL BLACK BUSINESS
SUPPLIERS
FACTORS INFLUENCING APPLICATION
Criteria: Initial criteria with minimum R
500 thousand annual turnover
Criteria: Cost sharing (initially 50/50 for ESS)
Impact evaluation: fear of not receiving
grant (initial design involved
randomization)
Marketing: Promotion coordination
problems
NF training: Delayed accreditation
Sampling and Data
Comprehensive list of small businesses that are potentially
eligible for BBSDP (30,000 businesses)
 Random assignment of 10 000 enterprises.
 Process of listing potentially eligible enterprises to be
repeated every 9 months (except for initial sample(s))
 Screening of enterprises in each window period and active
promotion
 Promotion to include incentive with 1 in 5 chance of getting
full payment
 Network Facilitators to assist interested businesses in
applying for BBSDP
 After 9 months, compare outcome for 1st and 2nd group of
applicants. Compare three targeted groups at endline.

BEE: 16,000
firms
Introye: 50,000 firms
SAWEN:
1,000
firms
Evaluation Design
SEDA: 9,500
firms
Desktop pre-screening for eligibility
New Impact
Evaluation Design
Sample frame:
30,000 firms
Updated every window
Window 2
10,000 firms
(9 months after)
Window 3
10,000 firms
(18 months after)
TIA Screening
and incentive promotion
TIA Screening
and incentive promotion
TIA Screening
and incentive promotion
600 w1 applicants
600 w2 applicants
600 w3 applicants
Survey with recall to baseline
Survey with recall to baseline
w1 applicants
w1 applicants
w2 applicants
w2 applicants
Window 1
10,000 firms
w3 applicants
Develop and prescreen firm
databases; structure
screening and
incentive guidelines
for TIAs
Train TIAs
TIA offers to
provide
supporting
documentation
and NF support
Firm
Interested
Y/N
NO
Retain list of all
businesses
contacted but
not interested
YES
Provide contact
details to
designated NF for
follow up PSC
and application
Provide
TIAs with
list of
business
contacts
TIA outlines
BBSDP in more
detail to business
and provides
incentive to apply
early**
TIAs make
contact with
firms to verify
eligibility*
YES
* Eligibility criteria questions:
1. Majority black ownership
2. Length of time business has
been operating
3. Turnover
4. Simple PSC pre-checks to
determine if firm is likely to
pass the PSC
Firm
eligible
Y/N
NO
Business
recorded
and marked
off the list
**Incentive:
1. Financial incentive: applicant
has a 1 in 5 chance of
receiving full cost borne by
DTI rather than cost-sharing
Timeline
Activity
Pre-pilot targeted marketing (400 contacts)
Final lists of businesses and sampling
Targeted marketing first window
Targeted marketing 2nd window and midline survey
Timing
July 2011
August 2011
September 2011
June 2012
Midline impact evaluation report
August 2012
Targeted marketing 3rd window and midline survey
March 2013
Final impact evaluation report
July 2013
Team and staffing
Team Member
Organisation
Gabriel Makhonxa
DTI
Nonceba Mashalaba
DTI
Varsha Harinath
DTI
Madeleine Maree
DTI
Francisca Strauss
DTI
Francisco Campos
WB
Aidan Coville
WB
Ana Fernandes
WB
Sabrina Roshan
WB
Budget
Team Member
Organisation
Targeted Marketing (I, II, III)
$10,000
Midline survey
$100,000
Endline survey
$130,000
Field coordinator, travelling, other costs
$50,000
Impact Evaluation Concept Note
South Africa
Matching grant scheme
Gabriel Makhonxa
Global Workshop on
Development Impact Evaluation
in Finance and Private Sector
Rio de Janeiro, June 6-10, 2011