Inter-Action EPEWG Meeting
“Measuring Agency-Level Results:
Can it work, and does it matter?”
Tracking Beneficiaries
and
Services Delivered
Harry (“Hap”) Carr, Senior Technical Advisor for M&E
Catholic Relief Services (CRS)
April 10, 2013
Beneficiary and Service Delivery Indicators
(BSDI)
Measure Agency-wide Performance
on
CRS Fundamental Program Outputs
Services Delivered
and
Beneficiaries Served
Inter-Action EPEWG Meeting Presentation
Types of Benefits in CRS Projects
Direct Beneficiaries
BSDI
Indirect Beneficiaries
Benefits /
Services
Policy Advocacy
Environmental Protection
Institutional Strengthening
Inter-Action EPEWG Meeting Presentation
Not
BSDI
CRS Services and Benefits
Eight Current CRS Program Areas:
Microfinance
Agriculture
Health
HIV-AIDS
Education
Peacebuilding
Water and Sanitation
Emergency
Inter-Action EPEWG Meeting Presentation
Water and Sanitation
Inter-Action EPEWG Meeting Presentation
Emergency Services and Beneficiaries
Illustrative Service Categories
Program
Area
SubProgram
Area
Service
Emergency
Shelter
Transitional
Shelter
Means of
Direct
Indirect
Service Beneficiary Beneficiary
Delivery Definition Definition
Training
Technical
Assistance
Assets
(Shelter Kit)
Commodities
Other
Inter-Action EPEWG Meeting Presentation
Individual
household
member
BSDI Data Flow Map
Point of Service Delivery
Typically data are recorded on
paper for individual beneficiary
Beneficiary and service
delivery data are registered
Regional/Partner Office
Beneficiary and service delivery
data are recorded and records
are updated
Typically data are stored
in a project database
Typically data are
transcribed to Excel files
Project Management Office
Beneficiary and services delivery
data are stored, maintained,
analyzed and reported
On-line Agency
Database (PIMS)
Donor
Inter-Action EPEWG Meeting Presentation
BSDI Performance
Measurement Products
BSDI allows accurate, annual agency-wide tracking of CRS performance
“Number” and “% targeted” of direct beneficiaries served” by
 Sub-program area service categories
 Geographic location (GPS) point of service delivery
 Gender and other beneficiary demographic characteristics
BSDI allows integrated technical and financial performance analysis
“Costs per beneficiary by service”
BSDI will track “double counted” beneficiaries in a multi-sector projects
Title II Project beneficiary of nutrition, agriculture and microfinance services
Inter-Action EPEWG Meeting Presentation
BSDI Actions in the Last Year
Field Validation Visits
West Africa region projects – March 2012
Latin America And Caribbean region projects – May 2012
Pilot Test of BSDI – June to December 2012
CRS Emergency projects in the Sahel Crisis Area
Coordination with other CRS data management initiatives
Project Information Management System
East Africa M&E/ICT initiative
Data Stewardship Group
Comparison with Other Agencies
Save the Children “Beneficiary Reach”
International Red Cross “People Reached Indicator”
USAID/State Foreign Assistance Framework – Elements and Sub-Elements
Inter-Action EPEWG Meeting Presentation
BSDI Catapult
March 2013 CRS Board Meeting
and
FY2012 CRS Annual Report
FY2012 Beneficiary Survey
Inter-Action EPEWG Meeting Presentation
FY2012 Beneficiary Survey
Short-fused request for FY12 beneficiary data:
From CEO – so it was important
Final data needed within 2 ½ months from request
Data to include Direct and Indirect Beneficiaries
By major CRS program areas
To be submitted through the CRS financial data reporting system
On newly designed screens and procedures for data entry
Three weeks were allowed for field data collection and reporting
Current CRS procedures for collecting beneficiary data:
In the BSDI Catalogue of Services and Beneficiaries –
Service area categories had been largely formulated
Most Direct Beneficiary definitions had been formulated
Few Indirect Beneficiary definitions had bee n formulated
There were no common procedures for beneficiary data collection
Inter-Action EPEWG Meeting Presentation
Beneficiary Survey Lessons Learned
Lessons Learned… for the type and level of detail in guidance
How best to formulate guidance for data gathering and data entry
Common definitions of Indirect Beneficiaries for similar projects was difficult
There were some problems with the eBudget software BUT
Data can be easily entered into the system
Lessons Learned…regarding our reporting of beneficiaries:
Beneficiary data reported in the past were generally –
Estimated from budget reports and were
Over-estimates
There is value in having accurate, timely beneficiary data
Inter-Action EPEWG Meeting Presentation
Output versus Outcome Indicators
There has been an on-going discussion at CRS –
Not formally organized,
Part of various data management efforts going on
In which the importance of measuring CRS results was paramount
We have tacitly settled on –
The primary importance of Beneficiary Data
In the longer-term goal of Outcome Indicators
…For selected selected program areas
Inter-Action EPEWG Meeting Presentation