Presentation - Better Evaluation

advertisement
Pact MERL webinar series
Use of Mobenzi Researcher in the
2012 MERL Communities of Practice
Lessons from Tanzania and Namibia Experiences
September 19, 2012
Jacqueline Ndirangu – Pact Namibia
Daisy Kisyombe – Pact Tanzania
What is Mobenzi Researcher?
• A technology that allows you to collect data using a
mobile phone.
• Data is uploaded and transmitted to a central server
using internet connection.
• Data from the server can easily be exported to excel and
SPSS for analysis
Process of setting up Mobenzi Researcher?
• Procured handsets that are compatible with the Mobenzi
Researcher technology and set them up with the Mobenzi
Researcher software
• Developed the questionnaire as usual (in MS Word) and
emailed the same to Mobenzi support personnel to set up for
use on the handsets
• Surveys were set up online with technical support from the
Mobenzi team
• Mobenzi support team can help to register each of the mobile
phones and issued log in and password information for
individuals authorized to view/work with the data online.
• In case you use local language; Mobenzi team sends English
questionnaires in Excel for translation and then set up online
Survey tool design
Survey tool design….
Survey on the Phone
Data collection with Mobenzi
Researcher
• For optimum results; a central controller is needed with
reliable internet access to monitor data entry and alert
on issues emerging as data comes in
• Once questionnaire is downloaded on the phone; data
can be collected offline and stored on the phone till when
phone is on network coverage area
• Data stored on the phone automatically uploads into
Mobenzi once network is recovered
• Be sure to give each field worker a phone that is
specifically assigned to him/her (no mixing up)
Mobenzi online dashboard: Analytical
Overview
Mobenzi online : Grid view
Advantages and disadvantages of
Mobenzi vs. paper based surveys
ADVANTAGES
DISADVANTAGES
Time saving (1258 households interviewed in
Tanzania in 4 days) cutting off massively on
time for data entry.
Not suitable for qualitative data collection
Affordable costs ( USD 2600 for platform
usage for Tanzania baseline including 5
surveys; USD 575 for Mobenzi survey set up
for Namibia)
Allows instant addressing of data errors while
team is still in the field
May be expensive initially (if the costs of the
phones cannot be spread out to other program
activities)
Allows technologically advanced data
collection e.g. digital photo, GPS coordinates
for maps alongside your data
Data can be synchronized to other databases
if needed
May create negative perceptions for
communities that are not aware of technology
Data quality and security assurance
• Built in quality checks ensuring no incomplete
submissions
• Data in server can only be viewed/edited by Pact
authorized personnel
• All edits are recorded including person who did the edit
and when. Original data submission is not lost in case
you need to reverse the action
• Data entered via phone once submitted cannot be viewed
from the phone
• Data in server is encrypted and password protected
What worked well?
• Easy to set up the phones with the Mobenzi Researcher
software
• Assigning of phones and codes to each of the data
collectors
• Easy to update the survey when changes were made after
the pilot test
• Data collection was standardized and systematic for all
data collectors
• Data export from Mobenzi server in Excel to SPSS for
analysis
What did not work so well?
• Limited time available to train data collectors and allow
time to familiarize with using the technology.
• Handsets were not GPS enabled
• Collection of the ‘Other’ data. Typing of text was time
taking, hence resorted to recording and transcribing
responses to ‘Other’ questions.
• Battery life for some type of phone not able to sustain
long hours of data entry
Tanzania results overview
OUTCOME RESULT
AREA
Child protection
FINDINGS
Education
Psychological well being
Transition to next level of education -18.7% - 42.8%
Hope for future - 27.7% - 38.4%
Stigma feelings - 9.9% - 11.2%
Sleeping hungry >10 times in 4 weeks – 11.4% - 29.5%
Household food security
Child labour (working and not attending school) - 9% - 18%
Health
Under 5 fully immunized 50.8%
Chronic illness 39.4% - 47.9%
HIV+ caregivers 12.7% - 28.8%
Household economy
75% subsistence farmers
5.2% (m) and 5.7% (f) caregivers have no income at all
Vulnerability status
Female headed households - 58.8%
44% Orphans
Namibia results overview
Evaluation Issue
Key findings
Capacity Development More years of support = lower usefulness rating, but
Effectiveness
higher OPI scores
Pact and partners see usefulness differently over the
duration of support
Graduation to Direct
Funding
Graduation prerequisites: Strategic planning,
governance, finance, admin, technical capacity and
M&E support
Key post-graduation needs: technical and resource
mobilization support
Lessons Learned for
Sustainability of capacity building depends on whose
Capacity Development decision it was to engage in capacity development
Organizations
On average partners tended to score lower in resource
mobilization on OPI scoring
Way Forward
• Future evaluations e.g. upcoming Tanzania baseline in
program expansion area/ mid term evaluation
• Global indicators data entry
• WORTH data collection
Thank you!
Download