DEMOCRACY AND GOVERNANCE:
…Realities behind the facts and figures
BROWN BAG FORUM (BBF)
Nigeria MEMS
Presenters:
Aliyu Aminu Ahmed
Ibrahim Sanusi
Snr. Monitoring and Evaluation Officer
National Democratic Institute (NDI)
Monitoring and Evaluation Officer
International Republican Institute (IRI)
Disclaimer: This presentation is entirely the opinion of presenters. It does not represent the opinion of individual
organizations they work for or USAID. It is intended to stimulate debate and brainstorming among colleagues
OBJECTIVES OF THE PRESENTATION

To stimulate discussion amongst BBF participants
towards improving or enriching M&E systems in
USAID Democracy and Governance (D&G)
Projects in Nigeria.

To elicit inputs and suggestions from colleagues
towards improving the D&G M&E system in Nigeria

To further enrich general understanding of the
issues/challenges posed in M&E for D&G programs
in Nigeria and proffer possible solutions with
colleagues
INTRODUCTION








USAID D&G Programs
USAID/CEPPS (Consortium for Elections and Political
Process Strengthening) Partners for D&G in Nigeria
USAID/CEPPS Activities (Amongst others)
USAID/CEPPS Target populations in Nigeria
USAID/CEPPS Anticipated results
National Democratic Institute (NDI) programs in Nigeria
International Republican Institute (IRI) programs in Nigeria
International Foundation for Electoral Systems (IFES)
program in Nigeria
USAID D&G Programs
USAID: SO11: Strengthened Foundations for Democratic Governance
Program Goals: Governing Justly and Democratically (GJD)
To build the capacity of civil society and legislative bodies, strengthen the
judicial processes, support efforts to respond to and manage conflict, and
address the need for professionalism and transparency in the future elections.
USAID/CEPPS Partners for D&G in Nigeria
CEPPS: Consortium for Electoral and Political Processes Support (CEPPS)
CEPPS Partners:
National Democratic Institute for International Affairs (NDI),
International Republican Institute (IRI),
International Foundation for Election Systems (IFES)
USAID/CEPPS Activities (amongst others)
-
Elections Assistance and Support (CEPPS)
Support to the Electoral Process in Nigeria (ELECTIONS)
National Assembly Strengthening and Support (NASS)
USAID/CEPPS Target populations in Nigeria
-
Government organizations/Government officials (e.g. INEC)
National and state houses of assembly/ Legislators and staff
CSOs in democracy and governance (Coalitions, networks …)
NGOs (with emphasis on People with disabilities, women and youths)
Political parties/Political party officials
Media ( reporters, editors and journalists)
USAID/CEPPS Anticipated Results





Participation by stakeholders in proceedings of the Electoral Reform
Committee (ERC)
A strengthened legal and regulatory framework for elections
Greater political party competitiveness
Successful coverage of the campaigns and elections by international and
domestic observers
Increased participation by people with disabilities, women and youths in the
electoral process
Background on NDI in Nigeria
The National Democratic Institute is a nonprofit, nonpartisan organization
working to support and strengthen democratic institutions worldwide through
citizen participation, openness and accountability in government
Founded in 1983,
Established by US Congress
Works in 5 continents
Presence in over 60 countries
Done work in over 110 countries
Works with governments,
parliaments, political parties and
civic groups
Started work in Nigeria since
Nigeria’s 1998-99 transition
from military to civilian rule.
-
-
-
NDI programs in Nigeria
-
Legislative Strengthening and Anti-corruption Program (LSAP)
(enhancing
-
the capacity of the NA to legislate effectively, conduct oversight of the executive branch and representation/CO)
ELECTIONS- Strengthening Electoral Processes
Strengthening National Assembly Programs (SNAP)/DFID ended 03-2009
Background on IRI in Nigeria
The International Republican Institute (IRI) is a nonprofit,
nonpartisan organization, advances freedom and democracy worldwide
by developing political parties, civic institutions,
open elections, good governance and
the rule of law
 Founded in 1983
 Established by US Congress
 Stated work in Nigeria since 1998.
 Presence in over 66 countries
IRI was established amongst other things to achieve the following:
-
Monitoring the electoral process.
Enhancing the capacity of political parties
Encouraging greater participation of women and youth in politics
Forging stronger ties and developing collaborative efforts between
political parties and civil society organizations.
IRI program in Nigeria





Supporting Political Parties’ contribution to promote electoral reform;
Increase cooperation between the Electoral Reform Committee (ERC) and political parties
Expand the participation of women and youth in the political process and reform phase
Examine and enhance the role of the media throughout the reform process.
Supporting marginalized groups (People with Disability – PWD) as they develop issuebased campaigns.
Background on IFES in Nigeria
The International Foundation for Electoral Systems (IFES)
is an independent, non governmental organization providing
professional support to electoral democracy. Through field
work, applied research and advocacy, they strive to promote
citizen participation, transparency, and accountability in political life
and civil society.
Founded in 1987
Established by US Congress
Started work in Nigeria since 1998.



IFES programs in Nigeria
•




Civil Society: civic and voter education, rights of marginalized groups
(including people with disabilities), victims of violence…
Elections: election administration, election operations, electoral integrity,
campaign finance, electoral system design…
Governance: legislative strengthening, anti-corruption efforts, money and
politics…
Rule of Law: legal reform, access to justice…
Applied Research: survey research, research studies …
CHALLENGES/LESSONS LEARNED
Challenges/lessons learned
1. Challenges in the M&E system/structure
2. Challenges working with stakeholders
3. Challenges using the M&E Documents
- PMP – Performance Monitoring Plan, - OP – Operational Plan/Standard Indicators,
- PDT – Performance Data Table, - PIRS – Performance Indicator Reference Sheet
4. Lessons learnt in relationship between personnel (Programs, Finance &
M&E)
5. Lessons learned on M&E Personnel capacity/training
6. Lessons learned on Data collection culture and processing
7. Challenges of obtaining Means of Verification (MOV) for data/attribution
to indicators
8. Lessons learned on Documentation, feedback and data storage/records
9. Perceptions/Relationship M&E, Home Office, MEMS and USAID
Challenges in the M&E system/structure
• Fact 1: Performance M&E is based on information from
essentially two sources (1) A monitoring system
and (2) An evaluation system both are essential, but they are not the
same.
• Our M&E system is essentially focused on Performance Monitoring
Plans (PMP) and a little on “Performance Evaluation Plans (PEP)”!
• Fact 2: There are two important documents in a project (1) Technical
Proposal and (2) Financial proposal
• Fact 3: The technical proposal contains (Program Description, Goals,
Objectives, activities and M&E Plan)
• We realized that project proposal focus on “program Description” mainly
with little attention initially on the M&E component. Mentioning lofty M&E
strategies e.g Appreciative enquiry, FGDs etc… without clear description
on where ,when and why we will use these concepts.
• M&E staff hardly have access to financial proposal/burn-rates
Continued….
•
Our Main Challenge: The M&E system struggles to capture quantitative
indicators with minimal interest in qualitative indicators. We recognize that
tracking qualitative is expensive and time consuming but we need them!
•
Standard GJD indicators are quantitative, with constant changes
(e.g. coding, definitions) and usually inadequate to capture all that is
intended to be tracked. Hence partners have a task of developing
own indicators which vary in definitions and characteristics
•
Formative/Baseline data are rarely available and is not used as a benchmark for performance
evaluation or tracking due to poor data culture in Nigeria
•
Some indicators are entirely dependent on political will and buy-in to be achieved e.g.
Milestones achieved towards NABRO
•
Sometimes targets are set either very high or unrealistic during project design, as achievement
of most targets are based on political will and buy-in because of the dynamic nature of the
environment
We track most complex and difficult indicators, individuals and unpredictable situations
•
•
Work plans often change to reflect constant changes of stakeholders/National Assembly
erratic time lines /the fact that D&G issues change rapidly in Nigeria
•
There are often more organization specific indicators (Unique Indicators) to SO level/GJD
Indicators. Organizations are interested in not how many people trained but what have they
done with the training!
•
Donors are interested in more numbers!
Actions taken
 We involve stakeholders in project design/re-design to get their
reflect their priorities , get their commitment and political will
and buy-in which is often documented in writing
 We ensure that stakeholder have strategic plan embedded in their
structures/committees so that we can follow up
 In addition to SO indicators, we try to get appropriate qualitative and
quantitative organization specific indicators and define them
appropriately
 Where funding is not available or program description changes (e.g.
sub-grants, political environment ) we inform USAID and drop the
indicators/reflect this on PMP
 From inception of the project we develop good, workable and detailed
work plan which we adhere to subject to change in program
description
 We develop and review PMP that is linked to work plan, program
description and funding available
 We try to link the 3 key documents M&E , finance and programs to
ensure conformity, consistency, accountability and transparency
 We set “critical-assumptions” in the program description e.g. political
will/buy-in
Challenges working with stakeholders

Sovereignty issues: we only build capacity but do
not enact or get laws passed

It takes awhile to get NA commitment

During a typical project period say 5 yrs (with possible extension)
we record high turn over of NA members and
political party leaders, this have programmatic
and M&E implications

Projects often rely significantly on individual relationship with MPs and when
they go it affects the project, and especially institution memory is lost
Institutional relationship can not be guaranteed due to the turn over of MPs
in the National Assembly



MPs are somewhat on “parallel terms” with CSOs. e.g. they perceive CSOs
as making unnecessary demands or championing opposition agenda .
MPs do not perceive CSOs as representatives of “peoples voice”

Working with stakeholders (especially NA members) is challenging as they
are unpredictable in terms of timing and honoring invitations (e.g. meetings).

It takes along time to schedule a training for committee members (6
months!!!)






A lot of effort is invested in planning process but adjournment affects
implementation (e.g. preparation on house retreat )
Stakeholders come with their aides (e.g. NA members often insist on coming with
their aides, disabled people come with aides ….) Who are not our target group
(e.g. 20 members with 20 aides = 40!).
Stakeholders perception on support and technical assistance is varied. E.g. NA
members prefer “foreign trips” to other capacity building activities
The house of assembly early recess and erratic timetable do affect program
timeline
Some National Assembly members prefer communicating with home office/HQ
Stakeholder are high profile people they prefer traveling first class, using
presidential suites and may not sign attendance sheet or have time for questions.
This affects programming and consequently M&E
Actions taken…






We engaged more committees/stakeholders than planned so that we address the
issue of institutional memory lost
We insist on evidence based documentation from the NA
We facilitate meetings between stakeholders e.g. CSOs and NA targeted
committees
We follow up with stakeholders (e.g. MPs, PWD) directly or through their aides to
ensure timely participation
We sometimes shift our work plan to reflect the changing situations, timelines
etc…of stakeholders
We also engage past NA members in capacity building activities
Challenges using the M&E Documents
(PMP, OP Standard Indicators, PDT, PIRS)

Some program personnel think that M&E documents are cumbersome

Understanding the M&E terminology/acronyms by program staff e.g. PMP,
PTD, Mechanism, Measures, milestones, …
Frequent changes in indicator definitions (is it National Assembly? Is it states
assembly? Is it consensus building or consensus building processes?)
Sometimes programs will tell you that they do not know issues they need to
track. It is only during reporting they realize (e.g. M&E is interested in number of
government officials but programs will report number of National Assembly
Members/staff excluding others from MDAs, sometimes M&E will be interested
in knowing # of PWD, youths participants etc
Sometimes funding organization demands for some results which were not
initially planned. E.g. comprehensive documentation of lessons learnt were
specifically required after close out
Changes in program description/focus usually affects what
is needed to be tracked
Sometimes we delete indicators and add new ones.
Some indicators can not be achieved within
project period due to lack of political will, time and funding
e.g. NABRO






Actions taken…
 Regular in-house meetings, brain storming session with
program and finance staff
 Meetings with programs , M&E and finance to review new
program descriptions and targets
 We have improved on our reporting systems to capture detailed
success stories, lessons learnt and verbatim quotes (e.g. the
recent W/Shop by IRI/NDI and youth CSOs in Jos Labor was
motivated to take up Electoral Reform issues on May Day)
 We develop organization specific indicators and identified
relevant SO GJD indicators to track new program description
 We ensure consistency in documentation (program report
figures= PMP figures) and
 We report changes against each indicator
Lessons relationship between personnel (PM, FN & M&E)

M & E and Finance:


Subsuming of M&E finances within project implementation
budget usually restricts M&E activities and influencing M&E
objectivity. E.g. priority is given to program visits/activities
than M&E visits
M & E and Programs:









Program staff often complain that the “M&E language” is ambiguous
Program view M&E as only there to “track numbers” not willing to engage
them in planning and implementation
There is sometimes erroneous assumption that project personnel are already
doing the monitoring hence M&E will only collect, analyze, summarize and
present data/results
Usually program staff have a designate person to collect data for central M&E
with minimal orientation/skills on M&E
Change in program focus do pose a challenge to the program staff in
attributing existing indicators to new program area and they inform M&E late
Two projects may be implemented by one partner (e.g. SNAP, LS all work
with NA) but there is difficulty in coordination amongst the program staff.
It is difficult measuring and quantifying some outcomes that are important to
projects.
The ability to generalize program results from one program to another can be
limited or impossible, considering the heterogeneity of data collected.
Limitations in attributing causes for a result to one program is difficult as many
other projects do work with stakeholders/National Assembly (e.g. UNDP)
ACTIONS Taken …
- M&E is careful in reporting results from different projects
to avoid double counting/double funding issues
- M&E , Programs and finance staff work hand-in-hand to
develop monthly activities and monitor burn-rates
- Coordination meetings are held by program staff from
different programs and M&E to avoid double counting
and attribution of wrong results
- Program staff are required to submit monthly financial
required
Lessons learned on M&E Personnel capacity/Training
•
•
•
•
Most personnel learn M&E on the job
Weak competency of program staff acting as M&E point persons
There is capacity gap and need for training and re-training of program/finance
officers on M&E
D&G program staff are mostly “arts inclined” hence weary of data analysis
Actions taken…
•
We hold once in a while in-house training session for program and finance staff
on data collection, documentation and analysis
•
Projects send Key M&E staff for training and re-training (e.g. USAID MEMS Evaluation
Training Kaduna)
•
We advocate for budgets to train staff on M&E in internationally recognized
training
Lessons learned on Data collection culture /processing






Difficult data culture environment among stakeholders affect quality of
data generation and collection (for example when we ask how many
sittings at a committee we need MOVs e.g. agenda)
CEPPS partners share events/activities but there is a challenge in
reporting/results sharing by individual organizations
Working with secondary data (as Nigerians have poor data collection
habit) hence secondary data in unreliable
Capturing data from activities of sub-grantees
Selecting appropriate tools for data collection
Data storage and back up
Actions taken…
 We organize joint meeting to share and report results and attributing
numbers (e.g. National Dialogue on Electoral Reform NDI is interested in
# of govt. officials but IRI is interested in # Political parties/grouping
represented, PWD etc…)
 We have trained sub-grantees on data collection and capture the data
appropriately
 We avoid data received by phone call we verify them with documented
evidence
Challenges of obtaining Means of Verification
for data /attribution to indicators



Program staff may be asked to track verbatim quotes by participants, they
will do that without providing reliable MOVs e.g. recorded conversations
National Assembly contacts would give us numbers of sitting the held within
a period but unfortunately they do not give us the MOVs e.g. Agenda for us
to know whether during these meetings they have actually discussed e.g.
Electoral Reform matters
Ooops how do we capture “people have learnt the are of democratic
process” – this is qualitative/survey related!!!
Actions taken…
•
All data reported are verified and for every trainings/w/shops conducted we
have attendance sheets, training curriculum, agenda, presentations and
report
•
We cut newspaper clips on events conducted
•
We also documents pictures of activities
•
We do not document verbal results until we get verifiable records
Lessons learned on Documentation, feedback and data
storage/records


Inadequate feed back mechanisms and failure to
involve stakeholders in M & E
Documentation of success stories (just a sentence
and no antecedent facts) in our previous reports
Actions taken…
•
Three levels of data storage are currently available.
1. Hardcopies in Country and home office
2. Electronic copies in Country and home office
3. Back-up copies : In databank, MEMs and USAID
•
Data quality check is ensured with requisite MOVs
before storage
•
We improved on our success story writing and lessons leant reporting to be
more detailed and result focused
•
We share reports (especially to assemblies) to enable make changes
Perceptions/Relationship M&E, Home Office, MEMS
and USAID
M & E , HQ and USAID/MEMS
 The issue of setting good indicators in-house vs the USAID
SO GJD indicators.
 Setting indicators that may not be tracked e.g. Milestones on
NABRO or ensuring passage of Election Reform bill has
potential sovereignty issues
 CEPPS partners are not contractors.
 Obligations and type of agreements (grants, cooperative
agreement, contract)
 Understanding CEPPS Partners routing processes.
USAID/MEMS may require M&E results or other information
within short notice (e.g. 24 hrs) and this has to be coordinated
through the home office which would require more than 1
week
 Even though MEMS is a good system but it is unique to
Nigeria
 Perception that partners are slow in submitting targets
Actions taken…..



Regular visits to MEMS to seek for advice
Coordinate meetings/conference calls
between MEMS D&G desk officer and home
office
Improvement in timely submission of targets
and results to MEMS
Prospects/suggestion for D&G M&E system
 Expected
from participants
Conclusion/Appreciation



We thank all the participants for attending and
contributing
We appreciate the support of USAID in funding the
project and promoting democratic and transparent
governance in Nigeria
We appreciate the technical support provided to us
by Nigeria MEMS
Thank You