Primary stakeholders participating in M&E

advertisement
PRIMARY STAKEHOLDERS PARTICIPATING IN M&E
There are different rationales for primary stakeholders participating in M&E:

Increasing primary stakeholder participation in an M/E activity contributes to the quality of the results. It
increases objectivity by achieving balanced analyses, recognising biases and reconciling perspectives of
different stakeholders and results in greater accuracy, completeness and fairness of M&E results.

Increasing primary stakeholders’ participation in M&E is consistent with a human rights-based approach
to programming. The process of M&E can contribute to empowerment and realisation of primary
stakeholders’ right to participate, which is established in both the Universal Declaration of Human Rights
and the Convention on the Rights of the Child.
DIVERSITY AND DISPARITY AMONG PRIMARY STAKEHOLDERS
Like any other group, primary stakeholders are not homogenous regardless of programme and context.
Whether primary stakeholders are women, children, poor families, or communities affected by humanitarian
crisis, they may have some common interests, but sub-groups within will face diverse realities and will have
correspondingly diverse vulnerabilities and capacities. (See core content sheets Expanded VulnerabilityCapacity Analysis and Gender and Age)
Efforts to expand the participation of primary stakeholders without attention to participation within the group,
including how to address the particular constraints faced by less powerful and marginalised groups, can do
more harm than good. At the same time, working to expand the participation of any group of people
engenders responsibilities to protect them (See core content sheets Ethical issues for field study – Dealing
with people and Children participating in research, M&E: Ethics and your responsibilities as a manager).
EXPANDING PRIMARY STAKEHOLDERS’ PARTICIPATION IN M&E
Expanding primary stakeholders’ participation in M/E activities requires thinking through a strategy for
participation which has to feed into the strategy for participation for the programme as a whole. Elaborating
a strategy requires defining:





What approach to participation is appropriate?
Primary stakeholder participation can be characterised in different ways. In general, the variation is
according to whose interests drive – other stakeholders, others and primary stakeholders together or
primary stakeholders in first priority -- and what the objective of participation is – whether effectiveness
or strengthening local capacities. (See also Roger Hart’s scales of participation for children in core
content sheet Children participating in research, M&E: Ethics and your responsibilities as a manager)
Is it better to engage with primary stakeholders directly or indirectly, i.e. using existing structures?
Which other stakeholders will be involved?
What tools and methods are appropriate?
(Adapted from ALNAP (2003): p. 32)
Answering the above questions requires realistic analysis of: the general context, the primary stakeholders,
other stakeholders including your own organisation, and the nature of the M/E activity as it is defined so far.
The following are some of the key considerations for each, with some issues clearly overlapping.
The general context (stakeholders, cultural, institutional, geographic and security)


How will different stakeholders be affected by primary stakeholder participation? Who will win and lose
and how are winners and losers likely to use their influence? (More on other stakeholders below)
How do attitudes/beliefs, tradition and customs constrain or support primary stakeholder
participation, in general or by particular sub-groups among primary stakeholders? How could constraints
be overcome? Who could serve as a bridge?
UNICEF M&E Training Resource
Primary stakeholders participating in M&E 1/7






What informal or institutionalised processes or fora are in place to support participation, nationally,
locally or even within the programme (committees, networks, meeting places etc) ? Who has access and
who doesn’t?
What are the patterns of exclusion and discrimination – in general but also within primary
stakeholders? Consider gender, age, ethnicity, language, socio-economic divisions. How could these
patterns be countered? What are the risks that they will be exacerbated?
What is the space for and response to controversial views? What are the social risks to participating
and voicing different opinions?
Especially in crisis and unstable contexts, what are the security risks for primary stakeholders who
participate and how can these be avoided? This can include both untargeted violence to which
participants are more exposed (e.g. robbery or assault along deserted roads) or targeted reprisal for
participating. Or can participation increase protection of primary stakeholders and how can this be
managed? What measures are in place to protect primary stakeholders if their views are controversial?
Are there limits to physical access to primary stakeholders that will negatively affect and how can they
be overcome?
What is the history behind and evolution of all of the above and, particularly in unstable contexts, how
have these changed more recently?
Primary stakeholders








Among primary stakeholders, what are the attitudes to participation and to sharing information and
decision-making? Consider different attitudes of leaders, dominant groups and marginalised groups
among primary stakeholders?
How are primary stakeholders represented in local structures or networks? How is this representation
perceived? Are there sub-groups among primary stakeholders who are excluded?
Are there any opportunities to feed M/E activities into ongoing decision-making processes
involving or affecting primary stakeholders? (e.g. decentralised budgeting or assemblies)
How do primary stakeholders perceive other stakeholders in the programme? Is there trust/distrust?
Is there anything that can be done to restore the situation?
What is the history of past efforts at increasing participation? Is there ‘participation’ fatigue?
What understanding do primary stakeholders have of/how do they perceive M/E activities – an
opportunity for more services/resources, a threat? How will this affect their participation?
What capacities do primary stakeholders have for different roles in the various stages of M/E activities?
Where are the gaps/constraints? Consider attitudes as above, but also organisational structures, skills
and knowledge and time.
How do primary stakeholders perceive the risks to participation – see above social and security risks?
Other stakeholders (including your organisation)







What other stakeholders should be involved in the M/E activity?
What are other stakeholders’ attitudes to participation of primary stakeholders? What does it mean to
them? How willing are they to be transparent and share information with primary stakeholders? To what
degree are more powerful stakeholders open to relinquishing control?
What motivates other stakeholders to get the participation of primary stakeholders? Consider:
organisational mandates and policies, any understanding of primary stakeholders right to participate,
concerns for completeness of information (e.g. needing primary stakeholders’ perspective on unintended
and negative effects of an intervention) or concerns about follow-up (e.g. where primary stakeholders
ownership is needed to ensure follow-up).
Are they interested and willing to invest in strengthening primary stakeholders’ capacities?
Are there any concerns about negative effects of primary stakeholders’ participation? Are different
stakeholders or organisations concerned about the independence and impartiality of the M/E activity?
Especially in crisis and unstable contexts, are international organisations concerned about losing their
independence and impartiality and getting involved with parties to a civil conflict or war?
Are other stakeholders aware of and sensitive to the expectations that will be created as primary
stakeholder participation increases? Are they ready to take responsibility for commitments made?
What are their attitudes to M/E methods that will likely be involved? Are there any standard M/E
methods and tools promoted by stakeholder organisations (or donors) that will limit primary stakeholder
involvement?
UNICEF M&E Training Resource
Primary stakeholders participating in M&E 2/7


How will participatory approaches to M/E fit with their existing M&E systems? Will they be an added
burden or can they be seen as a better substitute? How can different information needs be balanced in
an overall M/E activity or broader M&E system?
Where many other stakeholders are involved, how will primary stakeholder involvement fit in broader
coordination mechanisms?
Constraints on the M/E activity as it is defined so far








Is the focus of the M/E activity defined with reference to primary stakeholders’ priorities and
concerns or at least those of some sub-groups? If not, is there a subset of issues that are and that can
be managed as a separate more participatory facet of the M/E activity?
What are the numbers of primary stakeholders involved in the programme/project or affected by the
issue that is the focus of the M&E activity? The bigger the numbers, the harder it is to ensure
representative primary stakeholder participation, particularly where there are no existing institutions or
mechanisms for their participation?
Are there principles governing who participates that stakeholders, including primary stakeholders,
can all agree on? Can a human rights-based approach be adopted and how will the process be handled
if basic rights to participation are not acknowledged? Can any locally acceptable principles be found to
protect a space for the participation of marginalised groups?
What time and resources (human, financial) are available from different stakeholders to support
primary stakeholder participation? Can external M/E “experts” be found who understand their role as
facilitators of a process and have the necessary attitudes and skills to carry this out? Do those
managing the M/E activity have the necessary awareness, knowledge and skills to support a
participatory process?
To what degree can training and other capacity building be integrated into the M/E activity? How
much attitude change among all groups of stakeholders, primary and other, will be necessary to achieve
the level of participation desired and how realistically can this be built in to the process design?
Can the M/E activity be managed so that primary stakeholders see the results of their participation?
Can you provide quick and relevant feedback of findings locally? How long will it take for
recommendations and actions to be perceived in local level changes? How well can the M/E activity
balance feedback into immediate local learning and decision-making with feedback into learning and
decision-making at the level of wider programmes, organisations and institutions?
Can the M&E design be flexible, in terms of taking on and responding to new or different research/
evaluation questions in course, and/or adapting methods and instruments to different localities and subgroups?
What measures can be introduced to protect primary stakeholders and more vulnerable groups among
them and what capacity do organisations or committees managing the M/E activity have to ensure that
they are implemented? (See core content sheets Ethical issues for field study — Dealing with people
and Children’s participation in research, monitoring and evaluation (M&E): Ethics and your
responsibilities as a manager.)
Sources: ALNAP (2003); Ashton (1998); Davies (1997); Guijt, I. (1999); Harnmeijer, Waters-Bayer and Bayer (2000);
IFAD (????); Institute of Development Studies (1998).
UNICEF M&E Training Resource
Primary stakeholders participating in M&E 3/7
PARTICIPATION CONTINUUM IN M&E
WHEN AND IN WHAT ROLE DO PRIMARY STAKEHOLDERS PARTICIPATE?
Opportunities to expand primary stakeholders’ participation in M&E activities exist at two levels:
 The steps or activities in which primary stakeholders are involved — some steps or activities are more pivotal, giving stakeholders involved a greater ownership
of M&E results
 The role primary stakeholders have in each step or activity — roles entail higher or lower degrees of participation.
As you read the chart below, note that really high participation in some steps/activities, like data collection, may be a great achievement in a given context, but
cannot be expected to contribute greatly to empowerment if there are no efforts to increase participation in other more pivotal steps.
Steps/Activities
Why, what role, examples of how?
Considerations in
crisis and unstable contexts
Define the purpose
Participation in defining the purpose is important as the whole design should flow from this.
For emergency rapid assessment, and other
M&E activities in early phases after a crisis,
participation at this stage of an M&E activity
is likely over-ambitious. It might be
considered in chronic emergencies where
the context is right. See overall questions
above.
Range of roles:
High
empowerment
 High: P/S as active contributor — Using local institutions/organisations that represent primary
stakeholders, either taking negotiations of purpose for the M&E activity to these fora or inviting
representatives to be included equitably in a negotiation process. Some considerations: How can
the negotiation process be modified and facilitated to ensure that primary stakeholders’ voice is
heard, including sub-groups within? Are there institutions/organisations that can effectively
represent minority groups?
 Low: P/S providing validation — Using focus groups to bring in primary stakeholders’ perspective,
either feeding into the definition of purpose and/or critiquing what is already proposed. Again,
consider: How are focus groups structured to represent sub-groups among primary stakeholders?
How are different voices weighed?
In all cases in crisis and unstable contexts,
the purpose can be defined so as to include
reference to obtaining primary stakeholders’
perspective.
Where participation is not possible at this stage, greater involvement of primary stakeholders in later
stages is better ensured if the purpose includes reference to obtaining primary stakeholders’
perspective (e.g. on priority needs, on changes produced in the P/S’s situation).
UNICEF M&E Training Resource
Primary stakeholders participating in M&E 4/7
Focus the design
 define questions to be
answered
 define analytical
framework
 define methods/process
 define measures/
indicators
 define tools
High
empowerment
Collecting evidence
 data collection
Where primary stakeholders are not involved in design, the M&E activity could be biased, missing
important issues for the stakeholders. Primary stakeholders’ perspectives can challenge other
stakeholders’ more “established” analytical frameworks, bringing new perspectives on the interrelationship of different issues.
Range of roles:
 See roles under “Define the purpose” above
Where participation is not possible at this stage, M&E activities can be designed to accommodate
primary stakeholders’ proposed questions and measures as the data is being collected. This can be
done by including more open qualitative methods and tools. Techniques such as those drawn from
RAP and PRA will allow input by interviewees in data collection in terms of shaping the issues to be
explored.
For most M&E activities, at least collecting data from primary stakeholders is considered necessary
to ensure accurate, complete and fair results. Greater participation of primary stakeholders in
actually collecting data can add to this, particularly with qualitative methods. Primary stakeholders
may apply tools differently based on their translation of tools to local realities (e.g. perspectives on
how family and household are defined). Primary stakeholders may have different perspectives in
interpreting responses or suggest different directions for further probing and data collection.
Range of roles:
Low
empowerment
 High: P/S as trained member of qualitative data collection team — This allows a different
perspective to guide direction of probing by team; allows different interpretation of responses.
Consider: How aware is team and P/S of P/S’s own personal biases? What other perspectives of
primary stakeholders are not represented and how can this be addressed?
 Low: P/S as trained enumerator — Primary stakeholders may gain skills and understanding of the
M&E activity, but have little to no opportunity to identify new questions or issues.
 Lowest: P/S as interviewee — Even here the level can vary depending on method. The closed
questionnaire allows no input of a different perspective. With RAP techniques, while the primary
stakeholder is still an interviewee, their responses to more open questions will allow new
questions to emerge (if the interviewer is listening.)
For emergency rapid assessment, most of
the design issues should be defined before
the crisis hits as part of preparedness
activities, which makes primary stakeholder
participation difficult. (And if not done during
preparedness phase, time constraints make
it impossible.) The design should include
open tools and methods that allow for
greatest participation of primary stakeholders
in influencing the actual data collection.
For programme monitoring and evaluation in
chronic emergencies, opportunities might
exist to increase primary stakeholder
feedback on, if not participation in, design
choices.
Primary stakeholders must be at least
consulted in M&E in crisis and unstable
contexts, including careful attention to subgroups. However, even simple access for
consultation may be limited where there is
insecurity and intense tension/conflict, and
attention must always be given to protecting
interviewees. See above.
In the case of natural disaster or complex
emergencies where populations are
displaced, it is also important to ensure that
“primary stakeholders”, i.e. those affected,
include host populations.
Where there has been no participation at this stage, participation in subsequent stages will be
severely limited.
UNICEF M&E Training Resource
Primary stakeholders participating in M&E 5/7
Collecting evidence

Participation of primary stakeholders in a stage of validating findings allows an opportunity to correct
and clarify perspectives as well as provide contextual analysis from their perspective.
validation
Range of roles:
Medium
empowerment
Analysis, conclusions;
recommendations
High
empowerment
 High: P/Ss as participants in discussion of findings — Approaches can range from presentations
to community meetings to more carefully designed and facilitated dialogue process (e.g.
“Community Dialogue” processes) where participants representing different sub-groups are given
an opportunity to understand and discuss findings. Consider: How will participants best absorb
findings? How can the views of different sub-groups best be elicited? How will conflicting
perspectives be handled so as to protect individuals and balance views? How can local validation
processes be brought into a wider validation process with other stakeholders?
 Low: P/S as interviewees help to cross-check findings — Approaches can include focus groups
and key informants where interviewees respond to findings culled from wider groups and other
methods. Consider: How are focus groups/key informants chosen to represent sub-groups
among primary stakeholders? How are different voices weighed?
Participation in analysis, conclusions and recommendations can be an extension the validation
process described above, particularly in qualitative processes or mixed processes where both steps
can be more participatory with all stakeholders. Involvement of primary stakeholders becomes more
important and necessary where they will have a role in actual implementation of changes. The
biggest challenge is how local-level validation, analysis, conclusions and recommendations can be
brought into a wider process with other stakeholders without losing the specificity of the local-level
inputs.
Range of roles:
Rapid assessment techniques frequently use
techniques where data collection and
validation processes are merged; such as
community mapping, community meetings.
Similar techniques can be used in
programme monitoring and evaluation.
Particularly in these summary validation
exercises, there is a danger of high
expectations by primary stakeholders.
Transparency about how local findings are
aggregated in wider M/E activities, e.g. larger
needs assessment exercises, is essential.
This can be extremely sensitive, particularly
in emergency rapid assessments in conflict
settings where groups are polarised and
territories are divided. Even in natural
disasters, recommendations that follow out
of local processes may create unrealistic
expectations and can add to tensions
between affected populations and
humanitarian workers.
 See roles under “Define the purpose” above
UNICEF M&E Training Resource
Primary stakeholders participating in M&E 6/7
Using M&E results

dissemination
Participation of primary stakeholders can be a further means of building ownership of results,
particularly where these relate to locally visible follow-up; such as adjustments to existing
programmes. However, careful management of local-level expectations dictates that commitment to
follow-up on the part of other stakeholders should precede dissemination.
Range of roles:
Medium
empowerment
Using M&E results

translation into action
plans
High
empowerment
As in stable contexts, this is highly feasible
where M&E results relate to follow-up that
will be locally visible in the short term.
Ideally dissemination can be piggy-backed
on actual introduction of some agreed
actions and changes.
 High: PS as participant in developing dissemination strategy, deciding who to reach and how —
Using participatory planning approaches. As in earlier stages, attention to marginalised groups
among primary stakeholders is essential.
 Medium: PS participates in development of messages — Approaches can include working with
representative groups chosen through local structures as well as key informants and focus
groups.
 Low: PS as vehicle for dissemination messages — Working through local leaders or paid/
volunteer outreach workers.
Participation in the use of M&E results can be the ultimate participation in M&E and typically
requires some participation in early stages.
Range of roles:
 High: PS as participant in development of action plans — This will generally require working with
established local structures, but can also bring in more participatory planning processes to foster
greater involvement of marginalised groups.
 Low: PS validating decisions — Approaches can range from community meetings allowing a little
more discussion, to focus groups that will be more one –way.
UNICEF M&E Training Resource
This will likely be very difficult in rapid
assessments in the earliest phase after a
crisis. Even in natural disasters, time will
likely only allow co-ordination with other
agencies and local authorities.
For M&E in slow or chronic emergencies and
unstable contexts, possibilities will depend
entirely on the context.
Primary stakeholders participating in M&E 7/7
Download