PRIMARY STAKEHOLDERS PARTICIPATING IN M&E There are different rationales for primary stakeholders participating in M&E: Increasing primary stakeholder participation in an M/E activity contributes to the quality of the results. It increases objectivity by achieving balanced analyses, recognising biases and reconciling perspectives of different stakeholders and results in greater accuracy, completeness and fairness of M&E results. Increasing primary stakeholders’ participation in M&E is consistent with a human rights-based approach to programming. The process of M&E can contribute to empowerment and realisation of primary stakeholders’ right to participate, which is established in both the Universal Declaration of Human Rights and the Convention on the Rights of the Child. DIVERSITY AND DISPARITY AMONG PRIMARY STAKEHOLDERS Like any other group, primary stakeholders are not homogenous regardless of programme and context. Whether primary stakeholders are women, children, poor families, or communities affected by humanitarian crisis, they may have some common interests, but sub-groups within will face diverse realities and will have correspondingly diverse vulnerabilities and capacities. (See core content sheets Expanded VulnerabilityCapacity Analysis and Gender and Age) Efforts to expand the participation of primary stakeholders without attention to participation within the group, including how to address the particular constraints faced by less powerful and marginalised groups, can do more harm than good. At the same time, working to expand the participation of any group of people engenders responsibilities to protect them (See core content sheets Ethical issues for field study – Dealing with people and Children participating in research, M&E: Ethics and your responsibilities as a manager). EXPANDING PRIMARY STAKEHOLDERS’ PARTICIPATION IN M&E Expanding primary stakeholders’ participation in M/E activities requires thinking through a strategy for participation which has to feed into the strategy for participation for the programme as a whole. Elaborating a strategy requires defining: What approach to participation is appropriate? Primary stakeholder participation can be characterised in different ways. In general, the variation is according to whose interests drive – other stakeholders, others and primary stakeholders together or primary stakeholders in first priority -- and what the objective of participation is – whether effectiveness or strengthening local capacities. (See also Roger Hart’s scales of participation for children in core content sheet Children participating in research, M&E: Ethics and your responsibilities as a manager) Is it better to engage with primary stakeholders directly or indirectly, i.e. using existing structures? Which other stakeholders will be involved? What tools and methods are appropriate? (Adapted from ALNAP (2003): p. 32) Answering the above questions requires realistic analysis of: the general context, the primary stakeholders, other stakeholders including your own organisation, and the nature of the M/E activity as it is defined so far. The following are some of the key considerations for each, with some issues clearly overlapping. The general context (stakeholders, cultural, institutional, geographic and security) How will different stakeholders be affected by primary stakeholder participation? Who will win and lose and how are winners and losers likely to use their influence? (More on other stakeholders below) How do attitudes/beliefs, tradition and customs constrain or support primary stakeholder participation, in general or by particular sub-groups among primary stakeholders? How could constraints be overcome? Who could serve as a bridge? UNICEF M&E Training Resource Primary stakeholders participating in M&E 1/7 What informal or institutionalised processes or fora are in place to support participation, nationally, locally or even within the programme (committees, networks, meeting places etc) ? Who has access and who doesn’t? What are the patterns of exclusion and discrimination – in general but also within primary stakeholders? Consider gender, age, ethnicity, language, socio-economic divisions. How could these patterns be countered? What are the risks that they will be exacerbated? What is the space for and response to controversial views? What are the social risks to participating and voicing different opinions? Especially in crisis and unstable contexts, what are the security risks for primary stakeholders who participate and how can these be avoided? This can include both untargeted violence to which participants are more exposed (e.g. robbery or assault along deserted roads) or targeted reprisal for participating. Or can participation increase protection of primary stakeholders and how can this be managed? What measures are in place to protect primary stakeholders if their views are controversial? Are there limits to physical access to primary stakeholders that will negatively affect and how can they be overcome? What is the history behind and evolution of all of the above and, particularly in unstable contexts, how have these changed more recently? Primary stakeholders Among primary stakeholders, what are the attitudes to participation and to sharing information and decision-making? Consider different attitudes of leaders, dominant groups and marginalised groups among primary stakeholders? How are primary stakeholders represented in local structures or networks? How is this representation perceived? Are there sub-groups among primary stakeholders who are excluded? Are there any opportunities to feed M/E activities into ongoing decision-making processes involving or affecting primary stakeholders? (e.g. decentralised budgeting or assemblies) How do primary stakeholders perceive other stakeholders in the programme? Is there trust/distrust? Is there anything that can be done to restore the situation? What is the history of past efforts at increasing participation? Is there ‘participation’ fatigue? What understanding do primary stakeholders have of/how do they perceive M/E activities – an opportunity for more services/resources, a threat? How will this affect their participation? What capacities do primary stakeholders have for different roles in the various stages of M/E activities? Where are the gaps/constraints? Consider attitudes as above, but also organisational structures, skills and knowledge and time. How do primary stakeholders perceive the risks to participation – see above social and security risks? Other stakeholders (including your organisation) What other stakeholders should be involved in the M/E activity? What are other stakeholders’ attitudes to participation of primary stakeholders? What does it mean to them? How willing are they to be transparent and share information with primary stakeholders? To what degree are more powerful stakeholders open to relinquishing control? What motivates other stakeholders to get the participation of primary stakeholders? Consider: organisational mandates and policies, any understanding of primary stakeholders right to participate, concerns for completeness of information (e.g. needing primary stakeholders’ perspective on unintended and negative effects of an intervention) or concerns about follow-up (e.g. where primary stakeholders ownership is needed to ensure follow-up). Are they interested and willing to invest in strengthening primary stakeholders’ capacities? Are there any concerns about negative effects of primary stakeholders’ participation? Are different stakeholders or organisations concerned about the independence and impartiality of the M/E activity? Especially in crisis and unstable contexts, are international organisations concerned about losing their independence and impartiality and getting involved with parties to a civil conflict or war? Are other stakeholders aware of and sensitive to the expectations that will be created as primary stakeholder participation increases? Are they ready to take responsibility for commitments made? What are their attitudes to M/E methods that will likely be involved? Are there any standard M/E methods and tools promoted by stakeholder organisations (or donors) that will limit primary stakeholder involvement? UNICEF M&E Training Resource Primary stakeholders participating in M&E 2/7 How will participatory approaches to M/E fit with their existing M&E systems? Will they be an added burden or can they be seen as a better substitute? How can different information needs be balanced in an overall M/E activity or broader M&E system? Where many other stakeholders are involved, how will primary stakeholder involvement fit in broader coordination mechanisms? Constraints on the M/E activity as it is defined so far Is the focus of the M/E activity defined with reference to primary stakeholders’ priorities and concerns or at least those of some sub-groups? If not, is there a subset of issues that are and that can be managed as a separate more participatory facet of the M/E activity? What are the numbers of primary stakeholders involved in the programme/project or affected by the issue that is the focus of the M&E activity? The bigger the numbers, the harder it is to ensure representative primary stakeholder participation, particularly where there are no existing institutions or mechanisms for their participation? Are there principles governing who participates that stakeholders, including primary stakeholders, can all agree on? Can a human rights-based approach be adopted and how will the process be handled if basic rights to participation are not acknowledged? Can any locally acceptable principles be found to protect a space for the participation of marginalised groups? What time and resources (human, financial) are available from different stakeholders to support primary stakeholder participation? Can external M/E “experts” be found who understand their role as facilitators of a process and have the necessary attitudes and skills to carry this out? Do those managing the M/E activity have the necessary awareness, knowledge and skills to support a participatory process? To what degree can training and other capacity building be integrated into the M/E activity? How much attitude change among all groups of stakeholders, primary and other, will be necessary to achieve the level of participation desired and how realistically can this be built in to the process design? Can the M/E activity be managed so that primary stakeholders see the results of their participation? Can you provide quick and relevant feedback of findings locally? How long will it take for recommendations and actions to be perceived in local level changes? How well can the M/E activity balance feedback into immediate local learning and decision-making with feedback into learning and decision-making at the level of wider programmes, organisations and institutions? Can the M&E design be flexible, in terms of taking on and responding to new or different research/ evaluation questions in course, and/or adapting methods and instruments to different localities and subgroups? What measures can be introduced to protect primary stakeholders and more vulnerable groups among them and what capacity do organisations or committees managing the M/E activity have to ensure that they are implemented? (See core content sheets Ethical issues for field study — Dealing with people and Children’s participation in research, monitoring and evaluation (M&E): Ethics and your responsibilities as a manager.) Sources: ALNAP (2003); Ashton (1998); Davies (1997); Guijt, I. (1999); Harnmeijer, Waters-Bayer and Bayer (2000); IFAD (????); Institute of Development Studies (1998). UNICEF M&E Training Resource Primary stakeholders participating in M&E 3/7 PARTICIPATION CONTINUUM IN M&E WHEN AND IN WHAT ROLE DO PRIMARY STAKEHOLDERS PARTICIPATE? Opportunities to expand primary stakeholders’ participation in M&E activities exist at two levels: The steps or activities in which primary stakeholders are involved — some steps or activities are more pivotal, giving stakeholders involved a greater ownership of M&E results The role primary stakeholders have in each step or activity — roles entail higher or lower degrees of participation. As you read the chart below, note that really high participation in some steps/activities, like data collection, may be a great achievement in a given context, but cannot be expected to contribute greatly to empowerment if there are no efforts to increase participation in other more pivotal steps. Steps/Activities Why, what role, examples of how? Considerations in crisis and unstable contexts Define the purpose Participation in defining the purpose is important as the whole design should flow from this. For emergency rapid assessment, and other M&E activities in early phases after a crisis, participation at this stage of an M&E activity is likely over-ambitious. It might be considered in chronic emergencies where the context is right. See overall questions above. Range of roles: High empowerment High: P/S as active contributor — Using local institutions/organisations that represent primary stakeholders, either taking negotiations of purpose for the M&E activity to these fora or inviting representatives to be included equitably in a negotiation process. Some considerations: How can the negotiation process be modified and facilitated to ensure that primary stakeholders’ voice is heard, including sub-groups within? Are there institutions/organisations that can effectively represent minority groups? Low: P/S providing validation — Using focus groups to bring in primary stakeholders’ perspective, either feeding into the definition of purpose and/or critiquing what is already proposed. Again, consider: How are focus groups structured to represent sub-groups among primary stakeholders? How are different voices weighed? In all cases in crisis and unstable contexts, the purpose can be defined so as to include reference to obtaining primary stakeholders’ perspective. Where participation is not possible at this stage, greater involvement of primary stakeholders in later stages is better ensured if the purpose includes reference to obtaining primary stakeholders’ perspective (e.g. on priority needs, on changes produced in the P/S’s situation). UNICEF M&E Training Resource Primary stakeholders participating in M&E 4/7 Focus the design define questions to be answered define analytical framework define methods/process define measures/ indicators define tools High empowerment Collecting evidence data collection Where primary stakeholders are not involved in design, the M&E activity could be biased, missing important issues for the stakeholders. Primary stakeholders’ perspectives can challenge other stakeholders’ more “established” analytical frameworks, bringing new perspectives on the interrelationship of different issues. Range of roles: See roles under “Define the purpose” above Where participation is not possible at this stage, M&E activities can be designed to accommodate primary stakeholders’ proposed questions and measures as the data is being collected. This can be done by including more open qualitative methods and tools. Techniques such as those drawn from RAP and PRA will allow input by interviewees in data collection in terms of shaping the issues to be explored. For most M&E activities, at least collecting data from primary stakeholders is considered necessary to ensure accurate, complete and fair results. Greater participation of primary stakeholders in actually collecting data can add to this, particularly with qualitative methods. Primary stakeholders may apply tools differently based on their translation of tools to local realities (e.g. perspectives on how family and household are defined). Primary stakeholders may have different perspectives in interpreting responses or suggest different directions for further probing and data collection. Range of roles: Low empowerment High: P/S as trained member of qualitative data collection team — This allows a different perspective to guide direction of probing by team; allows different interpretation of responses. Consider: How aware is team and P/S of P/S’s own personal biases? What other perspectives of primary stakeholders are not represented and how can this be addressed? Low: P/S as trained enumerator — Primary stakeholders may gain skills and understanding of the M&E activity, but have little to no opportunity to identify new questions or issues. Lowest: P/S as interviewee — Even here the level can vary depending on method. The closed questionnaire allows no input of a different perspective. With RAP techniques, while the primary stakeholder is still an interviewee, their responses to more open questions will allow new questions to emerge (if the interviewer is listening.) For emergency rapid assessment, most of the design issues should be defined before the crisis hits as part of preparedness activities, which makes primary stakeholder participation difficult. (And if not done during preparedness phase, time constraints make it impossible.) The design should include open tools and methods that allow for greatest participation of primary stakeholders in influencing the actual data collection. For programme monitoring and evaluation in chronic emergencies, opportunities might exist to increase primary stakeholder feedback on, if not participation in, design choices. Primary stakeholders must be at least consulted in M&E in crisis and unstable contexts, including careful attention to subgroups. However, even simple access for consultation may be limited where there is insecurity and intense tension/conflict, and attention must always be given to protecting interviewees. See above. In the case of natural disaster or complex emergencies where populations are displaced, it is also important to ensure that “primary stakeholders”, i.e. those affected, include host populations. Where there has been no participation at this stage, participation in subsequent stages will be severely limited. UNICEF M&E Training Resource Primary stakeholders participating in M&E 5/7 Collecting evidence Participation of primary stakeholders in a stage of validating findings allows an opportunity to correct and clarify perspectives as well as provide contextual analysis from their perspective. validation Range of roles: Medium empowerment Analysis, conclusions; recommendations High empowerment High: P/Ss as participants in discussion of findings — Approaches can range from presentations to community meetings to more carefully designed and facilitated dialogue process (e.g. “Community Dialogue” processes) where participants representing different sub-groups are given an opportunity to understand and discuss findings. Consider: How will participants best absorb findings? How can the views of different sub-groups best be elicited? How will conflicting perspectives be handled so as to protect individuals and balance views? How can local validation processes be brought into a wider validation process with other stakeholders? Low: P/S as interviewees help to cross-check findings — Approaches can include focus groups and key informants where interviewees respond to findings culled from wider groups and other methods. Consider: How are focus groups/key informants chosen to represent sub-groups among primary stakeholders? How are different voices weighed? Participation in analysis, conclusions and recommendations can be an extension the validation process described above, particularly in qualitative processes or mixed processes where both steps can be more participatory with all stakeholders. Involvement of primary stakeholders becomes more important and necessary where they will have a role in actual implementation of changes. The biggest challenge is how local-level validation, analysis, conclusions and recommendations can be brought into a wider process with other stakeholders without losing the specificity of the local-level inputs. Range of roles: Rapid assessment techniques frequently use techniques where data collection and validation processes are merged; such as community mapping, community meetings. Similar techniques can be used in programme monitoring and evaluation. Particularly in these summary validation exercises, there is a danger of high expectations by primary stakeholders. Transparency about how local findings are aggregated in wider M/E activities, e.g. larger needs assessment exercises, is essential. This can be extremely sensitive, particularly in emergency rapid assessments in conflict settings where groups are polarised and territories are divided. Even in natural disasters, recommendations that follow out of local processes may create unrealistic expectations and can add to tensions between affected populations and humanitarian workers. See roles under “Define the purpose” above UNICEF M&E Training Resource Primary stakeholders participating in M&E 6/7 Using M&E results dissemination Participation of primary stakeholders can be a further means of building ownership of results, particularly where these relate to locally visible follow-up; such as adjustments to existing programmes. However, careful management of local-level expectations dictates that commitment to follow-up on the part of other stakeholders should precede dissemination. Range of roles: Medium empowerment Using M&E results translation into action plans High empowerment As in stable contexts, this is highly feasible where M&E results relate to follow-up that will be locally visible in the short term. Ideally dissemination can be piggy-backed on actual introduction of some agreed actions and changes. High: PS as participant in developing dissemination strategy, deciding who to reach and how — Using participatory planning approaches. As in earlier stages, attention to marginalised groups among primary stakeholders is essential. Medium: PS participates in development of messages — Approaches can include working with representative groups chosen through local structures as well as key informants and focus groups. Low: PS as vehicle for dissemination messages — Working through local leaders or paid/ volunteer outreach workers. Participation in the use of M&E results can be the ultimate participation in M&E and typically requires some participation in early stages. Range of roles: High: PS as participant in development of action plans — This will generally require working with established local structures, but can also bring in more participatory planning processes to foster greater involvement of marginalised groups. Low: PS validating decisions — Approaches can range from community meetings allowing a little more discussion, to focus groups that will be more one –way. UNICEF M&E Training Resource This will likely be very difficult in rapid assessments in the earliest phase after a crisis. Even in natural disasters, time will likely only allow co-ordination with other agencies and local authorities. For M&E in slow or chronic emergencies and unstable contexts, possibilities will depend entirely on the context. Primary stakeholders participating in M&E 7/7