Introduction to Developing, Monitoring and Evaluating Projects Noah Keuzenkamp TGEU Capacity Building Officer noah@tgeu.org Overview Project development Monitoring & Evaluation Putting both together: LogFrames Group Exercise Presentations A few points to keep in mind Questions Baseline Who has developed a project before? Who has written a fundraising application with a monitoring & evaluation plan before? Who has monitored a project before? Who has evaluated a project before? Who knows what a LogFrame is? Thinking about project development with a logic model Key terms Goal (Impact) Outcomes Outputs Activities •The broad impact to which the project contributes – e.g. at the national level or over the long-term. Your project may not achieve this immediately/by itself. •Example: The trans* community obtains more funding OR: To build the capacity of the trans* community to obtain funding •The outcome at the end of the project – often the expected benefits to the target group(s) •Key words: “increased, improved, etc.” •Example: Participants have increased knowledge of M&E •The direct/tangible results (goods &services) that the project delivers (mainly under your control) •Key words: “delivered, produced, conducted, etc.” •Example: M&E workshop delivered to 20 participants •The main tasks carried out to eventually deliver the outputs •Key words: “prepare, design, develop, research, etc.” •Example: Develop a workshop on M&E Goal Outcomes The broad impact to which the project contributes: Trans* concerns are better represented in politics Expected outcome/benefit at the end of the project: Trans* rights were mentioned 5 times in parliament INTENDED Key terms: another example Direct results delivered: Briefings delivered to 30 parliamentarians Activities Tasks carried out: Identify friendly parliamentarians and organise meetings PLANNED Outputs Thinking about project development with a logic model How do we know we are approaching our goal? Monitoring: the systematic and routine collection of information during the implementation of a project to assess whether the project is progressing as planned Evaluation: a systematic and ‘objective’ assessment of a completed project, especially its outcomes and impact, in order to inform future strategic decisions and projects Development, Monitoring & Evaluation in the Project Cycle Why conduct M&E? In practice often: Because it is a donor condition To show that you are doing what you are funded for To obtain more funding in the future But, it can also: Help you learn and improve Help you understand how change works (or doesn‘t) Help you create, preserve and disseminate knowledge Make you accountable to your communitie(s) Logical Framework Matrix (LogFrame) Helps you systematically develop, plan, implement, monitor and evaluate a project Presents clear and organised connections between impact, outcome, outputs and activities Many donors use/require (a kind of) LogFrame Using a LogFrame Project Summary Indicators Means of Assumptions Verification (MoV) Goal Outcomes Outputs Activities If... then Monitoring & Evaluation Horizontal logic Project Summary Goal Outcome Output Activity Indicators Means of Assumptions Verification (MoV) Variables/ information used to measure change in a process or phenomenon The source of the indicators and the means of obtaining them. = M&E method for obtaining information Assumptions that need to be fulfilled to eventually achieve your goal Indicators: examples „More positive public attitude towards trans* people“ Increase in number of positive news items Decrease in number of hostile statements by politicans Increase in positive attitude of individuals (survey) Gender identity is included in anti-discrimination laws ... Etc. Indicators: examples „Trans* people are more visible in society“ ... ... ... Indicators: SMART S Specific E.g. nr of people, % increase M Measurable Can it be measured in general? Can you measure it? A Achievable Is this realistic? R Relevant Does it measure the ‘concept’? Or: reliable T Time-bound By when or from when to when Not SMART: People will know more about M&E SMART: By the end of the workshop, 70% of people who didn‘t know what a LogFrame is report that they know „fairly well“ how to fill in a LogFrame Means of verification (Mov) Method for collecting indicators to conduct monitoring and evaluation, e.g. Surveys / Questionnaires Focus groups Observation Official statistics News reports Photos Testimonies / stories Etc... This workshop: full example Project Summary Goal Outcomes Outputs Activities The trans* activist community is well funded 70% of participants have increased knowledge of M&E Workshop on M&E delivered to 20 people Develop a workshop on M&E Indicators Means of Assumptions Verification (MoV) Income of participating organisations rises by 20% by 2016 Online survey - M&E plans are key in grant writing -Participants will write grants Nr. of participants who report having gained knowledge Nr. of participants Questionnaire handed out after session (selfreport) LogFrames are not too complex Sign-in sheet Inputs Staff, time, technology, etc. Cost Work time, travel, accommodation etc. = ? People are interested in M&E Preconditions TGEU has capacity to deliver Noah‘s Ark Project Summary Goal Earth remains populated by people and animals Outcomes All people and animals have survived the flood on the Ark Ark is loaded with people and pairs of all animals Outputs Activities Build ark Indicators Means of Assumptions Verification (MoV) - Plant life Census count Census survey returns 100 years after - All animal the flood pairs can/want to reproduce Census count Checklist upon - No one will after landing unloading die from illness - Food will not run out Census count on Checklist upon Animals can be the Ark boarding found and will happily get on the Ark Input Cost Preconditions Wood Barter Can find a safe Noah’s time and plot of land to skills build on Sometimes it‘s better not to be Noah When developing a new project, resist the temptation to (always) start with the Ark, i.e.(specific) activities: „We should run a training, because that‘s what we usually do“ Starting with the goal or the problem and then working from top to bottom is better for productive brainstorming and creative problem solving Working from top to bottom also helps identifying the best target groups or stakeholders Use the LogFrame as a tool for logical and creative thinking Group Exercise Brainstorm about what the main problems are Your project only needs to contribute to the goal Complete 1-2 outcomes/outputs/activities first, then add more if you have time Resist the temptation to start with the activities Write your LogFrame on flipchart paper Agree on presenter(s) Time for preparation: 30 mins A few points to keep in mind When writing a grant proposal Language matters (concise and precise) Terminology choice matters (what does the donor use?) Describing activities: usually active verbs and present tense, e.g. „compile a factsheet“, „perform a play“ Describing outputs and outcomes: usually past tense, e.g. „confidence has increased“, „question has been raised in parliament“ A few points to keep in mind Don’t let the language of donors confuse you e.g. outcome = purpose / (intermediate) objective / result The same outcomes can be measured with different indicators: chose the one(s) best suited to your goals and resources (money, time, skills, technology, …) Evaluation does not always have to be quantitative: qualitative data, such as personal stories, can be used, too M&E is political: who determines what the outcomes should be? who is asked for their opinion? who conducts the evaluation? ( feminist evaluation) Questions? Noah Keuzenkamp noah@tgeu.org What should be SMART? SMART outcome with a variable as indicator: Outcome: 80% of participants have increased their knowledge to a „good“ level by the end of the project Indicator (variable): Number of participants with „good“ knowledge Outcome with a SMART indicator: Outcome: Participants‘ knowledge is increased Indicator: 80% have a „good“ level of knowledge by the end of the project Concept vs Indicators 1: Trans* people‘s report of how safe they feel Trans* people are SAFER in our city 4: Nr. of trans* customers at LGBT bars 2: Nr. of attacks on trans* people reported to police 3: The nr. of anti-Pride protestors Indicators Sometimes they are direct: Planned outcome: A trans* person gets elected into parliament Indicator: Did a trans* person get elected into parliament? But usually they are indirect: Especially when talking about social change, attitudes, knowledge, awareness, skills, etc. Example: this workshop I could monitor: Baseline: what was people‘s knowledge before? Status: how many people are in the room? How many people are listening? Trend: are people leaving? Switching off? Getting more engaged? Implementation: is the workshop being implemented? Are there interruptions? (Ongoing) effectiveness: are people learning something right now? Example: this workshop I could evaluate: Efficiency: was a workshop the best/cheapest method for people to learn about M&E? Effectiveness: did people learn about M&E? Outcome/Impact: do people feel better prepared to complete an M&E plan? will they use the knowledge when writing funding applications? will they get more funding? Sustainability: will people still remember this in a year? will they pass the knowledge on? What is a project? „An individual or collaborative enterprise that is carefully planned to achieve a particular aim” (Oxford Dictionary) Has a start and end date Has constraints (time, money, resources,...) Aims to create change Introduction “I would like to have money for a project to…”