Introduction to Impact Assessment This introduction uses the Weaver’s Triangle, designed by Jane Weaver it has been developed by the Charities Evaluation Service (CES) as part of their monitoring and evaluation model. The Weaver’s triangle helps you identify the aims and objectives, so you can put in place methods for monitoring and evaluating the work you are carrying out using performance indicators such as: Impact or outcome and Output measures Self evaluation cycle Needs assessment Review project and implement recommendations Set aims and objectives: what do you want to achieve and how? Deliver work programme Set performance indicators: to measure your progress The Weaver’s Planning Triangle OVERALL AIM SPECIFIC AIMS OBJECTIVES Taken from Practical monitoring and evaluation – Charities evaluation services What you are hoping to achieve Statements about different aspects of your main purpose Identify what you will do The Triangle in context AIM Longer term effect 1:1 tutorials promote independent learning skills Adapted from CES Planning Triangle developed by Jane Weaver SPECIFIC AIMS The changes or differences we want to make - learners able to manage their own work - timely support for learners at risk - late hand-ins and drop out reduced - Increased learner confidence - tutors use the learning conversation effectively OBJECTIVES What we do to make those changes ▪ 1 year pilot managed by the Senior Tutor ▪ Learning Conversation training for Advanced Practitioners (APs) ▪ APs train programme teams ▪ Learner assessment for baselines ▪ Mentor support for tutors ▪ Monitoring and evaluation of pilot Aims relate to outcomes & impact Aims = the benefits or changes you are trying to achieve The overall aim should relate to the impact you are trying to achieve The specific aims should be linked to the outcomes start with words that indicate a change, for example: to increase, to enable, to develop, to improve etc. Objectives relate to products or outputs Objectives = the methods or the activities by which you achieve your aims The objectives should link to the products or outputs from your activities (eg events held, leaflets distributed, observations made) start with words that indicate activity, for example: to organise, to produce, to set up etc. The Weaver’s Planning Triangle OVERALL AIM SPECIFIC AIMS OBJECTIVES IMPACT WHY we do it Purposes WHAT we do Activities Taken from Practical monitoring and evaluation – Charities evaluation services OUTCOMES OUTPUTS The Triangle in context AIM Longer term effect 1:1 tutorials promote independent learning skills OUTCOMES Adapted from CES Planning Triangle developed by Jane Weaver What outcome measures will we use? The changes or differences we want to make - learners able to manage their own work - timely support for learners at risk - late hand-ins and drop out reduced - Increased learner confidence - tutors use the learning conversation effectively OUTPUTS/ACTIVITIES What we do to make those changes ▪ 1 year pilot managed by the Senior Tutor ▪ Learning Conversation training for Advanced Practitioners (APs) ▪ APs train programme teams ▪ Learner assessment for baselines ▪ Mentor support for tutors ▪ Monitoring and evaluation of pilot How will we monitor the outputs? What are outputs? Outputs are linked to the project objectives Outputs are all the detailed activities, services and products connected to the project. Eg training events, materials, leaflets, visits, observations etc Output example Objective: Set up ‘surgeries’ to discuss the new-build Outputs: • Drop in sessions for staff and the public to meet the senior executive and estates manager in the organisation • Targeted consultation and advice sessions Eg with learners with particular needs and their carers. Output Indicators The indicators are what you can use to assess whether you have achieved your outputs They assess progress towards meeting your project objectives They are a type of performance indicator Output indicators, example Output • Drop in sessions for staff and the public to meet the senior executive and estates manager in the organisation Output indicators: • Number of drop in sessions • Number of people attending • Types of drop in sessions • Profile of people attending What are outcomes? Outcomes are linked to the specific aims Outcomes are the changes, benefits, learning or other effects that actually occur as a result of your activities. They can be expected or unexpected, positive or negative. Outcome indicators or measures • are how you assess whether an expected outcome is occurring and assess progress towards meeting an aim • Impact and outcome measurement is basically about the same thing, a change that results from the improvement actions taken • Outcome indicators are a performance indicator and can be qualitative or quantitative Outcome example Specific aim: To improve the cascading of information regarding the new-build across the organisation and in the local community Outcomes: • Increased knowledge of the new-build programme • Increased marketing of the organisation • Increased confidence and reputation What indicators or measures could you use? Monitoring outputs and outcomes Use the indicators for monitoring • Collate information on your output indicators to show outputs have been delivered and to show progress towards meeting targets. • Collect information on your outcome indicators to demonstrate that outcomes are achieved. Collect baseline information before you start, in order to measure the impact of your improvement actions