How good is our third sector organisation? Elizabeth Morrison March 2014 The third sector makes a direct impact on the growth of Scotland's economy, the wellbeing of its citizens and the improvement of its public services. This resource is designed to help third sector organisations self-evaluate and plan for improvement more effectively, there is no expectation that this needs to be used for this or any other purpose. It is important that organisations use a toolkit of different approaches matched most appropriately to specific circumstances and differing contexts. Many third sector organisations already have very good systems and processes in place for this which they should continue to use. The culture for self-evaluation Looking inwards Looking outwards Looking forwards Looking inwards Basic process of self-evaluation How are we doing? How do we know? What are we going to do now? A ‘standard’ is a Benchmark You can measure whether or not you have reached a standard - achieved/not achieved STANDARDS achieved or not achieved yes/no seen or not seen ‘quality’, on the other hand, is associated with the idea of • • • • • • value worth degree of extent of merit significance When we evaluate quality we ask questions like To what extent? How good? How well? NOT ‘IS IT THERE?’ or HOW MANY? Evaluating quality Quality is evaluated through the use of ‘quality indicators’ or ‘QIs’. Why do we use quality indicators? • Evaluations against QIs are professional judgments. • By using QIs, we ensure consistency across different contexts Using indicators to evaluate quality The use of QIs requires the evaluator to balance the strengths and areas for improvement and come to an overall view. The evaluation is based on the relative importance of each of the strengths and areas for improvement, and their impact not on the numbers of strengths/weaknesses. It is generally easier to measure achievement of a standard than to evaluate quality. Evaluation of quality is therefore a professional skill. Quality indicators balance strengths and areas for improvement importance NOT numbers Public assurance requires us to measure whether organisations reach minimum standards. We need to evaluate quality if we want to help organisations to improve. Structure Based on outcomes, processes and leadership Structure in line with other public service self-evaluation frameworks QIs grouped under 6 important questions Six key questions What key outcomes have we achieved? How well do we meet the needs of our stakeholders How good is our delivery of key processes? How good is our operational management? How good is our strategic leadership? What is our capacity for improvement? Our self-evaluation frameworks outcomes processes inputs Evaluating quality Links between quality indicators within a quality framework QIs are related to each other. The relationships are causal. When you make evaluations in relation to one QI you also need to check the evaluations given for other QIs to make sure they make sense. How do you evaluate the overall quality of a third sector organisation? How do you evaluate the overall quality of anything? How do you evaluate the overall quality of a…….. football club? Outcomes and achievements How well is the club doing? 1. The team’s performance Is it a successful football club? Number of games won – getting better or worse? How many goals scored? Performance compared with that of other clubs? Outcomes and achievements How well is the club doing? Does it live up to people’s expectations? 2. Impact on people What do the fans think? How many people come to the games? Do people want to become regular supporters? Do people look sad or happy during the game? What do supporters think of the club? Outcomes and achievements How well is the club doing? How well does it live up to people’s expectations? Is it a good club to play for? Are the players highly motivated? Are there a lot of transfer requests Does it have a high absence rate ? 3. Impact on team members Does it have a high turnover of players? What do players think of the quality of coaching? Outcomes and achievements How well is the club doing? How well does it live up to people’s expectations? Does it have a good reputation? In the local area? Worldwide? In Europe? 4. Impact on the community In Scotland? Outcomes and achievements How well is the club doing? How well does it meet people’s expectations? Quality of play How well does the team play? 5. Teamwork Does the team play well? Is the team fit and prepared for the rigours of a full game? 6. Skills Does the team play with flair? 7. Fitness Does it have a good blend of strengths in defence, mid-field and attack? Outcomes and achievements How well is the club doing? How well does it live up to people’s expectations? Quality of play How well does the team play? Identifying areas for improvement Providing feedback to players Is the club well run? Management and leadership How well is the club managed? 8. Observation and monitoring of matches Identifying strengths in the team Outcomes and achievements How well is the club doing? Quality of play How well does it live up to people’s expectations? How well does the team play? Management and leadership How well is the club managed? Is there a good training programme? Does it deal well in the transfer market? 9. Management of staff Does it make sure players play to their strengths? Is the club well run? Quality of play Outcomes and achievements How well is the club doing? How well does it live up to people’s expectations? How well does the team play? Identifying areas for improvement Providing feedback to players Is the club well run? Management and leadership How well is the club managed? 8. Observation and monitoring of matches Identifying strengths in the team Outcomes and achievements How well is the club doing? Quality of play How well does it live up to people’s expectations? How well does the team play? Management and leadership How well is the club managed? Is there a good training programme? Does it deal well in the transfer market? 9. Management of staff Does it make sure players play to their strengths? Is the club well run? Outcomes and achievements How well is the club doing? How well does it live up to people’s expectations? Quality of play Management and leadership How well does the team play? How well is the club managed? Is it good at getting sponsorship? Is the club well run? Does it manage its money well? Does it manage its grounds and facilities well? 10. Management of Resources Quality of play Outcomes and achievements How well is the club doing? How well does it live up to people’s expectations? How well does the team play? Management and leadership How well is the club managed? Does the club have a clear vision of what it is aiming for? Does the club know its strengths and weaknesses and is it making improvements? Does the club have a plan for improving results? Is the club well led? Do the chief executive, manager and captain provide strong direction? Do they have the support of fans and players? Are they prepared to make hard decisions to improve performance? How well is the club led? 11. Leadership of the manager 12. Leadership of the Chief Executive & Board of Directors How do you evaluate the overall quality of a football club? Framework for evaluating the quality of a football club Outcomes and achievements How well is the club doing? 1. The team’s performance How well does it live up to people’s expectations? Quality of play How well does the team play? 2. Impact on people 5. Teamwork 3. Impact on team members 6. Skills 4. Impact on the community 7. Fitness Leadership and management How well is the club managed? 8. Observation and monitoring of matches 9. Management and support of team 10. Management of resources Cause and effect How well is the club led? 11. Leadership of the manager 12. Leadership of the Chief Executive & Board of Directors What key outcomes have we achieved? How well do we meet the needs of our stakeholders? How good is our delivery of key processes? How good is our operational management? How good is our strategic leadership? 1. Key performance outcomes 1.1 Improvement in performance 1.2 Adherence to statutory principles and fulfilment of statutory duties 2. Impact on service users 2.1 Impact on service users 5. Delivery of key processes 5.1 Delivering services 5.2 Developing, managing and improving relationships with service users and other stakeholders 5.3 Inclusion, equality and fairness 6. Operational management 6.1 Policy review and development 6.2 Participation of service users and other stakeholders 6.3 Planning of key processes 9. Strategic leadership 9.1 Vision, values and aims 9.2 Leadership and direction 9.3 Leading people and developing partnerships 9.4 Leadership of innovation, change and improvement 9.5 Securing improvement of quality and impact of services What is our capacity for improvement? 10. Capacity for improvement Global judgement based on evidence of all key areas, in particular, outcomes, impact and leadership 3. Impact on staff 3.1 Impact on staff 4. Impact on the community 4.1 Impact on the local community 4.2 Impact on the wider community 7. Staffing 7.1 Management and deployment of staff 7.2 Career-long professional learning 8. Partnerships and resources 8.1 Partnership working 8.2 Financial management 8.3 Resource management 8.4 Knowledge and information management www.educationscotland.gov.uk