How good is our
third sector
organisation?
Elizabeth Morrison
March 2014
The third sector makes a direct
impact on the growth of Scotland's
economy, the wellbeing of its
citizens and the improvement of its
public services.
This resource is designed to help third sector
organisations self-evaluate and plan for
improvement more effectively, there is no
expectation that this needs to be used for this or
any other purpose.
It is important that organisations use a toolkit of
different approaches matched most
appropriately to specific circumstances and
differing contexts.
Many third sector organisations already have
very good systems and processes in place for
this which they should continue to use.
The culture for self-evaluation
Looking inwards
Looking outwards
Looking forwards
Looking inwards
Basic process of self-evaluation
 How are we doing?
 How do we know?
 What are we going to do now?
A ‘standard’ is a
Benchmark
You can measure whether or
not you have reached a
standard
- achieved/not achieved
STANDARDS
achieved
or not
achieved
yes/no
seen
or not
seen
‘quality’, on the other hand, is
associated with the idea of
•
•
•
•
•
•
value
worth
degree of
extent of
merit
significance
When we evaluate quality
we ask questions like
To what
extent?
How
good?
How
well?
NOT
‘IS IT THERE?’
or
HOW MANY?
Evaluating quality
Quality is evaluated through the use of
‘quality indicators’ or ‘QIs’.
Why do we use quality indicators?
• Evaluations against QIs are professional
judgments.
• By using QIs, we ensure consistency
across different contexts
Using indicators to evaluate quality
 The use of QIs requires the evaluator to balance the
strengths and areas for improvement and come to
an overall view.
 The evaluation is based on the relative importance
of each of the strengths and areas for improvement,
and their impact
 not on the numbers of strengths/weaknesses.
 It is generally easier to measure achievement of a
standard than to evaluate quality.
 Evaluation of quality is therefore a professional skill.
Quality indicators balance
strengths and areas for
improvement
importance
NOT
numbers
Public assurance requires us to
measure whether organisations
reach minimum standards.
We need to evaluate quality if we
want to help organisations to
improve.
Structure
Based on outcomes, processes and
leadership
Structure in line with other public
service self-evaluation frameworks
QIs grouped under 6 important
questions
Six key questions
What key outcomes have we achieved?
How well do we meet the needs of our
stakeholders
How good is our delivery of key processes?
How good is our operational management?
How good is our strategic leadership?
What is our capacity for improvement?
Our self-evaluation frameworks
outcomes
processes
inputs
Evaluating quality
Links between quality indicators
within a quality framework
 QIs are related to each other.
 The relationships are causal.
 When you make evaluations in relation
to one QI you also need to check the
evaluations given for other QIs to
make sure they make sense.
How do you evaluate the
overall quality of a third
sector organisation?
How do you evaluate the
overall quality of anything?
How do you evaluate the
overall quality of a……..
football club?
Outcomes and achievements
How well is the
club doing?
1. The team’s
performance
Is it a successful football club?
Number of games
won – getting better
or worse?
How many goals
scored?
Performance
compared with that of
other clubs?
Outcomes and achievements
How well is
the club
doing?
Does it live up
to people’s
expectations?
2. Impact on
people
What do the fans
think?
How many people come
to the games?
Do people want to
become regular
supporters?
Do people look sad
or happy during the
game?
What do supporters think of the club?
Outcomes and achievements
How well is
the club
doing?
How well does
it live up to
people’s
expectations?
Is it a good club to play for?
Are the players
highly motivated?
Are there a
lot of
transfer
requests
Does it
have a
high
absence
rate ?
3. Impact on
team
members
Does it have a high
turnover of players?
What do players
think of the quality
of coaching?
Outcomes and achievements
How well is
the club
doing?
How well does
it live up to
people’s
expectations?
Does it have a good
reputation?
In the local area?
Worldwide?
In Europe?
4. Impact
on the
community
In Scotland?
Outcomes and achievements
How well is
the club
doing?
How well does
it meet
people’s
expectations?
Quality of play
How well does
the team play?
5. Teamwork
Does the team
play well?
Is the team fit and
prepared for the
rigours of a full
game?
6. Skills
Does the team play
with flair?
7. Fitness
Does it have a good
blend of strengths in
defence,
mid-field and attack?
Outcomes and achievements
How well is
the club
doing?
How well does
it live up to
people’s
expectations?
Quality of play
How well does
the team play?
Identifying areas for
improvement
Providing feedback
to players
Is the club well run?
Management and leadership
How well is
the club
managed?
8.
Observation
and
monitoring
of matches
Identifying strengths
in the team
Outcomes and achievements
How well is
the club
doing?
Quality of play
How well does
it live up to
people’s
expectations?
How well
does the
team play?
Management and leadership
How well is
the club
managed?
Is there a good
training programme?
Does it deal well in
the transfer market?
9.
Management
of staff
Does it make sure
players play to their
strengths?
Is the club well run?
Quality of play
Outcomes and achievements
How well is
the club
doing?
How well does
it live up to
people’s
expectations?
How well does
the team play?
Identifying areas for
improvement
Providing feedback
to players
Is the club well run?
Management and leadership
How well is
the club
managed?
8.
Observation
and
monitoring
of matches
Identifying strengths
in the team
Outcomes and achievements
How well is
the club
doing?
Quality of play
How well does
it live up to
people’s
expectations?
How well
does the
team play?
Management and leadership
How well is
the club
managed?
Is there a good
training programme?
Does it deal well in
the transfer market?
9.
Management
of staff
Does it make sure
players play to their
strengths?
Is the club well run?
Outcomes and achievements
How well is
the club
doing?
How well does
it live up to
people’s
expectations?
Quality of play
Management and leadership
How well does
the team play?
How well is
the club
managed?
Is it good at
getting
sponsorship?
Is the club well run?
Does it manage its money
well?
Does it manage
its grounds and
facilities well?
10.
Management
of Resources
Quality of play
Outcomes and achievements
How well is
the club
doing?
How well does it
live up to
people’s
expectations?
How well does
the team play?
Management and leadership
How well is
the club
managed?
Does the club have a
clear vision of what it is
aiming for?
Does the club know its
strengths and
weaknesses and is it
making improvements?
Does the club
have a plan for
improving
results?
Is the club well led?
Do the chief executive,
manager and captain
provide strong direction?
Do they have the
support of fans and
players?
Are they prepared to make
hard decisions to improve
performance?
How well is
the club led?
11.
Leadership
of the
manager
12.
Leadership
of the Chief
Executive &
Board of
Directors
How do you evaluate the overall
quality of a football club?
Framework for evaluating the quality of a football club
Outcomes and achievements
How well is
the club
doing?
1. The
team’s
performance
How well does it
live up to
people’s
expectations?
Quality of play
How well
does the
team play?
2. Impact on
people
5.
Teamwork
3. Impact on
team
members
6. Skills
4. Impact on
the
community
7. Fitness
Leadership and management
How well is
the club
managed?
8. Observation
and
monitoring of
matches
9.
Management
and support of
team
10.
Management
of resources
Cause and effect
How well is
the club
led?
11.
Leadership
of the
manager
12.
Leadership
of the Chief
Executive &
Board of
Directors
What key
outcomes have
we achieved?
How well do we
meet the needs of
our stakeholders?
How good is our
delivery of key
processes?
How good is our
operational
management?
How good is our
strategic
leadership?
1. Key
performance
outcomes
1.1 Improvement in
performance
1.2 Adherence to
statutory principles
and fulfilment of
statutory duties
2. Impact on service
users
2.1 Impact on service
users
5. Delivery of
key processes
5.1 Delivering
services
5.2 Developing,
managing and
improving
relationships with
service users and
other
stakeholders
5.3 Inclusion,
equality and
fairness
6. Operational
management
6.1 Policy review
and development
6.2 Participation of
service users and
other stakeholders
6.3 Planning of key
processes
9. Strategic
leadership
9.1 Vision, values
and aims
9.2 Leadership and
direction
9.3 Leading people
and developing
partnerships
9.4 Leadership of
innovation, change
and improvement
9.5 Securing
improvement of
quality and impact of
services
What is our
capacity for
improvement?
10. Capacity for
improvement
Global judgement
based on evidence
of all key areas, in
particular,
outcomes, impact
and leadership
3. Impact on staff
3.1 Impact on staff
4. Impact on the
community
4.1 Impact on the
local community
4.2 Impact on the
wider community
7. Staffing
7.1 Management
and deployment of
staff
7.2 Career-long
professional
learning
8. Partnerships and
resources
8.1 Partnership
working
8.2 Financial
management
8.3 Resource
management
8.4 Knowledge and
information
management
www.educationscotland.gov.uk