Being the best we can
Developing a self evaluation and
improvement model
Scottish Library and Information Council
Background
 How we got to the Public Library Quality Improvement
Matrix
 Legislative context
 Standards and benchmarks based on inputs and
outputs
 Changing paradigm - local government re-organisation,
People’s Network, SNP administration
 Increased scrutiny-Best Value, Efficient Government,
 Changing national policies on Culture and shared
services
 Currently working with 28 authorities – 9 reports
published, plus 9 pilot reports
Scottish Library and Information Council
Public Library Quality Improvement
Matrix
Developed 2004
European Framework of Quality
Management and schools inspection
7 Quality Indicators
4 service QIs and 3 management QIs
Toolkit to support roll out
Benchmarks and characteristics
 more at www.slainte.org.uk
Scottish Library and Information Council
So how did self evaluation work in
Scotland in 2008?
Moray QI 2 Community and personal participation and QI 6 Organisation and
use of resources and space
East Dunbartonshire QI 1 Access to information and QI 5 Ethos and values
Edinburgh – all 7 QIs
Dundee QI 3 Meeting readers’ needs and QI 5 Ethos and values
Shetland – all 7 QIs
Fife – QI 3 Meeting readers’ needs
Falkirk QI 1 Access to information and QI 6 Organisation and use of
resources and space
North Ayrshire QI 2 Community and personal participation and QI 3 Meeting
readers’ needs
Scottish Library and Information Council
So what were the outcomes of self
evaluation in Scotland in 2008?




10 Very good
12 Good
3 Satisfactory
Level 5s are predominantly in QI 2 Community and personal
participation. The level 3s are in Organisation and use of
resources and space and Leadership.
Scottish Library and Information Council
Some examples of strengths in 2008
Moray
Council
Accredited courses. Use of Individual Learning Accounts to
fund two posts which improved learning opportunities
offered by the library.
East
Range of information services available to residents in a
Dunbartonshi small area. Staff are well-trained, knowledgeable and keen
re
to share their enthusiasm for subjects and online sources
with the public
Edinburgh
Award winning Libraries4YOU initiative, working with young
teenagers to combat anti-social behaviour problems
Dundee
Staff commitment to needs of vulnerable groups: provision
for minority groups, Customer Care Standards, literacies
learners and engagement with drug and alcohol recovery
groups
Scottish Library and Information Council
Scottish Government Public Libraries
Improvement Fund
 The Scottish Government Public Libraries Improvement
Fund is a £500,000 fund for a 5 year period to
encourage service development in authorities who
already investing and achieving progress in their public
library service delivery.
 Authorities who undertake the self-evaluation process
and are assessed externally verified as satisfactory or
better will be able to submit proposals based on their
action plans to the Improvement Fund.
 The Public Library Quality Improvement Matrix can be
used as a performance management tool within local
authorities.
Scottish Library and Information Council
Excellence frameworks
 Japanese established the Deming Prize in 1951.
 In 1987 Australia established the Australian Quality
Award.
 In 1988 US established Malcolm Baldrige National
Quality Award.
 1991 European Foundation for Quality Management
(EFQM) established the European Model for TQM.
 All models are reviewed regularly and have developed
a focus on overall business excellence rather than
quality.
Australian Business Excellence Framework
Being the best we can
 Developed by the Library Network in Victoria
 Provides tools for local and state government to
meet service obligations and objectives
 Focus on planning and continuous improvement
 Demonstrates the continuing relevance,
contribution and impact of library services
 Based on the Scottish Library and Information
Council’s Public Library Quality Improvement Matrix
 Contextualized to reflect Australian policy, the
Australian Business Excellence Framework and
Best Value
What is self-evaluation?
3 basic steps
How are we doing?
How do we know?
What are we going to do now?
What benefits are there?
 Enables us to take stock of current
performance
 Measures services in terms of progress
towards objectives
 Allows us to identify and understand our key
strengths
 Allows us to identify areas for improvement
 Supports us as we prioritise areas for action
Outputs of self evaluation
 Improvement plan which helps us use resources
and finances effectively
 Reporting to stakeholders, so improved
understanding of services
 Benchmarks for ourselves and others
 Training resources
 Case studies of good practice
 Underpinning a culture of quality
Why is self evaluation effective?
 More a process than an event
 Needs to be owned not imposed
 Must be inclusive not exclusive
 Must not depend on measurability
 Built in not bolted on
 Size does NOT matter
 Art not science
Nine Step Process
 Step 1 Orientation
 Step 2 Training for staff
 Step 3 Deciding which Key Result Areas to
review
 Step 4 Preparation of the evidence and
identification of key strengths and areas for
improvement with completion of template in
toolkit for review
Nine Step Process 2
 Step 5 Award star score
 Step 6 Visit and assessment with external peer
reviewers
 Step 7 Feedback from external review team and
a agreement is reached on the star award
 Step 8 Report
 Step 9 Improvement Plan
Key Result Areas and users
 Providing information
 Building skills
 Developing social
capital
 Adults
 Children
 Special Needs
 Groups
 Businesses
What are we going to evaluate?
Key Result Areas for Library Services
1. Providing gateways to information
2. Building individual skills, capability and wellbeing
3. Developing social capital
4. Demonstrating leadership and values
5. Designing, managing and improving systems and
processes
Overall
evaluation
KRA 1 Providing gateways to
information
1. Sufficiency, range and suitability of resources
Why is this important?
Library services should offer consistent and equitable access to print and digital
collections, internet and to a range of other resources for their communities.
Provision of a range of resources, fit for purpose and specific need allows
individuals to make informed choices. A successful library should have a high
percentage of use by its local population.
Key question to ask
To what extent are the information needs of all met?
Things to think about
What range of print and digital collections does the library offer across a range
of user groups?
How is the selection of information resources made?
How well are the information services used by the population?
What are the most frequently used services?
Scottish Library and Information Council
Gathering the evidence
Consulting users, non-users and staff
Observing activities
Considering measurable outcomes
Examining documentation and
resources
Evaluating the evidence
Best done as a team
Does evidence show a service
strength or an area for improvement
Is everyday a strength?
Some evidence is neither but shows
effective and necessary processes
Template in the toolkit
5 Star Awards
 Outstanding delivery of core and a
range of enhanced services
Excellent
Sector leading and worthy of wider
dissemination
 Major strengths
High standard of provision
Any weaknesses do not impact on
users’ experience
5 Star Awards 2


Important strengths that have a
positive impact
Weaknesses do not have substantial
adverse effect
Weaknesses are recognised and
action is being taken to improve
Satisfactory provision of core services
Provision with a few strengths and
examples of good practice
Weaknesses have some adverse
effects on some users
5 Star Awards 3

Limited range of services or weaknesses in a
core service
Weaknesses have a significant impact on the
quality of the users’ experiences
Planned action required for improvement
Illustrative models
These provide an example of what star
awards mean in practice. Between the
descriptions set out in the five star rating
system and these illustration, services are
asked to make their own star award. As
more services go through the process the
Library Network will be able to collect
information and benchmarks which will
also be helpful.
External review
It is always useful to have a comparative
view. Peers will be acting as external
reviewers of the self-evaluation process
and outcomes.
The Library Network will help co-ordinate
peer review and there is support
documentation for them.
Being the Best We Can – Pilot
Initial training, pilot library
Self-assessment, pilot library
Verification & report
Review & modify process
Training, second round pilots
Second round self-assessments
April
May-June
July
August
September
October March
Thoughts from working group
 The rigour of the process
 It’s a journey – it’s constantly informed
 Self evaluation which leads to a cycle of
improvement
 Be creative and consistent where you find the
evidence
 It’s about telling your own story
 It’s all staff engagement
 It’s a learning and knowledge experience
 It builds the evidence base for advocacy – locally,
state-wide and nationally!