Putting Statistics into
Practice Strategies for effective
management
J Eric Davies & Claire Creaser
IFLA Satellite 14/8/02
1
Options for measuring &
managing
J Eric Davies
IFLA Satellite 14/8/02
2
Outline
•
•
•
•
•
Mission, vision, aims objectives
Range of data types
Applications of data
Methods of acquiring data
General principles
IFLA Satellite 14/8/02
3
Measuring with meaning
• Every organisation, no matter what its
mission or scope needs three kinds of
performance metrics • to measure its success in mobilizing its
resources,
• its staff’s effectiveness on the job, and
• its progress in fulfilling its mission.
– McKinsey Quarterly. 2001 -2
IFLA Satellite 14/8/02
4
Mission with meaning
STRATEGIC FOCUS
• WHAT • Organisation? NOW / LATER
• Service? NOW / LATER
• Direction? NOW / LATER
• Mission / Vision
• aims / objectives
IFLA Satellite 14/8/02
5
Criteria
AIMS/OBJECTIVES
Specific
Measurable
Acceptable
Realistic
Time-bound
{SMART}
Consistent
Testing
{CUTE}
IFLA Satellite 14/8/02
Unambiguous
Empowering
6
How (im)possible is your mission?
MISSION
• Statement of purpose and functions – why
service exists, what it does, who it serves
VISION
• Statement of desired future state – where
service wants to be
IFLA Satellite 14/8/02
7
How (im)possible is your mission?
MISSION/VISION STATEMENTS ~
•
•
•
•
MEANING
CREDIBILITY
ACCEPTABILITY
TESTABILITY
IFLA Satellite 14/8/02
8
How (im)possible is your mission?
MISSION/VISION STATEMENTS ~
Strathclyde University – Glasgow
• A place of useful learning [1726]
• The place of useful learning [2000]
IFLA Satellite 14/8/02
9
IFLA Satellite 14/8/02
10
Here’s one I prepared earlier!
MISSION/VISION STATEMENTS ~
• Public Library - The mission of the Library is to
serve as a cathedral of human knowledge—
an accessible database of knowledge that
serves as the community's memory—and as an
information and knowledge safety net, while
providing materials, programs, and services to
the people of the community.
IFLA Satellite 14/8/02
11
Digging deep for evidence
What Kind Of Evidence [Information]?
• Statistics and Performance Indicators
– [Quantitative + Qualitative ]
• Social Measures
– [Soft Indicators] ‘DISTANCE TRAVELLED’
IFLA Satellite 14/8/02
12
Who cares {or should do}?
AUDIENCE: [stakeholders]
• Funders
• Managers/Staff
• Users
• Community
• Vendors
• Global
IFLA Satellite 14/8/02
13
All kinds of measuring
• Inputs
• Outputs
• Outcomes
• Impacts
IFLA Satellite 14/8/02
SERVICE DOMAIN
s
e
r
v
i
c
e
USER RESPONSES
14
Social dimensions
Examples of ‘Soft’ Indicators:•
•
•
•
Attitudinal
Personal
Practical
Key Work Skills
DISTANCE TRAVELLED
IFLA Satellite 14/8/02
15
Social dimensions
• Personal development - individual selfconfidence, self awareness, creativity, new
skills and abilities.
• Social cohesion - Impact on group/community
identity
• Community involvement and empowerment • Health - people feeling better, happier etc.
IFLA Satellite 14/8/02
16
How does the evidence add up?
APPLICATIONS ~ SERVICES & PROJECTS
• Policies
• Strategies
• Tactics
• Processes and
• Operations
• Advocacy
IFLA Satellite 14/8/02
17
What does the evidence answer?
APPLICATIONS ~ SERVICES
• How have we done?
• How are we doing now?
• How can we do better?
• Where are we going?
• How do we get there?
• How are we making a difference?
• How do we get the resources
IFLA Satellite 14/8/02
18
What does the evidence answer?
APPLICATIONS ~ PROJECTS
• Did we achieve what we were seeking to achieve?
• Did we do what we said we would do?
• How did we do it?
• What did we use?
• What did we get out?
• What worked and what didn’t work?
• What could we do differently?
• What can we apply continuously?
• What difference did it make that we did it?
• Who benefited?
IFLA Satellite 14/8/02
19
Managing and measuring
Framework for Performance Measurement:• integration
• user satisfaction
• effectiveness (delivery)
• efficiency
• economy
Follett Report – academic libraries
IFLA Satellite 14/8/02
20
Managing and measuring
Three E’s
–
Economy in acquisition of resources
–
Efficiency in the use of resources
–
Effectiveness in the achievement of objectives
UK Treasury [1980’s] FMI
Sizer [1980’s]
IFLA Satellite 14/8/02
21
Comparing and changing
• BENCHMARKING
– Motorola + D.E.C. + Xerox
– To make changes that lead to quantum and
continuous improvements in products, processes
and services that result in total customer
satisfaction and competitive advantage
IFLA Satellite 14/8/02
22
Comparing and changing
BENCHMARKING –
• Evaluate the level of performance of various
services within an institution
• Overall level of institution performance
• Compare against published standards
• Compare performance over time
• Compare with other institutions
IFLA Satellite 14/8/02
23
Finding out
Gathering Evidence • What do you need to know?
• Where is the information?
• Who has the information?
• How will you get it?
• How accurate is it / do you need it to be?
• How will you interpret it?
• How will you act on it?
• How will you present it?
IFLA Satellite 14/8/02
24
Gathering evidence
Techniques/Tools/Options for Gathering Data:• MIS / Transaction Logs
• Databases / Publications
• Surveys : questionnaire,telephone, interview
• Focus Groups / Graffiti boards
• Observation / Diaries / Logs
• Press~Media Coverage
IFLA Satellite 14/8/02
25
Gathering evidence
TOOLS:• What Outcomes, Dimensions, Performance to
be measured?
– reliable + valid
– meaningful and precise
IFLA Satellite 14/8/02
26
Gathering evidence
Options for Gathering Data • … if the only tool you have is a hammer,
everything starts looking like a nail.
F.W. Huibregston - Partner: McKinsey’s
IFLA Satellite 14/8/02
27
Changing times; changing evidence
UPDATING EVIDENCE
• Service Evolution
– New/Discontinued services - methods technologies - clients
• Diminishing Variance
– Improvement - Gaming - Deception
IFLA Satellite 14/8/02
28
Manager beware!!
OVERDOING IT:
If you know everything, you know nothing
George Johnson: Fire in the Mind. [1996]
… a world that never measures or counts is
really beyond our control. The trouble is that
we’re in danger of doing little else.
David Boyle: RSA Lecture . [2001]
IFLA Satellite 14/8/02
29
How much evidence?
... data is not information. Information is data
endowed with relevance and purpose. A
company must decide what information it
needs to operate its affairs, otherwise it will
drown in data
Peter Drucker - Managing for the Future.
IFLA Satellite 14/8/02
30
Making sense of measuring
Sumsion’s Law of Statistical Dullness ~
In comparative statistics the great majority of
results are inherently close to the average
and consequently dull.
{Sumsion}
LIRN 2001 (79) p.3.
IFLA Satellite 14/8/02
31
Evidence for yesterday
Statistics, being essentially historical, can only
provide information after the event.
{Sumsion}
Int. Encyclopedia of Lib. and Info. Sci.(1997)p.432
IFLA Satellite 14/8/02
32
Measuring and managing
Information is a precondition for identifying
choices, reducing uncertainty about their
implications and facilitating their
implementation.
Center for Transnational Corporations:- CTC
Reporter 14 Winter 1983 p.34 -
IFLA Satellite 14/8/02
33
Managing and measuring; comparing and changing
• LISU: We’ve got the measure of information!
– A skilled team of experienced Managers,
Statisticians and Administrators all adding value
to statistical data and providing authoritative and
reliable information to support managers in
culture, information and related environments.
IFLA Satellite 14/8/02
34
Library and Information
Statistics Unit
lisu@lboro.ac.uk
IFLA Satellite 14/8/02
35
Mission possibilities
• Does it have
– Meaning?
• Does it actually mean anything?
– Credibility
• Do you believe it can be achieved?
– Acceptability
• Will all the stakeholders (funders, staff, users) ‘buy-in’ to
this mission?
– Testability
• How would you demonstrate you are achieving your mission?
IFLA Satellite 14/8/02
36
Library and Information
Statistics Unit
lisu@lboro.ac.uk
IFLA Satellite 14/8/02
37
Statistics for the faint
hearted
Claire Creaser CStat
IFLA Satellite 14/8/02
38
Introduction to statistics
• Basics
– What are statistics
– Useful techniques
• Sampling
– Surveys and sample sizes
– Questionnaire design
– Analysis
• Benchmarking
• Presentation of results
IFLA Satellite 14/8/02
39
What are statistics?
• Numbers with context
–
–
–
–
1,300 items issued last month
The average price paid for a CD is £12.50
25% of staff time is spent re-shelving books
Women borrow twice as many books on average as
men
– Serials cost three times as much as books
– The average spend per user has increased less
than general inflation over the last ten years
IFLA Satellite 14/8/02
40
Where to start
• What do you want to know?
– Evidence of good management
– Value for money
– Advocacy
• What data to collect?
– What do you want to know?
• Relevant
• Useful
• Current
IFLA Satellite 14/8/02
41
Where do they come from?
• Library management systems
– Stock statistics, financial data, staff . . . .
• Regular surveys
– User opinions, condition of stock . . . .
• Occasional surveys
– Project evaluation
IFLA Satellite 14/8/02
42
What do they look like?
• Categorical
– Gender; classmark; membership status
• Ordinal
– Stock condition; satisfaction ratings
• Ratio or interval
– Acquisitions; issues; expenditure
IFLA Satellite 14/8/02
43
What can you do with them?
• What do you want to know?
• Descriptive statistics
– Mean, range, distributions, proportions
– graphical presentations
• Inference from samples
– Estimates, error levels
• Advanced techniques
– Correlation, regression, analysis of variance
IFLA Satellite 14/8/02
44
Choosing the right technique
• Keep it simple!
• Categorical data
– Proportions in each category
– Comparisons
• Ordinal data
– Proportions in each category
– Medians
• Ratio data
– Means
IFLA Satellite 14/8/02
45
Sample surveys
• Why sample?
– Cost
– Practicalities
• Where to start?
– Sampling frame
– Sample design
– Sample size
IFLA Satellite 14/8/02
46
Types of sample
•
•
•
•
•
Simple random
Systematic
Stratification and clustering
Quota samples
Self-selected
IFLA Satellite 14/8/02
47
How many?
• Less than you think!
• Depends on:
– Level of detail
– Desired margin of error
– Expected response rate
• Does not depend on population size
– Unless small population
• 400 will give accuracy of ± 5%
• 1,000 for ± 3%
• 2,500 for ± 2%
IFLA Satellite 14/8/02
48
Questionnaire design
•
•
•
•
Self-completion or interview?
Clear, unambiguous questions
Clear, easy to follow layout
As short as possible
– Number of questions
– Number of pages
• Tick boxes or short answers
• Data entry issues
IFLA Satellite 14/8/02
49
Sampling times
• One period, or several?
• Periodicity
IFLA Satellite 14/8/02
50
Analysis
• Lots of software available
– Spreadsheets
• Excel, Lotus
– Specialist packages
• SPSS, Minitab
• Keep it simple!
• If in doubt
– bring in a statistician!
IFLA Satellite 14/8/02
51
Making comparisons - benchmarking
• Compare like with like!
• Within your service
– Performance of service points
– Behaviour of user groups
• Externally
– National standards
– Other libraries
• Over time
– Trends
IFLA Satellite 14/8/02
52
Presentation of results
• Appropriate to
– The data type
– The audience
• Tables or graphs?
• Key features:
–
–
–
–
Clarity
Integrity
Descriptive
Appropriateness
IFLA Satellite 14/8/02
53
Graphical forms
• Line graphs
– For presenting trends over time
• Pie charts
– For presenting proportions of a whole
• Bar or column charts
– For comparing categories
• Scatter graphs
– For examining relationships
IFLA Satellite 14/8/02
54
Issues per capita in UK public libraries
300
250
200
Adult nonfiction
150
100
Children's
AV
50
99-00
98-99
97-98
96-97
95-96
94-95
0
93-94
26
28
30
31
33
34
36
37
37
38
38
Adult
fiction
92-93
105
105
113
112
112
112
110
111
111
109
101
350
91-92
AV
90-91
333
326
325
314
303
289
275
261
246
234
221
Children's
89-90
89-90
90-91
91-92
92-93
93-94
94-95
95-96
96-97
97-98
98-99
99-00
Adult
nonfiction
129
129
138
136
136
133
129
127
122
114
107
millions
Adult
fiction
Source: LISU Annual Library Statistics
IFLA Satellite 14/8/02
55
Breakdown of UK academic library expenditure
Staff
Information
Of which:
Books
Serials
Electronic
ILL
Binding
Equipment
Other
£128m
£92m
£27.0m
£48.6m
£8.4m
£3.5m
£4.9m
£20m
£17m
Equipment
8%
Other
7%
Serials
19%
Electronic
3%
Information
provision
36%
ILL
1%
Staff
50%
Books
10%
Binding
2%
Source: LISU Annual Library Statistics
IFLA Satellite 14/8/02
56
Activity in public libraries
Proportion of visitors who:
Borrow/return books
74.2%
Borrow return cassettes 7.0%
Borrow/return CDs
4.9%
Borrow/return videos
8.4%
Read newspapers
13.8%
Seek information
22.9%
Use photocopier
6.0%
Browse
29.9%
Study
7.7%
Visit event
2.9%
Other
7.8%
Borrow/return books
Browse
Seek information
Read newspapers
Borrow/return videos
Study
Borrow return cassettes
Use photocopier
Borrow/return CDs
Visit event
Other
0
20
40
60
80
% of visitors
Source: CIPFA Public Library User Survey
IFLA Satellite 14/8/02
57
Issues
630,788
472,514
424,045
413,133
474,982
297,387
328,578
225,841
267,913
98,050
70,349
113,049
0.7
0.6
0.5
Issues
Acquisitions
A1
12,242
B1
10,730
B2
7,184
B3
8,121
B4
6,075
C1
2,895
C2
4,155
C3
3,572
C4
4,940
D1
1,488
D2
1,435
D3
2,471
………
Millions
Public library book acquisitions and issues
0.4
0.3
0.2
0.1
0.0
0
2
4
6
8
Acquisitions
10
12
14
Thousands
Source: UK public library authority data, unpublished
IFLA Satellite 14/8/02
58
Library and Information
Statistics Unit
lisu@lboro.ac.uk
IFLA Satellite 14/8/02
59
Managing scenarios, strategies and
statistics in practice
•
•
•
•
•
•
•
What do you need to know?
Where is the information?
Who has the information?
How will you get it?
How will you interpret it?
How will you act on it?
How will you present it?
IFLA Satellite 14/8/02
60
Library and Information
Statistics Unit
lisu@lboro.ac.uk
IFLA Satellite 14/8/02
61
ISI Glossary
Access from
http://europa.eu.int/comm/eurostat/research
/index.htm
IFLA Satellite 14/8/02
62