Maximising the impact of your service (Maxim)
Overview- Part 2
January 2009
The importance of impact
Introduction
The purpose of this briefing is to give you an overview of the context of impact as well as to
highlight the benefits of impact maximisation and measurement. In addition, the briefing
introduces the reader to the complexity of impact measurement.
The context of impact
Markless and Streatfield (1) list three “obvious pressure points” that are driving impact
evidence-gathering. These are ‘the accountability obsession’, ‘the value for money drive’ and
the evidence-based working aspiration’.(1)
Performance Management and Accountability in Public Institutions
Markless and Streatfield (1) acknowledge that the idea of performance evaluation is not new.
However, this notion has spread to libraries of all sectors where there is the demand for
evidence of service efficiency and effectiveness (1).
The Information strategy for the modern NHS 1998-2005 White Paper of 1998 (2), contained
the government's recognition that improved health and patient care depends upon the
availability of information that is accessible and of high quality. The strategy also set out plans
which included the requirement to evaluate care, health improvements, evidence, and Trust
performance (1). Hence, NHS library and information provision was also to be evaluated (3).
Following this period of modernisation within the NHS, the Hill review, a national review of NHS
Health library services (4) was published in 2008. This review highlight four purposes of NHS
library services and provides detailed recommendations as to how these purposes might be
achieved.
Value for Money
Closely linked with performance management and accountability is the concept of service
effectiveness in cost-benefit terms (1). Simply put this can mean ‘getting more for less money’;
this may lead to a ‘funding based on results’ type approach (1). This makes gathering evidence
of impact of your service essential.
Poll and Payne (2006) also note the importance of demonstrating benefits gained from running
a library service to our users, not only our funding institutions. Electronic information is often
now freely available (or perhaps appears to be freely available even when libraries have paid for
access) and physical library visits are declining (5). Funding libraries is expensive - Poll and
Payne (2006) note that “… in the competition for scarce resources, it becomes vital for libraries
to show evidence of the impact and value of their services, preferably in quantified results” (5)
Evidence-based working aspiration
In recent years, evidence-based practice has become widely used in the medical field. A
definition of evidence-based practice is, "an approach to health care wherein health
professionals use the best evidence possible, i.e. the most appropriate information available, to
make clinical decisions for individual patients" (6). Similarly, the evaluation of impact in health
library and information services is an evidence-based practice because any library service
changes or developments are based on evidence.
User centredness
Lastly, it is important that Impact maximisation must be user focused. Creating services that are
focused upon the needs of users is a widely used and effective strategy in library and
information services development: "a service that is not attuned to the needs of its users may
become underutilised, redundant or even obsolete" (7)
Why is impact important?
NHS libraries usually measure the performance of their service using quantitative methods such
as usage statistics and user surveys (8). However, although this information is highly useful, it
does not explain why users use library services, nor does this type of data explain the difference
that these services have made to users. Impact evaluation is important because it is primarily
concerned with the usefulness of library services to users i.e. - the effectiveness of outcomes
rather than the efficiency of processes in information services (8).
What are the benefits?
The benefits of maximising impacts include:
 Increased usefulness of services for users
 Greater user satisfaction with services
 Increased usage of library services
The benefits of measuring impacts are that impact studies:



Can be used as tools to communicate service benefits to users
Can be a powerful tool for demonstrating accountability and justifying funding needs to
funding bodies and those responsible for resource allocation
Can be used to build partnerships and collaborations with users


Can help information services to determine which services should be expanded or
replicated
Can be used as a tool to single out exemplary services for recognition (9)
What are the difficulties in measuring impact?
Impact is defined as “any effect of the service (or of an event or initiative) on an individual or
group” (1). However, measuring this 'difference' can be complex.
Markless and Streatfield (1) in their glossary of terms relating to impact distinguish between
impact evaluation and impact measurement.
Impact evaluation: Evaluating impact using qualitative terms
Impact measurement: Evaluating impact using quantitative methods (numbers) (1)
Markless and Streatfield (1) advise against using a purely quantitative approach since this can
lead to concentration on outputs rather than outcomes. In practice, a mix of quantitative and
qualitative data collection methods is often used to capture information about impact.
Quantitative performance measurement methods such as user surveys statistically display the
usage of services and user satisfaction with services. Qualitative data collection methods such
as interviews usually provide a follow-up to the quantitative methods. Qualitative data can help
to explain why users are satisfied (or dissatisfied!) with services as well as to explain why users
use the services that they have identified in the quantitative survey and how these services
have affected them.
Despite the usefulness of quantitative and qualitative data collection methods in impact
evaluation, impact can sometimes be difficult to prove (10). For example, if a clinician starts to
prescribe statins to his/her patients, it is hard to prove if this alteration has been influenced by
the results of a mediated literature search on stroke prevention that he/she has received from
his/her NHS library (11). In certain cases, quantitative and qualitative data collection methods
can only enable services to make informed assumptions about the impact that their services
has on its users (8). Nevertheless, two recent studies, one a systematic review, have
demonstrated how library services can have a positive impact directly on patient care (12, 13).
These issues will be explored later in the Maxim course.
WHAT YOU NEED TO DO:
Read the following self-reflective questions and record your answers in your portfolio:
a) How has your library or information service attempted to maximise the impact of its services
to users? b) Did these impact maximising activities have any effect on your library or
information service?*
*If your workplace has not attempted to maximise impact, please provide reasons as to why this
may be the case. In addition, please provide examples of the ways in which impact could be
maximised in your library or information service and provide examples of the potential effects
that maximising impact could have on your service.
References
1. Markless, S; Streatfield, D (2006). "Evaluating the impact of your library" London,
England, UK: Facet Publishing.
2. NHS Executive. Information strategy for the modern NHS 1998-2005. London,
Department of Health, 1998. [Online] [Accessed January 2009]
3. Banwell, L. "Evaluating information services" Chapter 13, pp173 - 181. In: (eds) Andrew
Booth and Graham Walton, Managing Knowledge in Health Services. London, Library
Association publishing; 2000 [Online] [Accessed January 2009]
4. Hill, P (2008). Report of a national review of NHS health library services in England:
From knowledge to health in the 21st Century [Online] [Accessed January 2009]
5. Poll, R and Payne, P (2006). Impact measures for libraries and information services.
Library Hi Tech 24 (4): 547-562. Preprint at: http://conference.ub.unibielefeld.de/2006/proceedings/payne_poll_final_web.pdf [Online] [Accessed January
2009]
6. McKibbon KA (1998). Evidence based practice. Bulletin of the Medical Library
Association 86 (3): 396-401.
7. Booth, A. 'Identifying users' needs Chapter 7, pp101-111. In: (eds) Andrew Booth and
Graham Walton, Managing Knowledge in Health Services. London, Library Association
publishing; 2000 [Online] [Accessed January 2009]
8. O'Connor, P. Determining the impact of health library services on patient care: a review
of the literature. Health Information and Libraries Journal 2002; 19: 1-13.
9. Rudd, PD. Documenting the difference: demonstrating the value of libraries through
outcome measurement. Perspectives on Outcome Based Evaluation for Libraries and
Museums. Washington, Institute of Museum and Library Services; 1999 [Online]
[Accessed January 2009]
10. Everest, K and Payne, P. The Impact of Libraries on Learning, Teaching and Research
Report of the LIRG seminar held in Leeds. 30th October 2001. [Online] [Accessed
January 2009]
11. Roberts. S. Do libraries aid learning? Approaches and methods for measuring impact
Report about the conference. 9-10 December 2002, Scarborough. [Online] [Accessed
January 2009]
12. Weightman, A. L. & Williamson, J. 2005, "The value and impact of information provided
through library services for patient care: a systematic review", Health Information and
Libraries Journal, 22, 1, 4-25.
13. Lacey Bryant, S and Gray, A (2006). Demonstrating the positive impact of information
support on patient care in primary care: a rapid literature review. Health Information and
Libraries Journal;. 23: 118-125