Best-Fit Evaluation Strategies:
Are They Possible?
John Carlo Bertot, John T. Snead,
& Charles R. McClure
Information Use Management and Policy Institute
College of Information, Florida State University
http://www.ii.fsu.edu
Introduction

Received a grant from the Institute of Museum and Library
Services (IMLS) to develop an online instructional system to
assist public libraries evaluate their services/resources



Evaluation Decision Management System (EDMS)
In year two of three year project
Why this project?




Little research provided comprehensive assistance in determining
what specific evaluation strategies serve libraries best relative to
specific library situational factors and contexts, data needs, and
other considerations
With so many evaluation options available, there was a need to
bridge information-need issues (i.e., situational factors, data needs,
stakeholder questions, etc.) with evaluation approaches
Understanding information needs and linking these needs to
evaluation approaches required evaluation strategies capable of
providing data that library decision makers can use to address
specific problems
Bridge practice an research
Best Fit Evaluation Approaches

The original intent of the project was to
develop


Best-fit evaluation strategies which match the data
needs of a library within a specific situational
context to the evaluation approaches that are
most appropriate to that particular situational
context
Identify which evaluation strategies supply the
best data within specific library organizational and
situational contexts for use to provide the greatest
impact to improve library services, or enable
libraries to better advocate the value of libraries to
their institutions or the communities they serve
Best-Fit Evaluation Strategy
Considerations





What evaluation approaches are available
Which evaluation approaches might best meet a
library’s data needs, either library developed or
imposed by external funders/organizations/etc.
How to develop an overall evaluation plan that makes
effective and efficient use of limited library resources
How to implement an evaluation strategy
How to use evaluation findings to advocate for local
library support
The Original Approach

Academic Literature








Outcomes assessment
Value demonstration
ROI
Outputs
Service Quality
Other
Stand alone literature and approaches
Bring these to the public library community
Different Perspectives

Stakeholder Perspective



Answer a range of questions asked by various stakeholders
groups (e.g., library boards, county or city executives)
regarding library services and resources
Make informed decisions regarding a library’s range or
availability of services and resources
Data Perspective, provide data to





Answer a range of questions asked by various stakeholders groups (usercentered evaluation perspective)
Make informed decisions regarding a library’s range or availability of services
and resources (library-centered evaluation perspective)
Demonstrate value and effectiveness of the library to the community that it
serves (community-centered evaluation perspective)
Frame the perceptions of the library in the local political environment (political
context-centered evaluation perspective)
Support the notion of the library as serving as a public good (customercentered evaluation perspective)
Then the Reality of Public
Libraries


Advocacy
Tell story





Value
Impact
Cuts across individual evaluation
approaches
Integrated
Fits library’s situation
Development of Best-Fit
Evaluation Strategies



Library decision makers are often faced with
difficulties matching their data needs with the
appropriate evaluation approaches
There are many different kinds of evaluation
data that a library may need and evaluation
approaches that a library might employ
As a result, many libraries struggle with the
problem of choosing the best evaluation
approaches to effectively and efficiently
demonstrate the value they provide

And…they don’t necessarily consider “evaluation”
Some Issues and Findings

Have not quite abandoned the “logic model”
approach to evaluation and “best fit”, but
have


Modified thinking significantly
Changed how we make evaluation tools available


Less “evaluation like”
Focused more on advocacy and story telling -with evaluation on the side

Though it does underlie the entire approach
Some Issues and Findings

In essence, developing a community of
practice



Resources
Access to experts
Information commons for asking questions and
sharing ideas, resources, and other material



Successes and failures
How to
Other
Some Issues and Findings

Public libraries don’t necessarily consider evaluation as part of their
advocacy strategy (if they have one)


The link between evaluation, data, and advocacy is not always apparent
Once burned, twice shy

If have used evaluation strategies and data collection efforts, and not met
with success, there is a reluctance to do so again



Library situational factors (organizational, community, other) affect the
successful use (or unsuccessful use) of leading evaluation approaches
Though libraries now talk about advocacy regularly -- the extent to
which they are able and/or ready to engage in advocacy strategies is in
question


Regardless of a linkage between evaluation and advocacy
The fear is that we may have to hide evaluation in an advocacy
wrapper


Waste of resources
These are two different things and require different skill sets
The ability of a library director to negotiate the political
environment of the library may trump evaluation
Some Issues and Findings

Need to provide as many tools as possible to
facilitate evaluation

If, then interfaces that enable EDMS to know
something about libraries



NCES data (operating budgets, population served, FTEs,
other)
Public Library Internet data (connectivity, bandwidth,
workstations, services, other)
Prepackaged reports which incorporate data



Peer comparisons
Brochures
Presentations (Budget, building, other)
Moving Forward

Challenges remain



Biggest is making link between evaluation
and advocacy
“Painless” evaluation that meets library’s
situational factors
Redefining success of project
Conclusions




Some libraries do continue to engage in a wide range of
evaluation efforts to assess services and resources provided to
the communities libraries serve
The evaluation environment is increasingly complex, and
requires knowledge of multiple evaluation frameworks,
methodologies, data analysis techniques, and communication
skills
The issue is not that libraries face a lack of available evaluation
approaches
The issue is helping libraries select the approach or approaches
that best meet advocacy needs of a library from the many
evaluation techniques that exist

Making the evaluation-advocacy link
Thank You

Questions/Comments?

Contact information

John Bertot: jbertot@fsu.edu

Information Institute: http://www.ii.fsu.edu