Measuring Value:
A Survey for Assessing Our Impact
Victoria H. Goode, MLIS
Welch Medical Library
Clinical Informationist
vgoode1@jhmi.edu
About the Survey
• Purpose:
– To better understand the effect of information and
informationist services on clinical and research decisions.
– To understand the value of selected library services.
• Population:
– Faculty, fellows, housestaff & residents in the Johns
Hopkins schools of medicine, nursing and public health.
• Dates:
– First two weeks in November 2011
About the Survey
• Used the Critical Incident Technique
• Consisted of 22 questions and a rating scale of 12
items.
• Distributed via email and an intranet site.
• Coincided with other campus wide
events/campaigns, and this may have limited
participation.
• Partners: Johns Hopkins Marketing and
Communications and the Johns Hopkins
Biostatistics Center.
Response Rate
• Response rate: 10.8%
– (809 completed surveys out of a possible 7,490)
School
SOM
SPH
1%
Status
SON
Faculty
Fellows
Housestaff/residents
13%
27%
29%
72%
58%
Information Needs
N=765
2%
Research
10%
Education
13%
Patient Care
75%
Management &
Other
Benefits to Research
Top Reported Benefits
N=552
Publication output
63%
Grant proposals
36%
Citation impact
Conference output
Textbook
33%
17%
11%
Patient Care
Top Reported Positive Changes
N=74
Advice given to patient or family
47%
Diagnosis
42%
Choice of other treatments
39%
Choice of test
31%
Choice of drugs
30%
Handled the situation differently
11%
Patient Care
Top Reported Avoided Events
N=47
Misdiagnosis
45%
Additional tests or procedures
32%
Medication error
Adverse drug reaction or interaction
30%
17%
The Value of Library Services
Library services were rated on a ten-point scale, with "1"
meaning not valuable and "10" meaning very valuable
Top Valued Services
Scale
Making a wide range of journal articles, books and databases available at
your computer
9.3
Bringing together all needed information sources to one online location
8.7
Analyzing the results of a search and selecting most relevant articles
7.9
Identify an applicable information source to find what you need
7.7
Training to improve your searches
7.4
The Value of Informationist Services
55% used the services of an informationist either recently or in the past.
91% of people who used informationist services would recommend it to others
Top Reported Benefits
N=405
Providing expertise in finding information
73%
Providing expertise in available databases
70%
Saving time
70%
Helping to find additional information
60%
Providing added thoroughness
59%
Reducing workload burden
40%
Comments
• Many of the improvements we made following
this survey came from the comments we received
in open-ended questions.
• A question regarding barriers to finding
information provided important examples of
access problems.
• The informationist program received almost all
positive results, but many people also
commented on not knowing it existed.
What Improvements Were
Made as a Result of This
Survey?
Access
• A large number of patrons access our
resources from off-campus, so improving
remote access to resources is vital.
• A more streamlined access issue reporting
structure was put in place so that problems
can be resolved faster and tracked over time
to see trends.
Library Service Development
• Based on comments, there is a high need for
information on data/statistics.
– Review and further develop library services in
support of data.
• We will continue to seek user input on new
library services and resources.
• Additionally, we will conduct periodic user
reviews of key services.
Marketing and Communication
• We need to improve communication broadly
with the community.
• New methods of communication include
manning information tables throughout the
campus and a monthly podcast.
• New and under-used resources are
highlighted on the Welch homepage.
New Informationist Collaborations
• Evidence-based Medicine Rounds were
implemented in August 2012 with the Junior
Assistant Residents (JARs) in Internal Medicine.
• We improved our information resource pages and
added new information portals, like the
Bioinformatics portal.
• ICU pilot project to determine how providers
answer questions that arise on an Intensive Care
Unit.
Models for the Survey
•
Grefsheim, S. F., Whitmore, S. C., Rapp, B. A., Rankin, J. A., Robison, R. R., &
Canto, C. C. (2010). The informationist: Building evidence for an emerging
health profession. Journal of the Medical Library Association, 98(2), 147-156.
•
Grefsheim, S. F., & Rankin, J. A. (2007). Information needs and information
seeking in a biomedical research setting: A study of scientists and science
administrators. Journal of the Medical Library Association, 95(4), 426-434.
•
Marshall, JG (Primary Investigator). Value of Library and Information Services in
Patient Care Study. A partnership of the National Network of Libraries of
Medicine, Middle Atlantic Region and the University of North Carolina at Chapel
Hill. http://nnlm.gov/mar/about/value.html
•
Association of College and Research Libraries. Value of Academic Libraries: A
Comprehensive Research Review and Report. Researched by Megan Oakleaf.
Chicago: Association of College and Research Libraries, 2010
www.acrl.org/value
Thank you
Welch Library Value Survey Team
Jaime Blanck
Changxin (Jack) Chen
Victoria Goode
Nancy Roderer
Lori Rosman
Stella Seal
Sue Woodson
Rob Wright
Contact Information:
Victoria Goode
Welch Medical Library
vgoode1@jhmi.edu