Social Sciences, Health, and Education Library
University of Illinois at Urbana-Champaign
New Service Models
Evaluation Report
January 22, 2016
Introduction
In 2011, the Social Sciences, Health, and Education Library Implementation Team was
charged to “recommend a plan for assessing the effectiveness of the new configuration
of collections and services.” The purpose for assessing the new model is to identify and
continue successful practices and to flag areas for further improvement. The successful
implementation of this merger into the Social Sciences, Health, and Education Library
(SSHEL) took three full years to plan, organize, and implement. While SSHEL was
administratively established as a merged unit in August 2012, the actual opening of the
renovated spaces occurred in August 2013. As the planning took place, a number of
subject areas were folded into the proposal and modifications to services and policies
subsequently occurred after implementation to address user needs. This report will
address an initial assessment of SSHEL’s success and will be used as a benchmark for
later evaluations.
Background/History
Planning for the creation of a social sciences and health library began as part of the
New Service Model program within the University Library to address changing user
needs and expectations and Library resource concerns. A planning team for the merger
of social science and health libraries was appointed and completed its work as
documented in the Report of the Social and Applied Health Sciences Planning Team
(University of Illinois, 2011). An implementation team was charged to work from the
planning team’s report and develop strategies to bring the vision of this library to fruition.
The Final Report of that team outlines the challenges and decisions that were made
throughout the process (University of Illinois, 2012). To measure the success of the
implementation an assessment component was included in the process. Surveys and
other measures were initiated during and after the implementation. A plan for tiered
reference services was developed and is discussed below in Other Findings.
Information about the results of the steps taken to evaluate implementation appears in
the following sections.
2
Methods of evaluation
Prior to implementation as much data was gathered as possible through routine
statistical reporting. This information included reference interactions, circulation data,
information about collection size and content, personnel assignments, and so on.
In addition, a survey to constituent user groups was distributed to gather feedback
about service, collection, and space needs. The “final report reflects survey feedback,
thoughtful discussions, and an overarching commitment to the provision of high quality
services, collections, and space for library users.” (University of Illinois, 2012, p. 4) The
availability of this information gathered during the planning and implementation period
allows for comparisons to be made at periodic intervals as SSHEL evolves. It is
important to note that SSHEL continues to develop and is a growing, healthy
organization with checks and balances to allow for innovation and change as well as a
constant commitment to high quality service.
For post-implementation assessment, two methods were employed in order to gather
information for this report.
In 2015, Applied Health Sciences Librarian JJ Pionke developed a survey of SSHEL
services to distribute to students and faculty, using a web form. The survey was
designed to ascertain SSHEL’s efficacy and what improvements, if any, were needed.
An appendix contains selected survey questions and recommendations. SSHEL
librarians were consulted about the content and format of the survey prior to its
distribution. The survey was then publicized through announcements to the
departments, programs, and schools affiliated with SSHEL (352 tenure system faculty,
2,149 graduate students, and 5,858 undergraduate students). Due to variations in how
programs distribute announcements it is unknown how many students and faculty
actually saw the announcement. Surveys were received from 158 respondents.
Simultaneously, quantitative data was gathered. The Library Assessment Coordinator,
Jen Yu, was contacted for information from DeskTracker and other data sources in
order to perform a comparison to the data that had originally been evaluated when
SSHEL was being formed.
Recommendations
Based on survey and assessment data, several improvements can be made to
SSHEL’s services and collections. Below is a brief list of recommendations followed by
more detailed information from the survey responses.
•
Reduce noise from collaboration rooms.
3
•
•
•
•
•
•
•
Allow SSHEL Oak Street (S-Collection) books to be delivered to any campus
location rather than directly to SSHEL for condition review.
Improve study space by providing additional power outlets, better table lights,
and additional seating.
Increase marketing about services and facilities provided by SSHEL, as well as
by general library workshops on topics of interest.
Improve signage inside and outside SSHEL.
Improve accessibility to SSHEL, especially for those with special needs.
Investigate temporary storage options for library users who exit SSHEL for brief
periods of time and do not want to carry belongings with them.
Review test collection circulation procedures.
Data from other sources
Data was compiled by Assessment Coordinator Jen-Chien Yu for two periods (July
2011-June 2012 (FY12) and July 2013-June 2014 (FY14)). The intervening year
included a lot of transfer of material, transitions, and updating of records from the
merging libraries, so was not considered as a data source.
•
•
In reviewing some of the standard statistical data available through the Voyager
and DeskTracker systems, it is clear that SSHEL continues to be one of the
major sources of material for circulation to Library users. Prior to SSHEL’s
implementation (July 2011-June 2012), the circulation data show that SSHEL’s
collections accounted for 7.87% of the overall Library circulation. After
implementation (July 2013-June 2014), SSHEL’s circulation remains at a similar
7.90% of the overall Library circulation despite the significant reduction of the
collection size in 2012. This high use is attributed to the interest in the subject
areas encompassed in SSHEL and the fact that much of the material is still in
print format. The steady circulation also suggests that decisions about what
materials to send to the stacks and Oak Street were effective in identifying the
items least likely to be used.
For reference and directional questions, SSHEL answered a total of 7,670
questions from July 2013 through June 2014. This number is similar to that of
the merging libraries data in 2011/2012 (8,299 questions). The highest number
of questions identified by subject were for education and children’s literature,
followed by library and information science. It is notable that a large number of
research and ready reference questions are not assigned a subject so that
further training of SSHEL reference staff will be undertaken about the importance
of assigning subjects. Designation of subject area provides a better
understanding of those subjects that are of interest to users and that may require
additional support or training of personnel. In addition, subject questions ranged
4
•
•
from agriculture to theatre, reinforcing the perception that as a first floor service
point, library users tend to ask questions at the most convenient location.
Gate counts for the two comparative time periods reflect an enormous increase
in use of the libraries. In 2011/2012 the combined gate counts for merging
libraries was 82,688. In 2013/2014 the gate count for SSHEL was 188,144. This
is a significant change that reflects the welcoming nature of the renovated space
and the increased popularity among students and others seeking study space as
well as requesting assistance. Supplemental data provided by the Library
Assessment Coordinator shows that SSHEL and Research and Information
Services have similar patterns of use and high room counts for fall 2014. This
affirms the need for large welcoming study spaces for researchers and others in
the Main Library building in easily accessible locations.
Call slip statistics prior to and following the merger are very similar (19,919 filled
in FY12 and 19,341 filled in FY14) indicating that use of material continues to be
high while collections were significantly reduced in size and transferred
elsewhere. Data for call slips requested from Education and SSHEL Oak Street
locations for those time periods is not available, which would have indicated if
making the children’s and young adult literature collection more browsable
decreased demand for off-site material.
Other findings
SSHEL Librarians and staff were successfully incorporated into reference services
provided more broadly within the Main Library and virtually through participation in the
central reference hub. SSHEL Graduate assistants contribute to the virtual reference
service during evenings and weekends, while designated SSHEL staff and all librarians
contribute regular weekly hours to both the central hub and SSHEL reference services.
SSHEL has used a tiered reference service model since its inception, and made
adjustments each year based on what works, best practices, and review of statistics on
peak hours and low use times of the day and week. Initially back-up reference staff
were assigned to each afternoon hour but after a year it was determined that this was
not the most effective use of time. The subsequent year a pilot was implemented to
have designated librarians on-call during afternoon hours but not seated at a back-up
desk specifically. Librarians were encouraged to rove with an iPad during those on-call
hours. After reviewing the data for the pilot program it was decided to eliminate back-up
or on-call services and depend instead on the general availability of experienced
librarians during daytime hours. That program has worked well in the fall 2015
semester, and will be evaluated at the end of the spring 2016 semester.
Instruction services increased notably from the four merged service points (AHS, ESSL,
Labor and Employee Relations, and Library and Information Science) with the increased
5
visibility and marketing that occurred when SSHEL was opened. The FY12 statistics
were for 128 instruction sessions for 2,570 participants while the FY14 statistics were
for 157 sessions with 3,397 participants.
Conclusion
The merger of multiple libraries and subject areas into SSHEL has been successful,
although the statistical data doesn’t fully reflect that. Much of the data shows little
change from the time prior to implementation to post-implementation. What is not
reflected in those figures, however, is the significant decrease in physical collections onsite, the increase in user seating in welcoming, pleasant facilities, and the impact of
having multiple subject specialist librarians available in a single facility. The fact that
circulation remains steady despite fewer books being available is notable. In the
process of implementation, every effort was made to keep the most current and heavily
used items in SSHEL while transferring lesser used material elsewhere. This is an
ongoing process and keeps the print collection vibrant and interesting. While material is
also selected in electronic format, much of it is acquired in print still, although that is
likely to change over time.
Another aspect of the merger is that it not only improved facilities for library users but
also for SSHEL personnel. With workspaces designed to encourage productivity, rather
than being cobbled from surplus equipment, personnel now have spaces in which to
work more effectively.
Library users have participated in multiple surveys since implementation so that we can
better determine if their needs are being met. Surveys included general information,
web page studies, and noise control surveys. SSHEL will continue to solicit feedback
from users through surveys, usability studies, focus groups, and interviews. Information
from those sources coupled with data from DeskTracker and Voyager reports will
continue to inform decisions to modify hours of service, services provided, and facilities
improvements.
Challenges facing SSHEL include user concerns about noise, especially noise
generated from the collaboration rooms, since SSHEL is clearly a preferred study space
for many of our constituents. Appropriate hours based on use and budgetary issues will
continue to be reviewed. While longer hours are preferable, they may not be practical.
Each of the recommendations listed on pages 2-3 will be investigated for possible
implementation since these are user-based recommendations. Some issues such as
improved signage and revisions to test collection procedures are already underway.
While this report is the post-implementation evaluation required as part of the New
Service Model program, ongoing evaluation will monitor if user needs are being
6
addressed and how we can better assist library users. SSHEL is not viewed as a static
operation but as an evolving, thriving part of the Library and university community.
References
University of Illinois at Urbana-Champaign, Social and Applied Health Sciences
Planning Team. (2011, January 10). Report of the Social and Applied Health Sciences
Planning Team. Retrieved from
http://www.library.illinois.edu/nsm/social/sahs_planning/SAHS_NSM_Final_Report.pdf
University of Illinois at Urbana-Champaign, Social Sciences, Health, and Education
Library New Service Models Implementation Team. (2012, March 1). Final report.
Retrieved from http://www.library.illinois.edu/nsm/social/SSHEL_Final_report.pdf
7
Appendix
2015 Survey and Results
Finding 1: Patrons like SSHEL for studying.
Generally speaking, respondents of the survey have a positive view of SSHEL. In
response to question 2: Using one or two words, how would you describe SSHEL? They
frequently used words like: helpful, useful, welcoming, and quiet. Question 4 asked why
respondents came to SSHEL. 32% said they come to study by themselves.
Recommendation
Continue modifying SSHEL to better suit study needs including more comfortable
seating, table lamps for better light quality, and more power outlets as indicated in
Question 8 which asked what improvements should be made.
Finding 2: Patrons aren’t sure where SSHEL is and what it offers.
Question 3 asked how often the respondent came to SSHEL in the last academic year.
The majority answer at 38% was Never. As the survey was sent to patrons of SSHEL
as designated by subject area, the fact that slightly over a third of them never actually
come to SSHEL indicates that more marketing and outreach is needed about our
location and also on the services and materials that we offer.
Recommendation
Develop a focused marketing and outreach campaign not only by the larger University
Library but also individually by librarians to their subject areas.
Finding 3: Patrons have difficulty finding materials.
Because of the split room nature of SSHEL, materials are located in specific areas.
One respondent in Question 12, which asked for one thing that made it hard to use
SSHEL, stated, “It’s hard to figure out where things are located/follow the directions to
find the location of a book you found online.” Another respondent, to the same
question, said that, “For printed material, the library is really divided and it’s hard to
know where to find what.”
Recommendation
Develop better signage including wayfinding, within SSHEL South and North as well as
clear signage outside of each room to indicate contents, such as quiet study areas, and
librarian locations.
8
Finding 4: SSHEL has some accessibility needs.
Question 9 specifically asked what would make SSHEL more accessible to patrons.
The question included a few examples such as having a table without chairs that a
wheelchair user could roll up to, having the security gates spaced wider apart, and
having an assistance bell located near the printer and scanner.
Recommendation
Implement these suggestions (table to roll up to, wider security gates, and an
assistance bell) that respondents indicated would be useful.
Finding 5: Patrons feel overwhelmed with finding information.
While this finding is not necessarily SSHEL specific, it does indicate that there might
need to be more outreach or workshops that focus on information literacy and critical
thinking skills.
Recommendation
Develop a series of workshops that focus on information literacy and critical thinking
that are held in SSHEL and then advertised to constituent groups.
Finding 6: Patrons are frustrated that there is no way to leave their stuff behind in
order to briefly leave SSHEL.
A common complaint throughout the survey was having to pack up everything in order
to run to the restroom or step into Marshall Gallery to have a snack.
Recommendation
Explore options for storing belongings either through a locker system or through a
cubby system that is behind the information desk. Either system will need clear signage
that items left behind will be discarded at the end of the day.
Finding 7: Patrons are also frustrated when ordering SSHEL S-Collection
materials from Oak Street and having to pick them up in SSHEL.
This policy was implemented in order to do a preservation assessment of S-Collection
materials. However, the policy then requires materials to be sent to SSHEL from Oak
Street which prevents the patron from having the material sent to their preferred library.
Recommendation
Review the policy.
9
Finding 8: Patrons that use the Speech and Hearing Test Collection are confused
by policies.
A consistent thread that emerged in the survey was a deep confusion over who could
use the Speech and Hearing Test Collection and what the policies for use were.
Recommendation
Review policies and clarify.
Finding 9: Patrons find the process of getting to articles burdensome.
While not necessarily a SSHEL issue, it is important to note that patrons struggle with
the multiple steps it takes to get to journal articles both on and off campus. One
respondent stated in Question 8, which asked for improvement recommendations,
“When working at home, I’m unable to access e-journals even though I have my VPN
activated.” Another respondent remarked in Question 10, which asked about barriers to
information, that, “[H]aving to go through multiple logins is really time consuming at
times.”
Recommendation
Forward comments and observations to the Web Team so that they may have the
feedback to consider for making changes in the future.
Finding 10: Patrons are conflicted about the study rooms.
While respondents like that there are designated areas for group study, the fact that
they have no ceiling creates a noise issue that is a burden on those studying quietly
nearby and for library staff. One respondent stated in Question 8, which asked for
improvement recommendations, “[I] avoid pair or group meeting because as much as it
was designed to control noise, it can get loud and distracting for those of us who are
working quietly at tables. This is an issue as I go to SSHEL specifically because of how
quiet it is and how well-lit it is for reading.” Another respondent, to the same question,
said, “An enclosed group meeting room. The current meeting room is not soundproof at
all and the entire library can hear what is happening inside.”
Recommendation
Library Facilities has stated unequivocally that it is impossible to enclose the study
rooms because of HVAC issues. Therefore, consider eliminating the study rooms as a
potential solution. This would free up space for more study areas such as tables,
carrels, or couches.