Subject Search Disconnect
Margaret Mellinger and Jane Nichols
Margaret Mellinger is Physical Sciences Librarian, e-mail: margaret.mellinger@oregonstate.edu,
and Jane Nichols is Humanities and Social Sciences Librarian, e-mail:
jane.nichols@oregonstate.edu, at Oregon State University, Corvallis Oregon.
Introduction
In 2005, OSU librarians considered creating online academic subject area portals, referred to as
“Virtual College Libraries.” At the time, Virtual College Libraries were loosely defined as
disciplinary information gateways tailored specifically for undergraduate use. Before building
the Virtual College Libraries, a task force of librarians wanted to investigate the concept with
undergraduate students through an online survey, focus groups and a usability study. As OSU
librarians explored the complex issues of subject searching they encountered a long-standing
disconnect between how libraries present academic subject related information and how students
find it.
Many questions were generated throughout the needs assessment. For example, do users
want simpler access to "best sources" for subject research? Should libraries filter information by
discipline, subject area, or course? Would undergraduates use library portals that were tailored
especially for them? How is subject searching defined and how should libraries address the
subject search disconnect?
1
We define subject searching from the user perspective, that is, how users search for
subject content or topical information, rather than how library tools organize information.
Obviously, librarians have long enabled subject access to information. From the time libraries
began arranging materials by subject in addition to title and author, new tools and techniques
have been created to facilitate subject searching. A wide range of options have been employed:
from Library of Congress Subject Headings and other controlled vocabularies; to pathfinders or
other subject guides; to browsing collections arranged alphabetically or by discipline; to thesauri,
taxonomies and hierarchical classifications; to online subject portals; and, most recently, to
tagging, and graphical displays. This is an impressive array of accomplishments, but for at least
the past twenty-five years, authors have noted the increasing failure of library subject
arrangements to meet the needs of real users.
Librarians see that students are frustrated with libraries and have difficulty using them to
access topical information. Students prefer commercial search tools to our catalogs, subject
guides and library web pages. Catalogs are bypassed by “a large and growing number of students
and scholars...in favor of other discovery tools.”1 Subject guides are largely unknown to students,
even though they are now readily available on most academic library web sites.2 Moreover, many
library web pages offer a fragmented and confusing array of starting points for discovery. This
model does not match users’ expectations for "one system or search to cover a wide information
universe."3 It is no wonder that a majority of undergraduates forego library search tools and start
their research with a commercial search engine as has been reported by a number of studies.4
Internet search engines shape users’ understanding, preferences and behavior; “the
popularity of the Web appears to have influenced users’ mental models and thus their
expectations and behavior when using a Web-based OPAC interface.”5This observation is true
2
for other online library search tools, as well. Rather than expecting users to conform to library
models and methods, librarians must study users and track their preferences for subject searching,
and for searching in general.
Methodology
OSU librarians administered an online survey, focus groups and a usability study to gauge OSU
students’ searching preferences and skills. The survey of thirty-three questions assessed users’
familiarity with computer technology and their preferences for web page customization. The
eighty-eight survey participants were evenly distributed among classes and the largest percentage
of them represented the three largest colleges. Four focus groups, with twenty-four total students,
tested the perception about the disconnection between the Libraries’ information presentation
and users’ information seeking preferences. Ten people participated in a usability study. The
usability study consisted of three questions and demonstrated users’ success when searching
OSU Libraries’ web pages to find topical information and known items. The needs assessment
results influenced our thinking about students and their search behavior.
Undergraduates' Preferences for Subject Searching
Because of their extensive experience using the Internet, undergraduates are predisposed to the
way Web search engines work. When undergraduates use library search tools, they expect to find
a Google-like interface. The needs assessment helped us understand undergraduate preferences
for starting points, search features and results lists and how those differ from current library
practices.
3
Starting points
Users prefer a clear starting point; usually this means one search box that covers a wide universe
of content. One survey respondent recommended that the library, "Make it more transparent
when searching. I don't care what external search engine it's coming from. Just let me type it (a
search query) into a search box." Even though OSU Libraries' web site included a single search
box for a metasearch tool that searched several multidisciplinary databases, the needs assessment
showed that not all users found or used it. Some found and used it but expressed frustration with
the results and the search speed. The results were too specific and the search was too slow. The
metasearch tool did not meet users' expectations for a starting point or a single search box.
A long list of library search tools stymies users. At the time of the needs assessment, the
OSU Libraries' web site presented an "array of 'silo' systems" including the Libraries' catalog,
databases and e-journals.6 Students needed up-front knowledge to choose from the libraries' list
of tools. Many students in the focus groups and the survey expressed that they wanted the
Libraries’ web pages to be easier to use. One comment, "sometimes I just get lost" neatly sums
up their experiences. OSU is not alone; a recent MIT Libraries study found that "when
confronted by the vast array of tools available to them, the students often exhibited uncertainty
about what to do next."7
Undergraduate students prefer a "task-based information access system" which is "more
useful compared to traditional approaches to information organisation and access."8 Students
experience confusion with library web sites that list search tools, because students do not
recognize that these tools will help them accomplish their tasks.9 MIT librarians noted that
undergraduates' "tasks were overwhelmingly applied towards course-related goals."10 They think
4
books and articles – we give them catalogs and databases. They want content and we give them
tools.
Familiarity is a driver for students when choosing search tools.11 Students in the focus
groups reported that they tend to start their searches for topical information on the web. Using
familiar search engines, students gain an overview of their subject and begin the process of
placing their knowledge in context. If a student learns to use a particular online library search
tool, such as Lexis-Nexis or J-STOR, and has success with it, the student has a tendency to use
that tool as a starting place, even though it may not be appropriate for new information needs.
Search features
Undergraduates prefer to employ short natural language queries, without carefully constructing
searches using Boolean operators or controlled vocabulary. In OSU's usability study, all
participants used natural language queries. Only one used a Boolean operator and none used
controlled vocabulary. The test question may have encouraged this response because they were
asked to search on the topic "endangered species," a common phrase. However, research on
online catalogs has demonstrated that "users build queries using only terms they have in mind
and do not consult a thesaurus before a search is conducted."12 Jansen and Pooch's meta-analysis
found little use of Boolean operators, and Amanda Spink notes that overall use of Boolean search
is limited.13
The needs assessment revealed that undergraduates sometimes experienced problems
finding the right terms to use, and that they would welcome keyword suggestions and listings of
popular searches. Large and Beheshti state that users encounter difficulties when "choosing
suitable search terms to represent their subject interests."14 Users tend to re-try their original
5
search terms in successive searches rather than reformulate their queries.15 Spell check, "did you
mean" and search term suggestions are highly useful for steering queries in the right direction.
Students in the focus groups indicated that this type of immediate and actionable feedback was
valuable. However, Spink reports that "more like this" relevance feedback was used only once in
twenty queries.16
Results lists
Users prefer acceptable matches to appear on the first page of results. Relevance-ranked results
lists support this expectation. Only the top ten or so hits are viewed, and users seldom scroll to
the second page of results.17 They prefer results with some indication that their search terms are
present, such as the highlighting of search terms. This gives them immediate feedback of the
success of their search terms. They also want to see brief summaries describing their hits. In
opposition to these preferences, libraries have traditionally offered results in publication date or
in alphabetical order. Because users view only one or two pages of results, a "combination of
relevance and date ranking" should be considered.18 In library catalog and database results lists, it
is not often clear to the user where their search terms appear, and summaries are not always
available. Recently, some databases and catalogs have added faceted browsing, so that users can
choose alternate ways to refine their search and see how their search terms are related to other
possible terms. Also, "Library OPACs are looking to FRBR (Functional Requirements for
Bibliographic Records) as an organizational concept that will help their catalogs intuitively
present content delivered in multiple forms."19
Users want "good enough" sources, not necessarily the best sources available. The
evaluation of "good enough" is based on a number of factors, the most important being ease of
6
access and availability of full-text. The sources people choose to use follow the Principle of
Least Effort; “people tend to choose perceived ease of access over quality of content in selecting
an information source or channel...”20 Libraries have traditionally touted the high quality of their
collections versus the open web, but quality alone will not bring users to libraries. Users will
choose high quality library resources if they are easy to access in full-text online. Students will
go beyond the principle of least effort if they perceive some value in doing so. For example,
focus group comments showed that students were keen to know what their professors might
recommend as worthwhile sources for their research. This finding may mean that the
recommender systems currently used by Amazon and other websites would be favorably viewed
in library sites.
We concluded that undergraduates’ expectations were not being met by OSU Libraries’
web pages, catalogs and search tools. Users do want easier access to sources for subject research.
Would personalization and customization tools facilitate easier access and meet users'
expectations? Would students gravitate to "virtual college libraries?"
Undergraduates' knowledge and use of customization tools
Personalization and customization tools seemed to promise a constructive approach to the
"subject search disconnect." We see personalization tools as the ability to tailor library resources
according to a personal profile, and customization as the users' ability to change the look and feel
of a web page. Coupled with the view that personalization and customization could help solve
the subject search disconnect was the general assumption that undergraduate students are
technologically savvy. Current undergraduates are the Net generation, those "born with the
7
chip."21 Therefore, it seemed that they would be the optimal audience for personalized online
library services.
There have been a number of library and institutional portal implementations. However,
some of these implementations, such as MyLibrary, initially had limited success.22 Other portal
projects such as MyWelch seemed to succeed because of a very well-defined community of
users.23 This prompted us to question the cost-effectiveness of developing such interfaces for
large groups of undergraduates. Why weren't these systems used as much as expected? To what
extent would undergraduates use a subject portal? Although undergraduates are known to be
technologically savvy, OSU librarians encounter many undergraduate students who are not as
technologically adept as anticipated. We questioned the extent of OSU students' technological
skills and wondered what applications they commonly used.
Given these assumptions and questions, the task force surveyed OSU undergraduates to
assess their familiarity with computer software and interaction with web pages, including the
OSU Libraries' web site. Survey participants were asked about their programming abilities; their
use of Web 2.0 tools such as blogs, wikis, and RSS feeds; and their preferences for
customization features on web sites such as MSN and Yahoo!, and in course management
software packages such as Blackboard.
The general findings of the survey corroborated previous findings and assumptions and
also delivered some surprises. It was not surprising that ninety percent of the students used
Google as their primary search engine; that all of the respondents used word processing software;
and that over three-quarters used presentation software. Students in our study heavily used
communication tools such as e-mail, instant messaging and text messaging. Their technological
skills and tools were deeply focused on communicating and the number of tools they used to do
8
so. “Undergraduates are communicators” states the recent ECAR (Educause Center for Applied
Research) Study of Undergraduate Students and Information Technology.24
We were surprised to find that over three-quarters of the participants in the OSU
assessment did not use blogs or RSS feeds. Also, a majority of students indicated that they were
not familiar with the term “Wiki.” Apparently, the extent of students’ technological skills was
less broad than we had expected, but we also see these results as a limitation of our study. User
studies are a moving target. These results may reflect the timing of the needs assessment which
took place in early 2005, before the upsurge of MySpace and Wikipedia.
As for whether students would choose to customize the OSU Libraries web site if given
the chance, an underwhelming number of the survey respondents (twelve) replied that they
would. Three students answered an open-ended version of the survey question by saying they
thought it would be useful to choose what content appeared on their library pages, and two
students thought it would be "cool" to change the colors. Focus group responses were also
lukewarm to the idea of customization. These responses, in addition to the other information we
had gathered led to our conclusions about the suitability of the Virtual College Library concept.
Suitability of Virtual College Library Concept
Our research concluded several things: a majority of students use non-library online resources,
they find library tools difficult to use, and they are not especially keen to customize the
Libraries’ web site. From the focus groups, we learned that undergraduates are very assignmentdriven, focused on particular courses, and do not necessarily identify with disciplines or colleges.
Students want to save time by quickly gathering the information they need for their assignments.
9
The task force concluded against pursuing the Virtual College Library concept as it did
not seem to match undergraduate students' preferences. Other determining factors in the decision
were the students' tepid response to customization and the high cost of purchasing, programming
and maintaining portal software. As OSU Libraries’ technological infrastructure and as new web
applications mature, we may revisit this decision, especially if we identify a suitable user
community.
As an alternative to Virtual College Libraries, the task force chose to focus on creating
course assignment web pages in collaboration with faculty. Course level pages more directly
mirror students’ assignment-driven thinking. Other academic libraries have come to the same
conclusion. Both University of Rochester and University of Minnesota Libraries have built tools
to streamline the construction of library web pages for particular courses. The latest generation
of MyLibrary offers "course-based access to library resources."25
Course assignment web pages are not the sole solution to the subject search disconnect.
There is a broader disjuncture between students and libraries that must be addressed. Libraries
will need to take a multi-faceted approach to crack this problem.
Recommendations
Users’ and librarians’ expectations for locating topical information have totally shifted.
Alternatives to library tools are so effective that users can get what they need from them without
learning to use the library. Moreover, librarians are no longer satisfied in serving only those who
are willing to learn how to use our systems. Libraries now have a greater opportunity to make
electronic journals, databases, datasets and other information more visible and easier to discover
and access by a broader group of users, particularly those who would not have used the physical
10
library. There are a number of approaches recommended by experts to bridge the gap between
libraries and potential users, and to meet the expectations of both users and librarians.
Libraries can take several strategies to position themselves for the future. Each library will need
to prioritize which to adopt based on their unique environment.
Enhance library search tools functionality.26 Clearly, libraries must match users'
expectations of searching for, identifying, and accessing relevant resources. Innovations are
occurring in several areas. Some libraries have focused on the catalog, as Endeca at NCSU,27
PINES in Georgia,28 or Lamson Library's WPopac.29 Others have centered their efforts on
creating a single search tool: Oregon State University's LibraryFind is one example.30 The
University of Rochester Libraries has concentrated on making articles easier to find and access.31
Many libraries are making their library web sites more user friendly, the University of
Minnesota's Virtual Library focuses on undergraduates.32
Make the library discoverable via the web. Ensuring that your web page is in the top
results of commercial search engines is one place to start. Abram recommends guiding users
from web results back to your library’s content via open URL resolvers and OCLC’s library
search.33 University of Pretoria Libraries describes their process of linking their openURL
resolver, SFX, with GoogleScholar to simplify access to journal articles.34 Open WorldCat is
making library collections “more accessible from the sites where many people start their search
for information.”35
Enhance users’ experiences of library web sites. There are many features that libraries
can employ to enrich users’ experiences of library web pages. Options include the user of RSS
feeds to highlight new books, and lists that display most popular searchers or top-requested
books. Libraries should find points at which users experience confusion and provide context-
11
sensitive help. Start creating online virtual library communities to mirror the social space of the
library’s physical building.36 Think about building a 3-D information landscape driven by the
user.
Determine niche access to library resources. As libraries web services mature, new
ways of manipulating content will emerge. NCSU Libraries37 and University of Minnesota
Libraries 38 offer examples to emulate. Create subject groupings of resources to be searched by a
federated search tool.
Market library services to users. Do not assume that our students know about us. They
want us to market to them and they are accustomed to marketing directed at them. One survey
comment encouraged: “More publicity! Let students know how to access the benefits of the
library from remote locations.” Another student said she would have used the library web page
had she known about it earlier in her career.
Continue to partner with faculty to integrate information literacy into the curriculum.
Faculty can highlight library content by requiring it. Students respond to faculty driven/course
driven pushes.
Partner with others working on these issues. Greater advances can be made through
partnership. Some partnerships are already formed, such as RLG Programs Partners39 and the
Digital Library Federation.40Others could form out of smaller conferences. Access41 and
code4lib42 are two examples. We can also leverage the work that is happening on an ad hoc basis
across many libraries. If libraries could organize efficiently, and decide to work in consort, much
could be accomplished.
Culture shift
12
The culture of libraries is changing; but perhaps librarians can be more conscious of their own
mindsets in order to move libraries and the field of librarianship forward faster. Changing the
culture of libraries and the point of view of librarians is just as important as implementing the
previous recommendations. Here are suggestions for creating a culture of change that will
support libraries as we move forward.
Create a culture of innovation. Innovating isn't just for the big players in the library
field. Open source and social software invite us all to explore and participate. Libraries and
librarians can and need to develop technical skills. In ten years, what aspects and how much of
your library will be physical?
Create more intuitive search interfaces. Continue creating our own tools such as
PINES, WPOpac and PennTags.43 Find ways to contextualize result lists with graphical
interfaces or other means of placing results in broader and narrower context. Grokker44 and
AquaBrowser45 offer examples of this multi-faceted browsing.
Create a participatory culture. Undergraduates are accustomed to participatory culture
from their experiences with MySpace, YouTube and multi-modal online games. In comparison,
communicating via email and web forms seems so...one-sided.46 Creating venues for
participation by our users will give us valuable first-hand information.
Create a culture of assessment. Are we meeting users' needs? How do we know? Find
ways to anticipate how users are changing and implement strategies to meet those changes. If
each library worked towards meeting just one anticipated change, imagine what the cumulative
effects will be.
Conclusion
13
The subject search disconnect is not a new phenomenon. But the convergence of users, librarians
and tools will profoundly change our future, promising a new era for libraries. We look forward
to seeing how libraries will integrate themselves into the broader information environment.
14
Notes
1. Karen Calhoun, The Changing Nature of the Catalog and its Integration with Other Discovery Tools,
(Final.Report, March 17, 2006): 5, http://www.loc.gov/catdir/calhoun-report-final.pdf.
2. Brenda Reeb and Susan Gibbons, “Students Librarians and Subject Guides, Improving a Poor Rate of
Return,” portal: Libraries and the Academy 4 no.1 (2004): 128,
http://muse.jhu.edu/journals/portal_libraries_and_the_academy/v004/4.1reeb.html.
3. University of California Libraries, Rethinking How We Provide Bibliographic Services for the University
of California Libraries, (Final report of the Bibliographic Services Task Force, December, 2005): 8,
http://libraries.universityofcalifornia.edu/sopag/BSTF/Final.pdf.
4. OCLC White Paper on the Information Habits of College Students: How Academic Librarians Can
Influence Students' Web-Based Information Choices, (2002): 3; Gail Salaway et al., The ECAR Study of
Undergraduate Students and Information Technology,(December 2006): 25,
http://www.educause.edu/ir/library/pdf/ers0607/ERS0607w.pdf ; Sarah Lippincott and Martha Kyrillidou, “How
ARL University Communities Access Information: Highlights from LibQUAL+,”ARL ARL Bimonthly Report, 236
(October, 2004). http://www.arl.org/newsltr/236/lqaccess.html.
5. Holly Yu and Margo Young, "The Impact of Web Search Engines on Subject Searching in OPAC,”
Information Technology and Libraries 23, no. 4 (2003): 168.
6. University of California, 2.
7. Nichole Hennig, Tracy Gabridge et.al, "User Needs Assessment of Information Seeking Activities of MIT
Students - Spring 2006," 19, http://hdl.handle.net/1721.1/33456.
15
8. Narayanan Meyyappan, Schubert Foo and G.G. Chowdhury, "Design and Evaluation of a Task-Based
Digital Library for the Academic Community," Journal of Documentation 60, no. 1 (2004): 472.
9. Barbara J. Cockrell and Elaine Anderson Jayne, “How Do I Find an Article? Insights from a Web
Usability Study,” Journal of Academic Librarianship 28, no.3 (2002): 122-132.
10. Hennig, Gabridge et al., 9.
11. Barbara Valentine, “Undergraduate Research Behavior: Using Focus Groups to Generate
Theory,” Journal of Academic Librarianship 19 (Nov. 1993): 304.; Anna M. Van Scoyoc and Caroline Cason, “The
Electronic Academic Library: Undergraduate Research Behavior in a Library Without Books,” portal: Libraries and
the Academy 6, no. 1 (2006): 47-58,
http://muse.jhu.edu/journals/portal_libraries_and_the_academy/v006/6.1van_scoyoc.html.
12. Yu and Young, 175.
13. Bernard Jansen and Udo Pooch, “A Review of Web Searching Studies and a Framework for Future
Research,” Journal of the American Society for Information Science and Technology 52, no.3 (2001): 241; Amanda
Spink, “Web Search: Emerging Patterns,” Library Trends 52, no.2 (Fall 2003): 301.
14. Andrew Large and Jamshid Beheshti, “OPACs: A Research Review,” Library & Information Science
Research 19, no2. (1997): 125.
15. Spink, 302; Jansen and Pooch, 2001.
16. Spink., 302.
17. Ibid., 303.
16
18. Yu and Young, 173.
19. Marshall Breeding, “Technology for the Next Generation,” Information Today (November/December
2005): 9.
20. Thomas Mann, Library Research Models, (New York. Oxford University Press, 1993): 93.
21. Stephen Abram and Judy Luther, “Born with the Chip,” Library Journal 129, no.8 (2004): 34-37.
22. Susan Gibbons, “Building Upon the MyLibrary Concept to Better Meet the Information Needs of
College Students,” D-Lib Magazine 3 (2003), http://www.dlib.org/dlib/march03/gibbons/03gibbons.html.
23. Dongming Zhang, et al., “User information seeking behavior in a medical web portal environment: a
preliminary study,” Journal of the American Society for Information Science and Technology 55, no. 8 (2004): 670684.
24. Gail Salaway, et al., 11.
25. Karen Ciccone, “MyLibrary@NCState: a library portal after five years,” Journal of Library
Administration 43, no.1/2 (2004.): 33.
26. University of California Libraries, 5.
27. North Carolina State University Libraries, Online Catalog, http://www.lib.ncsu.edu/catalog/.
28. Georgia Public Library Service, PINES catalog, http://www.georgialibraries.org/public/pines.html.
17
29. Plymouth State University Library, Online Catalog, http://www.plymouth.edu/library/opac/.
30. Oregon State University Libraries, open source federated search tool, LibraryFind,
http://search2.library.oregonstate.edu/record/search.
31. University of Rochester, River Campus Libraries, “Finding Articles,”http://www.library.rochester.edu/.
32. University of Minnesota Libraries, Undergraduate Virtual Library, http://www.lib.umn.edu/undergrad/.
33. Stephen Abram, “The Google Opportunity,” Library Journal 130, no. 2 (2005): 34-35.
34. Ina Smith and Sunette Steynberg, “Power-searching the ‘3-click’ world: a federated search solution for
clients at the University of Pretoria,” Presented at: SASLI Conference: An integrated approach to e-content, CSIR
International Conference Centre, 26 July 2006, http://hdl.handle.net/2263/569.
35. Online Computer Library Center (OCLC), Open WorldCat description,
http://www.oclc.org/worldcat/open/.
36. This is in reaction to Derek. Powazek, Design for Community: The Art of Connecting Real People in
Virtual Spaces, (Indianapolis, New Riders Publishing, 2002).
37. North Carolina State University Libraries, Browse Subjects pages,
http://www2.lib.ncsu.edu/browsesubjects/?subject=55&subject_group.
38. University of Minnesota Libraries, CourseLib pages, http://courses.lib.umn.edu/.
39. RLG Program Partners, description, http://www.rlg.org/en/page.php?Page_ID=382.
18
40. Digital Library Federation, description http://www.diglib.org/.
41. Access2006 description, http://www.access2006.uottawa.ca/.
42. code4lib, description, http://www.code4lib.org/.
43. PennTags description, http://tags.library.upenn.edu/.
44. Grokker company home page, http://www.grokker.com/.
45. AquaBrowser company home page with links to demos, http://www.medialab.nl/.
46. Breeding, 9.
19