CONFIDENTIAL TO UC LIBRARIANS EBSCO-Gale Database Evaluation Task Force Report to the Collection Development Committee University of California Libraries December 4, 2007 Submitted by Task Force members: Myra Appel, UC Davis, Chair Vicki Bloom, UC Riverside Harold Colson, UC San Diego Frank Gravier, UC Santa Cruz Cynthia Johnson, UC Irvine Wendy Parfrey, ex officio (CDL) Gail Yokote, ex officio (JSC) (Amended December 19, 2007) 1 Table of Contents I. Executive Summary ………………………………………… Pages 3-4 II. Introduction and charge ………………………………………… Pages 5-6 III. Methodologies ………………………………………… Pages 7-9 IV. Analysis ………………………………………… Pages 10-19 a. Charges 1 & 2: Unique coverage and overlap; Completeness of content b. Charge 3: Reliability and stability of content c. Charge 4: Functionality d. Charge 5: Barriers to use e. Charge 6: Cost-benefit analysis V. Conclusions and recommendation …………………………………… Pages 20-21 Appendix A: Charge to the EBSCO-Gale Database Evaluation Task Force .....………………………………. Pages 22-24 Appendix B: Comparison of Sample Search Topics (See Separate Attachment) Appendix C: EBSCO-Gale Functionality Matrix (See Separate Attachment) Appendix D: Summary of Survey Results ……………………………….. Pages 27-41 2 I. Executive Summary The EBSCO-Gale Database Evaluation Task Force was charged by the CDC to assess four multidisciplinary databases and recommend the product that best meets the needs now and in the near future of UC students and faculty seeking multidisciplinary journal articles and other content. After comparing Expanded Academic ASAP (Gale), Academic OneFile (Gale), Academic Search Premier (EBSCO) and Academic Search Complete (EBSCO), the Task Force submits the following observations and recommendations: I. Without licensing a multidisciplinary database, the UC would lose access to a significant percentage of content not available through other licensed resources. II. According to Gale representatives, the vendor does not plan to develop Expanded Academic ASAP significantly in the future; ASAP can be considered essentially complete with plans to add few titles. Vendor development will be focused on Academic OneFile. III. According to EBSCO representatives, the vendor does not plan to develop Academic Search Premier (ASP) significantly in the future; ASP can be considered essentially complete with plans to add few titles. Vendor development will be focused on Academic Search Complete (ASC). IV. ASC and OneFile contain some full text content that overlaps with content offered through existing UC-licensed resources. Typically, the overlap is less than 10%. V. Based on comparing coverage between Academic OneFile (Gale) and Academic Search Complete (EBSCO), OneFile retrieves the most hits and has the highest percentage of full-text content. The advantage typically comes from the “News” content coverage offered by OneFile. VI. Academic Search Complete at this time does not offer appreciably more content, including academic peer-reviewed content, than does the less expensive Academic Search Premier product. VII. Both Gale and EBSCO offer comparable levels of persistence and reliability for their products. VIII. Functionality and search, access, display and content manipulation options are robust on each vendor interface, with each vendor interface providing users with some unique strengths and features. IX. No barriers to use not already in place or unacceptable to the CDL exist for any of these products. 3 X. Respondents to UC and local campus trials did not clearly identify that the Gale Academic OneFile or the two EBSCO resources were superior to each other; assessments of the content and features ended up in a rough tie. XI. A cost-benefit analysis of the four products shows overall that the EBSCO Academic Search Complete best fits with the UC Tier 1 economic model funded by the CDL; the Academic Search Complete meets both current and near-future user needs for multidisciplinary full text content in the arts, social sciences, humanities and sciences; and that Academic Search Complete meets an acceptable standard for ease of use by both novice and experienced searchers. 4 II. Introduction and Charge For over a decade the University of California libraries have licensed Gale’s Expanded Academic ASAP database to meet the multidisciplinary information needs of our undergraduate students and other users seeking journal and magazine content. Since the UC libraries initially licensed Expanded Academic ASAP, the descendent of the MAGS database once accessible through MELVYL, the information industry has undergone considerable change and reorganization. Vendors and libraries have introduced new products and adopted numerous technological developments and enhancements that facilitate patron access to information on site within the library and remotely off site using personal computers. With substantial experience gained using a variety of electronic resources, including the Internet, Google and both commercial and freely accessible web-based databases, librarian and user expectations for the access, delivery and manipulation of periodical content have evolved to increasingly discriminating levels. Concurrently, the interdisciplinary nature of research and instruction continues to flourish throughout the academic community and impact on our decision making as we develop library collections and design library instructional services. With the University of California’s contract for Expanded Academic ASAP coming up for renewal in 2008, the UC Joint Steering Committee (JSC) recognized that now is the opportune time to evaluate if Expanded Academic ASAP continues to address the needs and expectations of students and faculty seeking multidisciplinary content, particularly full-text content, or if one of the newer competitors in the marketplace better serves our patrons. In April 2007 the JSC formed the EBSCO-Gale Database Evaluation Task Force to compare Expanded Academic ASAP (ASAP) with three other major multidisciplinary databases – Academic OneFile (OneFile), a Gale resource larger and more comprehensive than their ASAP product; Academic Search Premier (ASP), an EBSCO resource; and Academic Search Complete (ASC), EBSCO’s larger and more costly multidisciplinary database. The Task Force members, Vicki Bloom, (UCR, Chair of the General Reference Bibliographers Group), Harold Colson (UCSD), Frank Gravier (UCSC, Resource Liaison for Expanded Academic ASAP), Cynthia Johnson (UCI) and Myra Appel (UCD, Task Force Chair), represent diverse public services units that include general reference departments and more specialized subject departments, each serving the multidisciplinary information needs of undergraduates and other users researching topics that cross disciplinary lines. Wendy Parfrey (CDL) and Gail Yokote (JSC) serve as ex officio members. The UC Collection Development Committee (CDC) requested that the Task Force identify from the two EBSCO and the two Gale products the database that: Best meets the needs of undergraduate students. 5 Best meets the needs of any user interested in multidisciplinary coverage of subjects. Best meets the criteria of ease of use. Specifically, the Task Force was charged to: 1) Identify unique content and coverage; to assess the full-text content within each database and examine overlap in coverage with other resources currently licensed by the UC. 2) Assess completeness of content (i.e., abstracts/full-text). 3) Assess the reliability of and stability of content. 4) Identify any functionality that affects the usability of or access to the content (i.e., search, retrieval, display, manipulation). 5) Highlight barriers to use (i.e., licensing features, such as perpetual access, ILL, or instructional/reserve use). 6) Analyze the cost-benefits for each product, considering the impact of campus coinvestments if the Task Force made the recommendation to discontinue Expanded Academic ASAP. Finally, the Task Force was asked to recommend the “most appropriate general, multidisciplinary database which meets the current and near-term future needs of students.” (See Appendix A. Charge to EBSCO-Gale Database Evaluation Task Force, for a full discussion of Task Force charges.) 6 III. Methodologies Evaluating the “strengths and limitations” of the various EBSCO and Gale databases to help determine the one that “best meets” the needs of undergraduate and multidisciplinary searchers is a complex task that involved numerous dimensions of assessment. Given that three of our assigned evaluation components (1, 2, 3) deal specifically with product content, and that another (6) properly subsumes content considerations, the Task Force also devoted substantial effort to reviewing and analyzing database content via assorted quantitative and qualitative lenses, using vendor-supplied materials as well as independent investigations. Cognizant throughout the entire process that a general multidisciplinary database licensed by the UC as a Tier 1 resource must meet the needs of undergraduates and other user groups in all disciplinary areas of the social sciences, arts, humanities and the life, physical and health sciences on all ten UC campuses, the Task Force from the onset of the study realized that the broadest possible range of input was essential to the assessment. To collect data and develop an in-depth familiarity with the strengths and limitations of ASAP, OneFile, ASP and ASC, Task Force members identified 29 representative multidisciplinary search topics from recent reference desk transactions with undergraduate students. The searches included subjects and search terms that spanned the social sciences, arts, humanities and the life, physical and medical sciences disciplines. Several of the topics crossed disciplinary lines between the sciences and social sciences. We searched these topics across each of the four databases, using to the extent possible the same features and functionalities, as well as trying out options unique to the resource. Limits were imposed to determine the percentage of full-text content. Task Force members recorded the number of academic and peer-reviewed articles retrieved, as well as the number of news source items, popular magazine articles and other items such as images, book reviews and government documents. To explore more fully the content available from popular and hobbyist magazines, we also searched MasterFile Premier (EBSCO), using the same 29 search terms and limit options that we had used in testing ASAP, OneFile, ASP and ASC. Search results are discussed in Section IV of the report. (See Appendix B. Comparison of Sample Search Topics, for a spreadsheet listing all 29 searches and the number of hits categorized by format and date criteria.) Both vendors use the same interface with each of their family of products. EBSCO’s Academic Search Premier, Academic Search Complete and MasterFILE share the same interface, with only a few minor variations among the three databases. Gale’s OneFile builds on the Expanded Academic ASAP interface. Using several sample searches across the five resources, Task Force members observed and tracked the search and navigation features offered with the EBSCO and the Gale interfaces and compared the functionality and ease of use between the two vendors’ interface designs. (See 7 Appendix C. EBSCO – Gale Functionality Matrix, for a detailed comparison of the two interfaces.) To supplement the information and data gathered from the comparative work performed by the Task Force, we invited UC librarians and, to a lesser extent, users to participate in the testing process. The Task Force adopted the following approaches to encourage as many individuals as possible to participate: Offered trial access to all UC librarians: Task Force members felt that an assessment of this scope and magnitude needed the subject expertise and input from bibliographers representing all UC campus libraries. Both EBSCO and Gale agreed to provide free access to UC librarians for the duration of the study. Links, with password protected access, were set up on CDL web pages. A survey form, developed by the Task Force members, was linked to the trial pages. Although access to both ASP and ASC, the two EBSCO products, and the Gale OneFile product continued into November, the survey closed down in mid-September with the expiration of the CDL contract with Websurveyor. The survey was available to accept submissions for 88 days, from June 15, 2007 - September 10, 2007. (See Appendix D. Summary of Survey Results, for a copy of the survey questions and survey responses.) To publicize the trials, disseminate information about the charge to the Task Force and encourage as many librarians as possible to participate in testing, an informational letter from the Task Force describing the project was sent to the chairs of the UC Bibliographer Groups and to all UC Resource Liaisons. Recipients of the letter were asked to distribute the letter broadly to their bibliographer group members and to colleagues on their respective campuses. The Task Force drafted a separate letter for the CDC, requesting that CDC members also encourage their local librarians to take part in the trials. Task Force members also publicized the trials on our respective campuses and encouraged our colleagues to provide their feedback. Offered trial access to local campus communities: EBSCO and Gale willingly agreed to provide free trial access for all faculty and students on each UC campus. UCD and UCR opted to make the three unlicensed databases (OneFile, ASP and ACS) available to their users by setting up links on campus library home pages; each campus created survey forms to encourage feedback. UCSD made available to local users the two EBSCO products, ASP and ASC. The Davis trial extended from April 29, 2007 – September 20, 2007. The Riverside trials ran from August 2007 - October 2007. (See Appendix D. Summary of Survey Results, for a copy of the survey questions and survey responses.) Conducted usability tests and focus groups: Task Force members at UCI and UCR conducted usability tests with representative user groups. At UCI librarians voluntarily met in two groups to participate in comparing the retrieval and performance of the four databases under consideration. Each reference 8 librarian/subject specialist librarian constructed searches relevant to his/her academic discipline assignment, submitting their input via the CDL-hosted survey. The UCR Task Force member met with a focus group of five students to obtain feedback using a scripted set of questions. Conducted vendor interviews: Task Force members attending the ALA 2007 Annual conference held in Washington, D.C. met with EBSCO and Gale company officers and customer service representatives to discuss the current state of each of the four databases, each company’s future plans for developing their respective products, and potential changes the companies anticipate in their business models. At the request of the Task Force, EBSCO and Gale provided informational, rather than promotional, demonstrations of OneFile, ASP and ASC. Conversations with both vendors have continued intermittently over the past months as the Task Force developed additional questions and the vendors updated the Task Force about impending changes in their products or business plans. These ongoing conversations have proven invaluable to the work of the Task Force. At the ALA meeting, the Task Force learned that Gale plans to dedicate the majority of their efforts to growing Academic OneFile, with a diminishing emphasis on Expanded Academic ASAP. EBSCO also is redirecting their focus from developing Academic Search Premier to accelerate the expansion of Academic Search Complete. Conducted librarian interviews: During the assessment, the Task Force learned that other libraries had undertaken and completed similar studies to compare the effectiveness and success of the Gale products with the EBSCO resources in meeting the multidisciplinary information needs of diverse academic communities. Conversations with two California State University librarians, subsequent to the announcement that many CSU campuses had opted to participate in a California State University consortial licensing agreement with EBSCO, reaffirmed the conclusions of the Task Force that the EBSCO products met our users’ current and near-future multidisciplinary information needs. Likewise, a conversation with a librarian charged to head an assessment team comparing the two EBSCO databases, ASP and ASC, and determine the best resource to license for use by a large, renowned research institution confirmed the findings of the Task Force that Academic Search Complete did not offer appreciably more academic content than did Academic Search Premier. Although these three librarians generously shared verbally their insights and observations, issues of confidentiality governed their willingness to make their formal assessments public outside their institutions. 9 IV. Analysis Charge #1: Assess Unique Coverage and Overlap Charge#2: Assess Completeness of Content When viewed against the formidable installed base of content available to UC users from scores of other article databases and e-journal packages, parsing notable pockets of unique content and coverage from four general databases featuring several thousand journals, magazines, newspapers, and related publications is a daunting task. If our legacy Expanded Academic ASAP subscription was dropped without replacement, then there would certainly be some loss of exclusive content (690 of the database’s 2224 SCP-cataloged titles have no additional SCP links to full text), but any substitution by the Gale or EBSCO successors now under consideration would render unique content losses minor or even null. Some 250 of the current ASAP exclusives would be provided by ASC and MasterFILE Premier, and both potential successors would bring countervailing fresh exclusives not presently within the UC stable of full-text periodicals; licensing OneFile as the replacement for ASAP would assure complete coverage of the 690 exclusives. The Gale and EBSCO contenders are perhaps best viewed not as primary contributors of unique or exclusive content but as convenient multipurpose gateways serving broad rather than specialty audiences. The candidate databases do carry full-text content that overlaps (usually in partial dates) with offerings from existing system-wide licensed resources, including LexisNexis Academic, Business Source Premier, JSTOR, Project MUSE, Literature Online, Periodicals Index Online, Education Full Text, some SAGE subject collections, and numerous publisher e-journal packages. In most instances, the overlaps involve small portions (less than 10%) of our large candidate databases and somewhat larger portions of smaller external sources. In the absence of an actual database overlap tool to aid our analysis, it appears from searches of Melvyl and campus catalogs that the highest fulltext replication rates (partial date overlap) with the 222 Expanded Academic ASAP titles with SCP cataloging hit Project MUSE (125 of 348 titles), JSTOR (201 of 1048), and Literature Online (99 of 202) journals. The typical patron encounter with a database is not via its title list, but with its search interface and results screen, and thus the Task Force performed more than two dozen sample searches drawn from recent campus experiences to help gauge the “real world” retrieval performance of each candidate product, plus the related MasterFILE Premier database. These searches, which comprise single-term, phrase, and Boolean entries, were conducted via the default (keyword) search box; tallies were then made of total hits, total full text hits, hits by vendor publication type (academic, magazine, news, other), and hits (and full text hits) from 2000 and 2006 onwards. The raw results from these test searches are presented as Appendix B. Compiled totals and category percentages are shown below in Table 1. 10 Our principal findings from these retrieval snapshots are: 1) Gale Academic OneFile, as perhaps befits a product covering more than 10,600 publications and boasting in excess of 24 million articles, is the overall retrieval champion, outpacing the legacy Expanded Academic ASAP and the EBSCO competition with the most hits in nearly all search category instances. Much of its retrieval lead in total hits and total full text hits comes from a large advantage in “News” category results, drawn considerably from The New York Times, The Financial Times, and The Times (London), all in full text. 2) Although EBSCO Academic Search Complete carries 1000 or so more titles than its Academic Search Premier sibling, its more robust title list does not translate much into actual search retrieval totals. The hits differences between the two EBSCO databases are rather slight, with the advantage for Complete consistently running around 5% or less across all spreadsheet categories. The Premier and Complete “News” tallies are practically identical, drawn as they are from the same four newspapers (The New York Times and The Wall Street Journal as abstracts, Christian Science Monitor and USA Today with full text). 3) The retrievals competition between Expanded Academic ASAP and the Academic Search siblings reveals a rough equivalence between the vendor camps. Some searches yield more total results in the Gale product and some searches perform better via the EBSCO pair, with the overall effect being an approximate draw. The aggregated search figures show considerably stronger “News” category performance from Expanded Academic and higher overall “Academic” and “Magazine” totals from Premier/Complete. Indeed, both Gale products distribute their hits in roughly equal shares across the “Academic,” “Magazine,” and “News” categories; the category distributions from the Academic Search databases are weighted heavily to “Academic” and “Magazine” hits. 4) The percentage of internal full text content (embedded HTML or PDF) in the aggregated retrieval sets is highest in Academic Onefile (almost 70%), with Expanded Academic ASAP and the Academic Search duo delivering just over half of all hits with full text. Of course, full-text availability is further enhanced by UC-eLinks connections to external targets, but the Task Force has not measured or compared these quantities across the candidate databases. 5) The largest clusters of non-print content (see “Other” category) are found in Academic Onefile (links to NPR broadcast audio) and MasterFILE Premier (UPI and Getty images). 11 Table 1 – Aggregate Results From Sample Searches Database Academic OneFile Academic Search Complete Academic Search Premier Expanded Academic ASAP MasterFILE Premier Academic OneFile Academic Search Complete Academic Search Premier Expanded Academic ASAP MasterFILE Premier Total Hits 40992 28830 Full Text Academic Magazine News Other 28623 16303 11278 12136 12384 12405 16195 3673 1139 708 28127 15862 12014 11955 3638 64 27147 14548 9336 9297 8498 16 26586 17491 n/a 21535 4098 1155 40992 28830 % Full Text 69.8% 56.5% % Academic 27.5% 42.1% % Magazine 30.2% 43.0% % News 39.5% 12.7% % Other 2.8% 2.5% 28127 56.4% 42.7% 42.5% 12.9% 0.2% 27147 53.6% 34.4% 34.2% 31.3% 0.1% 26586 65.8% n/a 81.0% 15.4% 4.3% Time did not allow for a comparison between EBSCO’s Newspaper Source and Gale’s news content coverage. However, the description appearing on EBSCO’s website states that 28 national and international newspapers, including The Christian Science Monitor, USA Today, The Washington Post, The Washington Times, The Times (London), and the Toronto Star, are available in cover-to-cover full text. From a recent EBSCO email communication, it appears that The New York Times, now freely available online, also will be conveniently accessible in Newspaper Source. For each major newspaper EBSCO negotiated a contract directly with the publisher. The database contains, in addition, selected full-text articles from 260 regional U.S. newspapers, as well as full-text television and radio news transcripts for ABC, CBS News, CNN, Fox News and NPR programming. EBSCO negotiated a separate contract with The McClatchy Company to gain access to this mix of full-text content. With this additional news coverage, EBSCO resources, particularly ASC, match or parallel OneFile more closely in the scope of coverage and the number of records retrieved, although the boosted “News” retrievals from EBSCO would come from a separate database. 12 Charge #3: Assess reliability and stability of content Completeness: Each vendor provides regularly updated lists detailing the content of each of the databases under review (links provided below). For the purpose of this review lists for July/August 2007 were used to provide a comparative snapshot of the relative scope and size of the databases. Both vendors provide similar information in the lists including ISSN, publication title, publisher, abstracting and indexing coverage dates, full-text coverage dates, embargoes or full-text publication delays, peer-review status, etc. The table below provides comparisons for total number of titles, total full-text titles, total inactive full-text titles and number of peer-reviewed titles for each database. Table 2 -- Title counts (Based on July/Aug. 2007 lists): EBSCO GALE Complete PremierASAP OneFile Total 9,386 8,240 4,808 10,673 Total full txt 5,411 4,511 2,516 4,342 A striking feature of the above table is the relatively small difference between the number of titles in EBSCO’s Academic Search Premier (ASP) and Academic Search Total inactive full txtThe vendor has 878indicated that 836this is a result 732of the fact the 918 Complete (ASC). ASC is a relatively new product, still in active development with titles being continually added. This is borne out by the fact that a more recent title list shows an increase of close to 500 titles in ASC and only 10 new titles to the7,480 vendor, Totaladditional peer reviewed 8,165 7,159in ASP. According 2946 while ASC will continue to add titles and grow, ASP can be considered essentially complete with few titles planned for addition in the future. The high percentage of peer-reviewed titles for each vendor provides some indication of the level of scholarly content available from each of the databases. Both vendors provide information detailing their policies for determining whether or not a title is peer-reviewed. Both vendors identify peer-reviewed titles as those identified as such in a standard library reference source (e.g.,Ulrich’s or Serials Directory), or in the publisher’s marketing or prefatory material concerning the publication. EBSCO additionally uses “[f]eedback from librarians and professionals” and “EBSCO Publishing staff librarians” as a basis for determining whether or not a title is considered to be peer reviewed. Coverage is better for current years. In a sampling of titles from Academic Search Complete and Academic Onefile, indexing earlier than 10/1997 was available for 13 approximately 34% of the titles in the EBSCO product and 40% of the titles in the Gale database, indicating that Gale provides a deeper index. The issue of embargoed titles needs to be considered in any assessment of completeness of content. The number of embargoed titles for each of the databases under review is shown below. Table 3 -- Embargoed titles Complete EBSCO GALE 2,441 Premier2,225 ASAP OneFile 430 2,208 (12 months or more) 2,033 (12 months or more) 332 197 (12 months or more) 239 (12 months or more) As this table demonstrates, EBSCO databases have a significant number of embargoed titles, a substantial portion of which are embargoed for one year or more. EBSCO representatives have asserted, and spot checks of the embargoed titles have confirmed, that the majority of these heavily embargoed titles are from publishers that UC has ejournal package subscriptions (Taylor & Francis, MIT, Blackwell/Synergy, for example) with. As a result, while current content isn’t available in the aggregator’s database it is readily available to our users via UC-eLinks. We feel that it is important to note that, should an EBSCO product be selected, UC librarians will need to emphasize the teaching of UC-eLinks use to our undergraduates, since embargoed titles available on UC-eLinks will be missed by users who limit to full-text articles. Persistence/Reliability: Persistence and reliability can be viewed in at least two different ways, in terms of either database availability or stability of content. Both vendors provide reports detailing additions and deletions for their respective databases. Because both vendors, as aggregators, are bound by the terms publishers are willing to agree to in terms of providing content, it is understood that there may be fluctuation in availability of content. A review of the title change lists indicates that significant numbers of deleted titles are infrequent for both vendors. The addition of titles has been fairly robust for both Academic Search Complete and Gale Academic Onefile, reflecting the publishers’ intent 14 to increase the number of titles for these resources while capping the number for Academic Search Premier and Expanded Academic ASAP in order to target other markets for these databases. Both vendors provide fairly reliable access to their products. EBSCO has stated that their databases have been available over 99% of the time over the last two years. While similar information hasn’t been made available from Gale, the CDL Resource Liaison for Expanded Academic ASAP can report that there have been several outages of Gale databases over the last year, although, to Gale’s credit, notification to customers and restoration of access has been prompt in almost every case. LINKS: For EBSCO’s Academic Search Premier and Academic Search Complete title lists see: http://www.ebscohost.com/titleLists.php?topicID=380&tabForward=titleLists&mar ketID Gale title lists for Expanded Academic ASAP and Academic OneFile can be found at http://gale.cengage.com/title_lists/ EBSCO Publishing’s full peer review policy is available at: http://www.elib.gov.ph/edatabase/elibgetdb.php/http/www.ebscohost.com/thisTop ic.php?topicID=396&marketID=17. The Gale peer review policy can be found at: http://gale.cengage.com/title_lists/peer_review.htm Charge #4: Identify any functionality that affects the usability of or access to the content Overall, the interface and functionality offered by both EBSCO and Gale is good. Users inexperienced with the interface of each product can perform successful searches and retrieve a range of citations relevant to their search query. Nevertheless, as is the case with most commercial resources, library instruction can facilitate and enhance patron use and search techniques for a database. Listed below are key features of functionality frequently cited by librarians and patrons as essential or very important to successful and satisfying search experiences; differences between EBSCO and Gale are noted: Search Options: Search interfaces: Although EBSCO and Gale have similar features, each vendor’s interface has unique aspects as well. Basic keyword searches: Only Gale allows a subject search directly from the Basic search screen. 15 Adjacency features: EBSCO defaults to an Adjacency search when doing a keyword search; Gale interprets the space as a proximity (Near 4 words) search. Default operator to AND: EBSCO allows the user to set the default operator to AND, whereas Gale does not. Full text searches: Both vendors’ interfaces allow full text searching of a document and both provide proximity search options, a boon with full text searching. Gale’s interface makes it easier to identify that full text searching is available. Limits: The types of limits available in each interface are very similar. EBSCO’s are more obvious from the initial screen; Gale provides an option to hide the limits. Overall, EBSCO also has more limiting options than Gale at the search screen: by publication type (newspaper, magazine, etc.), if there are references in the article, number of pages, cover stories. Subject searches: Both EBSCO and Gale interfaces allow subject searching, but Gale has a more sophisticated subject structure with better authority control. Publications searches: Gale and EBSCO allow users to search for specific publications; Gale allows limits on the publication search, a feature which EBSCO does not incorporate. In Gale, a user can search for publications meeting certain criteria, not just known titles (can limit to publication type, format, audience, language, and country of publication). Advanced searches: Both the Gale and EBSCO interfaces allow the patron to search on a specific field. EBSCO has many more fields available for searchers to limit retrieval. Additional search options: EBSCO provides more alternative types of searches, including the Visual Search, which allows patrons to view results using graphics, not pure text. EBSCO also provides the Image Search, which is much more robust than Gale’s ability to limit to articles with images. In EBSCO’s Image Search, the patron actually searches the image captions, providing better results when looking for graphs or images of particular events or topics. Manipulating, revising and viewing results: Display result options: The two vendors’ interfaces are very similar, although EBSCO allows patrons to configure their results (full versus brief display, for example). Gale allows patrons to change their preferences for elements such as number of results per page and font style. Limit by article type: Gale provides the ability to limit by article type (editorial; full-text) at the result screen, which EBSCO does not provide. 16 Limit to material type: EBSCO and Gale interfaces allow the patron to limit to material type (journal versus magazine, for example) from the results display. Limit to full text: Gale can easily limit to full text from the results screen; EBSCO does not provide the option from the results screen. Revising original search: While both the EBSCO and Gale interfaces provide the means of revising the original search, the feature is much easier to find on the EBSCO interface; the feature appears on a visible tab. To revise a search in Gale, the researcher must first click on Previous Searches and then click on Revise. Typographical errors: Both vendors offer a “Did you mean” feature in case of typos. PDF quality: Image quality is difficult to judge, since clarity appears to vary by publication and subject. In comparing art titles, Gale seemed to have a slightly clearer font, and slightly truer colors to the original print title. In contrast, EBSCO offered up medical images with more clarity and detail than did Gale. (See “Clinical, Histologic, and Radiographic Outcomes of Distal Femoral Resurfacing with Hypothermically Stored Osteoarticular Allografts.” By Davidson, Philip A., et al. American Journal of Sports Medicine, July 2007, Vol. 35, Issue 7, p1082-1090 as an example for comparison.) ADA: Both vendors recognize ADA requirements and claim to meet section 508. However, EBSCO’s website promotes a more vigilant stance towards ensuring ADA compliance than Gale’s does on their website. Help Screens: Both vendor interfaces provide help screens, but Gale’s help feature is context-sensitive; additionally Gale offers a dictionary (Merriam-Webster’s) on their site. In sum, each vendor offers some unique, notable strengths in interface functionality. Perhaps the most compelling feature unique to Gale is their more powerful subject heading structure that offers users rational and powerful alternative headings to search to increase their retrieval base. In addition, some of the Gale features (e.g., limiting to full text) are more easily identified on the screen. EBSCO, on the other hand, provides more limiting features and a greater variety of robust search options, including the Visual Search and the Image Search. Charge #5: Highlight barriers to use The UC libraries consortially and individually have enjoyed contractual relationships with both Gale and EBSCO for the past several years. Products offered by both vendors are staples in the mix of Tier I and Tier 2 electronic resources available to UC students and faculty. Both vendors have assured the Task Force that the licensing features they currently offer UC in regard to perpetual access, interlibrary loan sharing of resources, 17 and instructional and reserves uses will remain consistent if UC libraries opt to upgrade to Gale’s OneFile or move to EBSCO’s ASP/ASC products. Thomson Gale recently was purchased by an investment firm; the business is now called Cengage Learning. With the recent acquisition, it is premature to know what new directions, different philosophies or changing business practices incompatible with UC expectations and standards might be introduced by Gale’s new parent. EBSCO is a privately held company which conceivably can respond more quickly to customer feedback. Both Gale and EBSCO are aggregators of journal content published by other companies. As such, both vendors are dependent on and subject to the publishers’ agendas and demands. At this time it appears that no barrier to use exists beyond what is stated in our current contracts with Gale and EBSCO. Charge #6: Analyze the cost benefits for each product, considering the impact of campus co-investments if the Task Force made the recommendation to discontinue Expanded Academic ASAP. As implied earlier in several sections of this report, a simple cost-benefit analysis of ASAP, OneFile, ASP and ASC is neither straightforward, nor concise. While the report has addressed the various aspects delineated by the Task Force charges, one major dimension in any cost/benefit analysis of licensing a costly electronic product is missing – patron success and satisfaction in using the resource. To acquire this essential perspective the Task Force turned to the responses submitted by skilled users, our UC librarian-colleagues, and less skilled searchers, typically UC faculty and students, who participated in our surveys, focus groups and usability tests. Although the surveys yielded imperfect data, all comments submitted by participants are nevertheless useful. Each of the surveys asked different questions; consequently, the responses elicited varied in specificity and character. Other variables among respondents include: Affiliation/status of respondent (librarian, graduate student, undergraduate, etc.) Extent of respondent expertise with searching the Gale and/or EBSCO databases and interfaces Subject expertise of respondent Previous use of resource and/or interface Time and depth of prior search experience Whether respondent completed the comparisons among two or more databases 18 Selection of search topic Complexity of search Despite the variability in the surveys and the respondents, some observations and comments can be made: Survey participation: 1) Participation in the surveys was surprisingly low. 2) Several campuses did not contribute feedback, or contributed very minimally. 3) Faculty did not participate in any of the local campus surveys, with the possible exception of the two UCLA responses. 4) Graduate students contributed very minimally to the local campus surveys. 5) Few librarians from the sciences participated in the surveys. Database use: 1) Previous users familiar with EBSCO’s ASP or ASC submitted more favorable comments for the EBSCO resources than for the Gale products. 2) Many students had little or no familiarity with Gale’s Expanded Academic ASAP. 3) Some participants opted not to perform comparative searches across the four different databases and instead searched one or more databases offered by one vendor. Searchability: 1) Many respondents commented about both the Gale and EBSCO interfaces and search features; comments for both interfaces ranged from positive to negative. Results: 1) More users opted to test the EBSCO databases, while not trying the same searches in the Gale products. 2) Many librarian-respondents searched specialized topics, rather than the more general, multidisciplinary topics undergraduates typically explore. 3) Users could not discern much, if any, difference in the comprehensiveness and scope between EBSCO’s Academic Search Premier and Academic Search Complete. 19 V. Conclusions: The Task Force reaffirms that the UC academic community needs access to a convenient multidisciplinary database offering undergraduates and other users a resource that covers a diversity of scholarly publications, popular periodicals and additional content such as images, book reviews and other materials. We also agree that users increasingly want to retrieve and access full-text content whenever possible. After learning from Gale that the company plans to direct their development to Academic OneFile, the Task Force concluded that Expanded Academic ASAP is likely to become a static product not sufficiently robust to support the multidisciplinary needs of students and scholars within the UC system. During this assessment EBSCO conveyed to the Task Force that the company plans to focus development on their Academic Search Complete (ASC) product. Although Task Force members, librarian testers and even some student participants noted that at this time Academic Search Complete did not offer an appreciable amount of content over the less expensive Academic Search Premier, we question what ASP will offer in the near future if, indeed, EBSCO focuses on developing ASC to the exclusion of ASP. EBSCO, in fact, contends that they are directing their attention to quickly adding many more academic titles, expanding content coverage and increasing the percentage of full-text content in ASC. Evidence gathered over the past few months seems to bear out their assertion. An additional criticism of ASC is that the resource lacked the scope of news content readily available in OneFile. With access to MasterFILE Premier and Newspaper Source, along with ASP or ASC, the gap in coverage between Gale and EBSCO is quickly closing. Task Force members did note that EBSCO has a higher percentage of embargoed titles than does Gale. However, evidence shows that the UC has full text access to the majority of these academic, peer-reviewed titles through licenses currently in place with large vendors of electronic journal content such as Taylor and Francis, Elsevier and others electronic journal publishers. By subscribing to OneFile, UC libraries will be paying twice for the same content. Neither vendor interface nor functionality package obviously stands out as superior to the other. Novice users can undertake successful searches with little to no previous instruction. Of course, better searches result from more experience and library instruction. Participants and Task Force members vary in their preferences and opinions about which interface and functionalities better serve users. While Gale offers some outstanding options, including a hierarchical structure of suggested search term alternatives, EBSCO seems to be preferred by patrons who have used the vendor interface in the past. The one science librarian who participated in the test noted that EBSCO maintains a more favorable reputation for providing the image quality and clarity essential to health sciences patrons. Although the Task Force was not asked to evaluate the customer service ethic or practice of the two vendors, we cannot help but notice differences in communication 20 style between Gale and EBSCO. Representatives from both companies are helpful, informative, professional, friendly and communicative when asked for assistance. We noted that EBSCO representatives offer a higher, more holistic, level of service and communication that is the hallmark of the company. With their reputation built on proactive service as a print serials jobber, EBSCO has continued to offer the same quality of fast, efficient and effective communication and problem solving with their own electronic products. During the assessment, for example, EBSCO worked conscientiously and tirelessly to resolve access problems that some UC campuses were experiencing accessing EBSCO products through VPN; eventually the vendor diagnosed the problem to local outdated computer hardware. Throughout the duration of the study, EBSCO representatives regularly kept the Task Force informed of any recent developments or changes with the ASP and ACS databases, changes in business plans or new functionality or technological developments on the horizon. The Task Force recently learned, for example, that EBSCO is working on developing federated search options to use with their databases, and that in the months ahead thumbnail images of the tables, graphs and other illustrative material within an article will display on the citation record. Gale, perhaps because of their recent acquisition by Cengage Learning, their new organizational structure and the departure of the representative assigned to liaison with UC over the past few years, was much less proactive in developing and maintaining communication channels with the Task Force. Recommendations: The last charge to the Task Force was to assess the cost benefits for each product, and “consider the impact of campus co-investments” if we recommend that the Expanded Academic ASAP contract not be renewed. For reasons stated above we feel that ASAP is no longer a viable renewal option for the UC. The EBSCO Academic Search Premier is the product comparably priced with ASAP, and as noted above, the majority of testers stated that they currently are as satisfied with Premier as with Academic Search Complete, a more expensive resource comparable in price to the Gale OneFile. Again, for reasons stated above, ASP appears destined to join ASAP as a resource inadequate to meet future UC needs, despite its popularity with testers at present. While the Gale Academic Onefile offers many positive features, it does not have the competitive pricing advantage that the EBSCO Academic Search Complete package offers. Therefore, the Task Force recommends that the UC accept the EBSCO offer for Academic Search Complete. 21 APPENDIX A: Charge to the EBSCO-Gale Database Evaluation Task Force EBSCO-Gale Database Evaluation Task Force April 6, 2007 Background: Currently, UC libraries license Gale’s Expanded Academic ASAP database to function as a general multidisciplinary database, aimed at an undergraduate population. Expanded Academic ASAP is funded centrally by SCAP, and campuses pay no coinvest share. In the years since this license was formalized, offerings by database vendors have changed. Gale has introduced Academic OneFile, which offers significantly more content at a significantly higher price. EBSCO is currently offering two similar products: Academic Search Premier (which has been offered as a trial for UC libraries until December, 2007) and Academic Search Complete. Among these four products, CDC wishes to identify the general database that best meets the needs of undergraduate students best meets the needs of any user group interested in multidisciplinary coverage of subjects; and best meets the criteria of ease of use. In considering the added-content pair (Academic OneFile and Academic Search Complete), there are cost implications. While both basic products would be cost neutral for the campuses, both added-content databases would require campus co-investments on top of the CDL share. It is recognized that exchanging one existing UC licensed resource for a comparable other vended resource is a labor intensive process that should not be undertaken lightly. Should such a decision be made, CDL will make every attempt to arrange a transition period of at least 6 months during which both the new resource and old resource will be available. 22 Charge: A task force is charged by JSC to conduct an evaluation comparing the following products from 2 major general multidisciplinary database vendors (Gale and EBSCO): Basic Products – Cost Neutral Added-Content Products – Requires additional campus co-shares Gale’s Expanded Academic ASAP Gale’s Academic OneFile EBSCO’s Academic Search Premier EBSCO’s Academic Search Complete The evaluation should include the strengths and limitations of each database and a recommendation which names the most appropriate general, multidisciplinary database which meets the current and near-term future needs of students. A recommendation to upgrade to either Gale Academic OneFile or EBSCO Academic Search Complete will require campus co-investment. Any recommendation to change the current vended resource, Expanded Academic ASAP, to another resource will have an effective date of July, 2008. Specific Evaluation Components: 1) Identify unique (i.e. title/subject exclusives) content and coverage (i.e. dates, selective or cover to cover) for each database evaluated. In assessing the value of the full-text contained within each database, the task force should also consider any overlap with existing systemwide licensed resources (e.g. with journals licensed directly from the publisher). 2) Assess completeness of content (i.e. abstracts only, full-text) 3) Assess reliability/stability of content 4) Identify any functionality that affects the usability of or access to content (i.e. search, retrieval, display, manipulation) 5) Highlight any barriers to use (i.e. licensing features, such as perpetual access, ILL, or instructional/reserve use) 6) Analyze cost benefits for each database product, including the campus coinvestment implications if a recommendation is made to discontinue Expanded Academic ASAP. 23 Timeframe: Progress report due to UC CDC: August 1, 2007 Final report due: October 1, 2007 Evaluation Task Force Members: Myra Appel (Chair) Vicki Bloom (UCR, chair of General Reference Bibliographers Group) Harold Colson (UCSD) Frank Gravier (UCSC) Cynthia Johnson (UCI) Wendy Parfrey, ex officio (CDL) Gail Yokote, ex officio (JSC) CDL Contact: Wendy Parfrey – business and licensing comparative information JSC Contact: Gail Yokote – questions about charge, progress, etc. 24 Appendix B: Comparison of Sample Search Topics (Separate spreadsheet attached) 25 Appendix C: EBSCO-Gale Functionality Matrix (Separate spreadsheet attached) 26 Appendix D: Summary of Survey Results I. System-wide Survey Responses: Location: CDL internal web site (password protected) Survey Tool: Websurveyor Time period: 88 days (June 15, 2007 - September 10, 2007) Respondents: 15 librarians UCSD – 5 surveys (2 respondents) UCSB – 1 survey (1 respondent) UCR – 3 surveys (2 respondents) UCI – 11 surveys (8 respondents) UCD – 2 surveys (2 respondents) UCB – 1 email response to Task Force chair; respondent expressed a preference for an EBSCO product, since ASP/ASC would complement EBSCO’s Business Source Premier and offer users the same interface Academic Search Premier (7 surveys submitted) Highly recommend as CDL general multidisciplinary resource = 3 Recommend as CDL general multidisciplinary resource= 2 Do not recommend as CDL general multidisciplinary resource= 2 Overall rating (1-5 - 5 being the highest) 3.71 Academic Search Complete (9 surveys submitted) Highly recommend as CDL general multidisciplinary resource= 6 Recommend as CDL general multidisciplinary resource=1 1 Do not recommend as CDL general multidisciplinary resource= 1 No comment =1 Overall rating (1-5 - 5 being the highest) 4.37 *it appears that one librarian got the ratings confused Gale Academic OneFile (5 surveys submitted) 27 Highly recommend as CDL general multidisciplinary resource=1 Recommend as CDL general multidisciplinary resource=2 Do not recommend as CDL general multidisciplinary resource=2 Overall rating (1-5 - 5 being the highest) 3.6 Expanded Academic ASAP(1 survey submitted) Recommend as CDL general multidisciplinary resource=1 Overall rating (1-5 - 5 being the highest) 3.0 UC Survey Questions: Q1. Evaluator/Searcher name: Q2. Bibliographer Group/Area of Expertise: Q3. Campus: Q4. Resource/Vendor: Please rate on a scale of 1-5 (1 is poor; 5 is excellent) Q5. Usefulness for most undergraduate research? Q6. Usefulness for most interdisciplinary research? Q7. Amount of full text articles of core journals in the discipline? Q8. Access to content of relevant non-scholarly articles? Q9. Friendliness (ease of navigation, clarity of presentation, aesthetics)? Q10. Scope/Coverage (comprehensiveness of coverage, years included)? Q11. Ease of remote access (requires client download?)? Q12. On a scale of 1-5, my overall rating is? Q13: Check the one that closely resembles your opinion: I highly recommend as the CDL general multidisciplinary resource. I recommend as the CDL general multidisciplinary resource. I do not recommend as the CDL general multidisciplinary resource. Q14: Other comments or feedback? Please provide specific examples. II. UC Davis Survey Responses: 28 Location: UC Davis Library website Time period: April 29, 2007 – September 20, 2007 Respondents: 9 (OneFile – 1 librarian; 1 student; ASP – 2 students; 1 graduate student; 1 library student employee, 1 staff member; ASC – 2 librarians) The Davis survey did not query for overall recommendation. However, most user responses were favorable to Ebsco. Only one person looked at Gale’s Academic OneFile. UC Davis Survey Questions: Q1. Your UC Davis affiliation? Q2. List the resources (Internet search engines, databases) that you use to look for similar information. Q3. Describe your experience using this resource. Q4. Describe the subject that you searched for in this resource. Q5. Describe why and how you might use this resource. III. UCLA Survey: Location: UCLA Library website Time period: October 2007 Respondents: 2 (OneFile – 1, seemingly a faculty member or graduate student; ASC 1, seemingly a faculty member or graduate student) The UCLA survey did not query for overall recommendation. Respondents liked the databases as a source(s) for locating additional materials not covered by their subject specific database (ASC & OneFile), for finding international content (OneFile) and for offering an RSS feature (OneFile). Both respondents cited the user-friendly interfaces of both products. IV. UC Riverside Survey: Location: UC Riverside Libraries website + one student survey using a canned question Time period: August 2007 - October 2007 29 Respondents: 10 (OneFile -1 staff member; ASP - 3 librarians; 1 graduate student; ASC - 3 librarians) Recommendations: mixed ASP – 2 out of 4 - not as good as Expanded; 1 out of 4 - better; 1 - no recommendation ASC – 1 out of 2 - better, 1 out of 2 - not as good as Expanded OneFile – no recommendation UCR Student response (ASP v. Expanded): ASP Expanded Ease of use 9 8 Limiting 6 7 Full text 80% 70-80% Relevancy 6 7 Marking & 10 6 Exporting UC Riverside Survey Questions: Q1. Your UCR affiliation? Q2. What topic(s) did you search? Q3. How easy was the database to use? Q4. Describe your experience using this resource. Q5. How relevant were the results found? Q6. Describe why and how you might use this resource. Q7. How does this resource compare to the other Gale general database, Expanded Academic ASAP? Q8. How does this resource compare to the other Ebsco general databases, namely Academic Search Premier (ASP) and/or Academic Search Complete (ASC)? UC Riverside focus group 30 Location: Rivera Library (using H-ITT classroom response system “clickers”) Time period: October 16, 2007 Respondents: 4 undergraduate students; 1 graduate student UC Riverside Focus Group Questions: Ebsco Q1. How easy is this database to use? 33% - EASY 50% - AVERAGE 17% - No opinion Q2. How relevant are the results found? 40% - HIGHLY RELEVANT RELEVANCE 40% - RELEVANT 20% - LIMITED Q3. Is there an acceptable mix of scholarly articles, popular magazines, newspapers, etc? 60% - GOOD MIX 20% - NEED MORE SCHOLARLY 20% - DON’T KNOW Q4. Grade IT: the amount of full text 60% - A 40% - B Q5. How useful is the “Narrow Results by Subject”? 80% - HIGHLY USEFUL 20% -USEFUL Q6. How useful is “Limit your Results”? 20% - HIGHLY USEFUL NOTICE 20% -USEFUL 60% - DID NOT Gale OneFile Q1. How easy is this database to use? 80% - AVERAGE 20% - COMPLICATED Q2. How relevant are the results found? 40% - HIGHLY RELEVANT 60% - RELEVANT Q3. Is there an acceptable mix of scholarly articles, popular magazines, newspapers, etc? 31 60% - GOOD MIX 20% - NEED MORE SCHOLARLY 20% - NEED MORE POPULAR Q4. Grade IT: the amount of full text 40% - A 60% - B Q5. How useful is the “Related Subjects”? (asked twice – confusion on location) 33% – HIGHLY USEFUL 60% - USEFUL USEFULNESS 67%- USEFUL 20% - USEFULNESS 20%- LIMITED Q6. How useful is “Limit results”? 80% - USEFUL 20% - did not notice them Q7. How does this resource compare to the first database? The same - 40% Not as good – 60% 32 SURVEY COMMENTS ORGANIZED BY CATEGORY Legend: UClibn - UC wide survey UCDlibn, staff, grad, stud - UCD survey UCRlibn, grad, stud - UCR survey or focus group INTERFACE & SEARCHING/INDEXING – “positive comments” GALE EBSCO “Gale's original screen is simpler and its display is simpler, which I think undergrads would like.” UClibn ”I really like the Overall user interface of EBSCOhost. Students will find the database useful friendly and rich in content.” UClibn “Gale's highlighting is in red, which is helpful on the screen. I like the Tools box in the Gale product.” UClibn “Gale One File seems to have one unique feature that Academic Search Complete lacks: a browsable author index.” UClibn “The search within results tool in ASAP was nice.” UCDlibn “I like the tabs.” UCRstud “It looks cleaner to me than Gale.” UClibn “Academic Search Complete has richer interface that includes more basic and advanced search options, including limiting by publication /document type and the number of article pages. It's easy to clear or refine the existing search, and had a better set of discovery options: related records, hotlinked author names and subject terms.” UClibn “It’s much more intuitive than Gale…” UCDstud “It’s simpler looking.” UCRstud “I like that your options for viewing articles (pdf, html, etc.) are noted at the ASC does a much better job of displaying search results and offering the user options to narrow them. You can immediately select subsets by document type (academic journal, 33 bottom of each individual search results. This eliminates using UC-eLINKs unless you absolutely have to.” UCRstud magazines, newspapers, etc.) and the "Narrow Results by Subject" may be useful to narrow a larger search set. UClibn “I think that separating the search results into tabs (magazines, journals, book reviews, etc.) is really helpful – it really helps cut out the kinds of reviews or articles that you’re not looking for.” UCRstud “While returning fewer results, I was significantly happier with Ebsco…the results …were displayed as a mixed group with clear and self-explanatory links. Browsing the results was simple and intuitive.” UCDlibn “I also really like the “search within the results option.” Though there are no suggested keywords to search for (like the ones Ebsco provides), the search within makes up for it.” UCRstud “Ebsco uses larger icons with titles and file sizes allowing a user to browse quickly.” “I think Expanded Academic’s publication search option is much more accessible.” UCRstud “The interface was straight forward and it was not necessary for library help.” UCRstud “The citation view of a single article is very neatly laid out and easy to read.” UCRstud “Accessing the full text was easier using the Expanded Academic database.” UCRstud “This is an easy database to use and it makes accessibility of non-traditional journals easy.” UCRgrad “Browsing publication titles was easier to modify because the link is easily noticed.” UCRstud “Ebsco seemed to have more ways of limiting the search by publications type, document type, and articles with images.” UCRstud “It was easier to narrow my search using the Narrow Results by Subject column on the left.” UCRstud 34 INTERFACE & SEARCHING/INDEXING – “negative comments” GALE EBSCO “I find the Gale interface a little busys and the text layout is not as easy to read as EBSCO.” UClibn “I found the EBSCO interface cluttered and confusing as compared to Expanded Academic ASAP.” UClibn “OneFile also lacked other features the other database had (hotlinked authors, related records, richer subject terms), and overall, it's just a clunkier interface.” UClibn “It was missing some other basic features, like a "clear search" button or the ability to refine searches.” UClibn “I also don't like how Gale identifies articles without full-text as "Author Abstracts." This is extremely misleading since users are looking for articles and might stop instead of taking the extra step to click the UC-eLinks button.” UClibn “…Gale does not allow one to look at a mixed list…[of results].” UCDlibn “Gale’s interface is simpler, but not as good.” UCRstud “I don’t want to see the titles only, I want to see the abstracts displayed.” UCRstud “Why aren’t subject headings displayed when you first do a search, why do they show up at individual search results?” “I did not like the inconsistency of the bib.record- sometimes full page ranges were given (p1342-1350, 9p) & sometimes only the first page and total pages were given (p464, 20p).” UClibn “Some of the subject headings just don’t make sense.” UCRlibn “One of the things that did not work was the "FIND MORE LIKE THIS." UClibn “The search limiter to Government Documents was a little misleading.” UClibn “They need to explain the tabs.” UCRstud It is “a little distracting…they need to do more [user] surveys.” UCRstud “I don’t like the Ebsco interface much: had 35 trouble navigating, such as finding my search history, using folders to print, etc.” UCRlibn UCRstud “I would have liked to be able to filter out "book reviews" from my results.” UCRstaff It’s surprising that Ebsco Keyword is only author supplied, rather than information rich fields.” UCRlibn “Took awhile to find tab to refine search. I imagine you can customize your default search capabilities so it’s not so hard to find such things?” UCRlibn “Using Ebsco, browsing took a little more work to identify the link.” UCRstud COVERAGE – “positive comments” GALE EBSCO “In my areas [government], the content for this product was better than the other, Ebsco. The list of government documents is much more complete and has some good sources such as the CRS reports and briefs.” UClibn “Academic Search Complete also provides better coverage of [science] disciplines, indexing most/all of the journals from the American Chemical Society, Royal Society of Chemistry, and Institute of Chemical Engineers. They index the Science Direct and Springer journals, and do a better job with Blackwell, Annual Reviews and Nature Publishing Group. Among the publishers they index that Gale does not: Thieme, World Scientific, Bentham, Taylor & Francis, and Portland Press, and the Royal Society. Ebsco indexes more of the key science journals.” UClibn “The news coverage is far superior to the other product.” UClibn “Expanded Academic did give me more relevant newspaper articles.” UCRstud “I find more scholarly resources when I 36 “The results in ASAP seemed more relevant for my [under-graduate] audience.” UClibn search for 18th century authors.” UClibn “There are more full-text articles.” UCRstud “There are more scholarly articles. UCRstud “Expanded did better in finding more relevant hits and eliminating the riff raff.” UCRlibn COVERAGE – “negative comments” “I didn't always get results when I narrowed by subject.” UClibn “[Gale] does not as reliably retrieve scholarly resources as EBSCO databases.” UClibn “I was extremely disappointed that some sociology journals - had no full text and some had 12 month most recent embargo. My searches were primarily in the social sciences.” UClibn “…the coverage by date is uneven for many journals.” UClibn “I got fewer results with Gale One File. This product also does not cover my disciplines thoroughly. It only indexes about half of the American Chemical Society journals, and doesn't even cover Royal Society of Chemistry, American Society for Biochemistry & Molecular Biology, or Institute of Chemical Engineers. And while it looks like most of the Elsevier and Springer journals are indexed, their coverage of Blackwell and Nature Publishing Group is weak and other publishers (Taylor & Francis, Bentham, World Scientific, and most notably Wiley) “News content is limited comparatively.” UClibn “Government document content is also much less than the other database.” UClibn “Some titles were repeated or unrelated.” UCDlibn 37 aren't included at all.” UClibn “Not as good. Ebsco had 0 scholarly articles on my subject, Gale had 8.” UCRlibn “[ASC] is not a good source for research materials in Spanish, Portuguese, and Latin American Literature -- lacks a number of key titles in the field” UClibn “I was very disappointed by the very limited foreign language content of [ASP] and even more disappointed by the Spanish & Portuguse Language titles contained in the database. Dutch 187 References French 25,715 Reference German 4254 References Portuguese 1299 References Spanish 58,676 References.” UClibn “[ASC] seems to lack any foreign language sources.” UClibn SAVING/EMAILING/DOWNLOADING – “positive comments” EBSCO “Emailing was also simple.” UClibn “It was easy to download the PDF and …easy to email.” UCDlibn “The personal folder option was the BEST feature of the entire database. Keeping track of articles was very easy, and the option to save, email or print any 38 article in just a click was really great.” UCRstud SAVING/EMAILING/DOWNLOADING – “negative comments” GALE EBSCO “When adding to a folder, the “add” icon changes and a second image must be loaded. Ebsco does not show how many items have been added to the folder.” UCDstud “The format of the email was long and somewhat confusing, but there appears to be options which would email less information.” UCDstud “The Sort by box is kinda clunky.” UCRlibn OTHER FEATURES - “positive comments” GALE EBSCO “I think that Gale's help feature is superior to Academic's. Gale's help is context-sensitive and easy to use.” UClibn “[It] can create RSS feed based on searches, without having to create a profile first. “UClibn “Academic's help feature is not contextsensitive or even database-specific which I found off-putting.” UClibn “I like the visual search option.” UCDgrad “Images are better than in Gale.” UCDlibn 39 “I did like cited references and the fact it listed if cartoons, images, etc. were given.” UClibn OTHER FEATURES - “negative comments” GALE EBSCO “Images are too poor to be used in academic or clinical medicine. This may be an issue in other scientific disciplines and in humanities and social sciences, such as art.” UCDlibn “Tried using the visual search…it seemed odd. When I clicked on it I thought it would find images.” UCRlibn It needs related field/databases… subject categories for the databases not just the names of the databases.” UCRstud OVERALL STATEMENTS “I still like EBSCO.” UClibn “I cannot tell the difference in results between this [ASP] and "Complete". UClibn “I don’t see much difference between the 2 Ebsco databases.” UCRlibn “Not as Good [Ebsco].” UCRlibn “There is nothing better on the market [Ebsco].” UCRlibn “This database could be used for dissertation and thesis research. Given much literature 40 review work is done by graduate students, this database is invaluable. Better than ASAP.” UCRgrad “I just don't like the Gale products as much, but One File is definitely better than Expanded Academic ASAP.” UClibn 41