Checklist for Digital/Hybrid Library Web
Sites
Based on Pitchmann's (2001) article
 Includes content as well as interface
design and underlying technology
 Expandable to metrics which relate
content, operations/processes and
systems/technological performance

E. O’Brien Digital Libraries Term Project
Due May 13, 2002
Libraries Evaluated:







MIT – http://libraries.mit.edu
University of Delaware – http://www.udel.edu
UIUC – http://www.library.uiuc.edu
Rice (Fondren) – http://www.rice.edu/frondren
Trinity U. (Coates) – http://lib.trinity.edu
Georgetown U. –
http://www.georgetown.edu/dml
Rutgers U. – http://www.libraries.rutgers.edu
E. O’Brien Digital Libraries Term Project
Due May 13, 2002
Sample Criteria - Content

Is there one formal selection policy statement
for the whole library on each website?
 Is there any statement on the web site that
helps users understand that the services
offered are selective and quality controlled?
 Does the library identify whether or not the
collection is:
–
–
–
–
–
Comprehensive
Research
Study
Basic
Minimal
E. O’Brien Digital Libraries Term Project
Due May 13, 2002
Sample Criteria – Operations/Processes

Is the research process leading to the
creation of the content described?
 Is there evidence that the intended audience
has been identified?
 Does the site adhere to recognized standards
and appropriate technologies?
 What level of user support is available?
(FAQs only, interactive support, help screens,
contact information)
E. O’Brien Digital Libraries Term Project
Due May 13, 2002
Sample Criteria – Systems/Technological
Performance

Are there process or technical criteria?
(Technical features that measure the integrity of the
site and availability of content reported to be
provided)
 Information integrity (Age of site, frequency of
updates, frequency of revisions, version control, link
maintenance)
 System integrity (Server stability, frequency of
downtime, response time, traffic volume and delays,
downtime announcements, projected availability time
and date announcements)

What metadata are used at each web site?
E. O’Brien Digital Libraries Term Project
Due May 13, 2002
Content, process, technology and metrics

Components of electronic libraries (Lakos,
1997)
–
–
–
–
–

Infrastructure
Services
Contents
Management
Ownership
Performance measurement compares what
library is doing (actual performance) with
what it means to do (mission) and what it
wants to achieve (goals)
E. O’Brien Digital Libraries Term Project
Due May 13, 2002
Connecting the components:
Collection
development policy
statement
Acquisition
Digitizing
Using
Retrieving
Cataloging
Linking
Serving
E. O’Brien Digital Libraries Term Project
Due May 13, 2002
Results:
MIT
UIUC
Rice
UDEL
Trinity
RU
GU
Total !
30
28
27
24
18
18
9
Total ?
8
7
9
9
12
9
15
Total 13
18
15
18
21
24
27
Highlights:







Rice/UIUC had excellent collection development policy
descriptions
Only Trinity U describes mission, objectives and
progress assessment
UIUC documents library staff operations yr/yr and
compares data to other libraries.
Only GU and RU did not have info about quality control
and selectivity of services
Only RU announced anticipated downtimes
All libraries described how they identified different user
populations
MIT, UDEL, UIUC had process criteria to measure
integrity of site and availability of content
E. O’Brien Digital Libraries Term Project
Due May 13, 2002
Conclusions:

Pitchmann's criteria can be adapted to
evaluation of academic digital libraries
 There may be better sources for evaluation
(Brophy, 1998 & Lakos, 1997, others)
 Libraries don’t plan for evaluation &
performance measurement
 Plenty of room for future work in this area but
it can be done!
E. O’Brien Digital Libraries Term Project
Due May 13, 2002
Specific recommendations for Rutgers University
Libraries are:






Define AAU criteria that libraries are to help RU meet. State how
libraries will contribute. Include measurements of progress
toward goals and when and to whom results will be reported.
Create mission or vision statement for libraries that includes
mission, goals, current performance. (See UIUC, MIT, Trinity U.
for examples.)
Develop metrics for infrastructure, services, content,
management and ownership of libraries that link the impacts of
performance in each area. (See Lakos article.)
Provide information on RU libraries’ web site about service and
collection quality. (See MIT, UDEL, Rice and Trinity U. as
examples.)
Update all pages on web sites that deal with mission statement,
metrics, collection development.
Add information to web sites about how interests and needs of
users are explored and how often their input is sought. (See
MIT, UIUC as examples.)
E. O’Brien Digital Libraries Term Project
Due May 13, 2002