Denise A. Troll
Distinguished Fellow, Digital Library Federation
Associate University Librarian, Carnegie Mellon
June 16, 2001 – LRRT, ALA

Initiative on usage, usability, & user support http://www.clir.org/diglib/use/useframe.htm
Distinguished Fellowship
Convene library directors to outline research into how & why libraries are changing
Conduct a usage & usability survey to discover DLF library issues & practices

Traditional
Inputs & Outputs
Digital
Inputs & Outputs
What other changes are occurring in libraries?
What do we know about the context?
What’s really happening & why?

Motivational white paper http://www.clir.org/diglib/use/whitepaper.htm
Library Director Meeting
Read & comment on the white paper
Submit the 5 most important reasons for tracking trends , the audiences , & the indicators
Expected consensus on 5-6 new measures

Purpose of survey
Study use of internal & external information
Examine relevant environmental factors
Identify gaps
Provide trend data
Random sample > 3000
Conduct survey fall 2001
Raise $$

What data do DLF sites gather?
What do they do with the data?
What works well? What doesn’t work well?
Institutions
Individuals
Contacted Participated Rate
26 24 92%
83 71 86%

Traditional & digital INPUTS
Traditional & digital OUTPUTS
Traditional & digital OUTCOMES
User satisfaction ratings
Service quality gaps
Usability
Referral URLs
What about cost-effectiveness & learning & research outcomes?

Gathering the right data
Gathering comparable data
Interpreting the data
Compiling & managing the data
How do we do this cost-effectively?
Figuring out how to use the data

Can chat transcripts, email, & FAQ be used to create a reference database?
Ref Quick Ref Real Ref Direct Tech Refer
Ref Desk
In person
Ref Desk
Telephone
Librarian
Desk
Telephone
Email
Chat
Other
What about user demographics?
What about user privacy?
What data to gather?
For what purpose?
How often?
At what cost?
Compare with what?
What about turn-around time?

Costs increase
Technology
Staff
Staff training
Disk space
Revenues decrease
Photocopying
Fines
Should we operate both print
& e-reserves?

Reorganizing, recruiting, & creating positions
Usability & Interface Specialist
Director of Organizational Effectiveness
Distance Education Department
Training staff to gather , use ,
& present data effectively
Stretching the budget

Based on # page hits
Put links to high-use pages high in the hierarchy
Keep high-use pages complete & up-to-date
Put links to low-use pages low in the hierarchy
Publicize or reduce maintenance of low-use pages
Frequent queries
Put links to queried pages high in the hierarchy
Revise site vocabulary

Use by domain, location, or user group
Facilitates collection & service development
Indicates user satisfaction
Helps target publicity
Referral URLs indicate satisfaction & impact
Do publicity or redesign increase use?
Low use times are for system maintenance

Why conduct?
– To assess usage patterns; shifts in attitude; user needs, expectations, & satisfaction; service quality; market niche; & collection strengths, weaknesses, & relevance to the curriculum
How apply the results?
– Inform budget requests
& investments; justify expenditures; reallocate resources; set priorities; target training & instruction

Why conduct? – To understand what users do or want to do; assess the use, effectiveness, & usefulness of collections & services; verify the results of previous research; inform new research
How apply the results? – Confirm hypotheses, assumptions, & user needs; plan library renovations, training, instruction, publicity; inform collection development & access decisions; inform or validate interface design plans; reorganize the library

Why conduct? – To identify problems in interface design, functionality, navigation, & vocabulary
How apply the results? – Inform redesign of the library web site & digital collections; inform customization of the OPAC interface; verify improvements in service quality & interface design; develop library instruction; revise metadata schemes

Card sorting exercises – To group web site links,
& label links & groups
Heuristic evaluations – To identify problems in interface design, functionality, & navigation
Paper prototypes & scenarios – (like protocols)

Lack of standards, guidelines, benchmarks, comparable data, & articulated proficiencies
Recruiting representative human subjects
Bias & bureaucracy
Redundancy
Interpretation
Application