Quality of information
Considerations for
library & information services in
the networked world
Tefko Saracevic, PhD
tefkos@rutgers.edu
http://comminfo.rutgers.edu/~tefko/
Tefko Saracevic
1
Table of Content
•
•
•
•
Components of quality & definitions
General problem & the Web
Information quality metrics
Information service quality metrics usability
• Methods
• Conclusions
Tefko Saracevic
2
Components of quality
Quality of
information
Tefko Saracevic
Quality of
information
services
3
Interaction
•
•
•
•
Quality of information = content
Quality of information service = usability
Two components of quality interact
But for a user it is mostly a single
experience, they blend
– a service can help or hinder
 assist in getting of information
 or make it hard or even obstruct
Tefko Saracevic
4
What?
Quality of
information
• A pragmatic definition:
The fitness for use of the information
provided
– key concept: fitness for use
• A measure of the value which the
information provides to the user of that
information.
– key concept: value to the user
Tefko Saracevic
5
Problem
• To use Web resources effectively we need
both: providing or finding quality
information & providing quality services
– but Web information, data & databases fuzzy
– structures vary widely; no consistency
– constantly evolve over time
– plus: Web is value neutral
 indiscriminately contains both pearls & junk, truth &
distortions, leading & misleading information ...
Tefko Saracevic
6
Solution
• Some standardized metrics to describe,
recognize, & test quality
– a mechanism for a more precise description &
evaluation of quality things on the Web & in
associated information services
• Many efforts to specify metrics for
information quality & for quality of
information services
Tefko Saracevic
7
quality
Tefko Saracevic
8
Quality & metrics
Quality should be:
– Observable
– Quantifiable, comparable, testable
• Done by use of metrics:
Parameters or measures of quantitative
assessment used for measurement,
comparison, or to track performance
Tefko Saracevic
9
Metrics for quality information
- how to describe, measure it
General categories describing qualities of
information:
1. Intrinsic – given internal characteristics
2. Context – relation to task, matter at hand
3. Representation – relation to user
4. Access – provisions provided
Tefko Saracevic
10
Quality information metrics:
1. Intrinsic
internal characteristics of information
• Authority
– Who did it? Credibility? Reputation?
• Verifiability
– Can be verified?
• Objectivity
– Trustworthy? Bias? Prejudice? Partiality?
• Validity, reliability
– How accurate? Believable? Correct?
11
Quality information metrics:
2. Context
within task or matter at hand
• Relevance
– Relation to topic?
• Appropriateness
– Significant bearing on matter or task at hand?
• Timeliness
– Current? for the matter or task at hand
• Comprehensiveness
– Complete? for the matter or task at hand
Tefko Saracevic
12
Quality information metrics:
3. Representation
within level & need of user
• Organization
– Presentation in a logical, coherent manner?
• Suitability
– Ease of understanding for given user(s)?
• Consistency
– Compatibility in presentation of different sources?
• Conciseness
– Compact presentation; not overwhelming?
Tefko Saracevic
13
Quality information metrics:
4. Accessibility
provisions provided by system
• Availability
– Access? Ability to obtain information in full?
• Convenience
– To use? Perform actions?
• Security
– Restrictions? Protections?
• Integrity
– Adherence to ethical principles?
Tefko Saracevic
14
What?
Quality of
information
services
• Linked to usability:
“Extent to which a user can achieve goals
with effectiveness, efficiency and
satisfaction in context of use”
International Standards Organization - ISO 9241-11 (1998)
• Key concept: achievement of goals in
context of use
Tefko Saracevic
15
What?
Usability
“Usability is a quality attribute that
assesses how easy user interfaces are
to use. The word "usability" also refers
to methods for improving ease-of-use
during the design process.”
Jacob Nielsen (usability guru) definition
• Key concept: ease-of-use
Tefko Saracevic
16
Essential features of
usability
• A user is involved
• That user is doing something
• That user is doing something with a
product, system or other thing
Tom Tullis & Bill Albert (2008). Measuring the user experience:
Collecting, analyzing and presenting usability metrics. Elsevier p. 4.
Tefko Saracevic
17
Essential qualities of
usability
• As far as users are concerned
information should be
– findable
– understandable
– utilizable
• Thus information services should be
built to follow these qualities
– & tested to see if they indeed do that
Tefko Saracevic
18
Metrics for quality
information service - usability
- how to describe, measure it
General categories describing usability:
1. Effectiveness – fit for purpose
2. Efficiency – time for learning, work, use
3. Functionality – browsing, searching ...
4. Satisfaction – user thinking & feeling,
Tefko Saracevic
19
Metrics for quality information service –
usability
1. Effectiveness
in achieving tasks, goals
• Success rate
– How successful, in finding, understanding,
utilizing?
• Accuracy
– Precision in retrieval? Amount of junk?
• Completeness
– Recall in retrieval? Missing?
Tefko Saracevic
20
Metrics for quality information service –
usability
2. Efficiency
in achieving tasks, goals
• Learning: time, cost, effort
– How much to master?
• Using: time, cost, effort
– How much to use?
• Completion rate
– Completed tasks?
Tefko Saracevic
21
Metrics for quality information service –
usability
3. Functionality
• Navigation
– What provided? Hard? Easy?
• Browsing, searching
– What provided? Effective?
• Features available
– Links, relations, guides, help ...?
• Outputs
– Range?
Tefko Saracevic
22
Metrics for quality information service –
usability
4. Satisfaction
subjective responses by users
Satisfaction with:
• Features
• Interaction, process
• Results
• Overall experience
Tefko Saracevic
23
Effectiveness
Intrinsic
success rate
accuracy
completeness
authority
verifiability
objectivity
validity
Context
Efficiency
relevance
appropriateness
timeliness
Information
time, cost
in learning it
in using it
…
Representation
Quality
Functionality
navigation
searching, browsing,
features
outputs
organization
suitability
consistency
conciseness
Tefko Saracevic
Service
Access
Satisfaction
availability
convenience
security
integrity
features
interaction, process
results
overall experience
24
Methods for quality studies
• Libraries & digital libraries in particular
are complex systems
– many methods for study, evaluation
appropriate
 each has strengths, weaknesses
– range of methods used is wide
 there is no “best” method
– but, no agreement or standardization on
any method
Tefko Saracevic
25
Methodologies used
• Surveys (most prevalent)
• Interviews
• Observations
• Focus groups
• Case studies
• Experiments
Tefko Saracevic
26
A popular method:
LibQUAL
Association of Research Libraries
• Suite of services for libraries to solicit, track,
understand & act upon users opinions of
service quality (various fees)
• A Web based survey for users
– 22 core questions plus open ended comments
– tools for analyzing, relating, displaying results
– bundled with training & tutorials for librarians
• Widely used in many countries
Tefko Saracevic
27
What do digital users want
from academic libraries?
• Access to more digital content of all
kinds & formats
• Enhanced functionality:
 e.g. access to full text of articles
• Enhanced content:
 e.g. subject information, summaries, tables of
content, assistance in evaluation of resources
From: Digital information seekers: How academic libraries can support the use of
digital resources. Survey of 12 studies
http://www.jisc.ac.uk/publications/reports/2010/digitalinformationseekers.aspx
Tefko Saracevic
28
How academic libraries can
meet the needs of their users?
• Seamless access to a variety of resources
 catalogs: more direct links
 more resources from e-journals to datasets
• Respond to changing user information
behavior
 e.g. power browsing for specific information in
articles, books
• Function more like search engines &
popular sites – these are familiar to users
Tefko Saracevic
29
How academic libraries can
meet the needs of their users?
... continued
• Include high quality meta data – for
searching, browsing
• Provide guidance to students on
finding & evaluating content
• Advertise its brand & resources to
users; demonstrate value clearly
From same report
http://www.jisc.ac.uk/publications/reports/2010/digitalinformationseekers.aspx
Tefko Saracevic
30
A possible future
• Going from machine-readable & searchable to machine-understandable
– possibly semantic Web
– quality related meta data
– missing in original Web architecture
• So far quality is a human assessment
• Future: machines assisting in
determining quality
Tefko Saracevic
31
Competition
“everybody wants to be in
information”
First time in history:
• Libraries are facing growing competition
from many institutions & fields, & even
commercial firms e.g.
– many built digital libraries or provide
various & innovative information services
– e-books are making major inroads
• What are appropriate responses?
Tefko Saracevic
32
Conclusion:
Value-added
• For libraries & information systems,
librarians & information specialists
– assessing quality of information resources
– & providing quality information service
is the highest level of value added
• Not easy to do but doable
• A major problem: changing mindsets
Tefko Saracevic
33
Thank you
for inviting me!
Tefko Saracevic
34
Presentation in Worlde
http://www.wordle.net/
Tefko Saracevic
35