evaluating
information on the web
Tefko Saracevic
School of Communication, Information
and Library Studies
Rutgers University
http://www.scils.rutgers.edu/~tefko
© Tefko Saracevic, Rutgers
University
1
evaluating internet
resources
• impossible? not really
• hard? very
• help? exists if you persist
• LECTURE TOPICS:
– how to go about it?
– what are the main criteria?
– where to verify?
© Tefko Saracevic, Rutgers
University
2
the web
• fastest growing technology in history
• explosive growth of WWW provided
– ubiquity of information and access
– but also information chaos & anarchy
• growing difficulty in identifying, searching,
retrieving and EVALUATING
• metaphors: ‘lost in an ocean’ ‘finding pearls
in garbage dumps’ ‘needle in haystack’
© Tefko Saracevic, Rutgers
University
3
web is value neutral
• all kinds of information can be found
– misinformation
• deliberate, just plain wrong or plain stupid
– disinformation, censored
– hate information
– propaganda, spin doctored information
– questionable, inaccurate,
– harmful, objectionable, insulting
© Tefko Saracevic, Rutgers
University
4
value neutral ...
• but information that is also
– valid, reliable, useful, relevant,
accurate, factual, timely, credible …
• to a high degree
• appropriate to many problems & tasks
– otherwise hard or impossible to find,
retrieve & access
– from sources that are trustworthy
© Tefko Saracevic, Rutgers
University
5
prerequisite knowledge
• to evaluate web information needed
knowledge about
– web structure & mode of operandi of the
internet & domain name system
– notion & characteristics of cognitive
authority
– criteria adapted for the web
© Tefko Saracevic, Rutgers
University
6
evaluated
© Tefko Saracevic, Rutgers
University
7
cognitive authority
“influence on one’s thoughts that one
would consciously recognize as proper”
Patrick Wilson
• related to assignment of credibility
– two components:
competence & trustworthiness
• ascribed to particular individual,
institution, organization, action
© Tefko Saracevic, Rutgers
University
8
problems
• on the web
– traditional authority indicators difficult
to attribute - often absent
• authorship? title? version? place of origin?
• author qualification? credentials?
– no filtering
– vanity publishing
© Tefko Saracevic, Rutgers
University
9
problems ...
– sometimes even attribution difficult
• identity? reputation? qualifications?
• can be published by anyone
• anyone can claim to be somebody else
• assigning credibility to Web
information a BIG problem
© Tefko Saracevic, Rutgers
University
10
solutions
• rigorous evaluation
– more detailed than print sources
• depending on known authority sites,
authors, organizations
• following many sites that did
evaluation already
© Tefko Saracevic, Rutgers
University
11
evaluation criteria
• many traditional criteria remain
but with new interpretations
• a number of new criteria have
emerged specific to digital nature
of resources & access
• many are stated & can be found on the
Web e.g. library sites
© Tefko Saracevic, Rutgers
University
12
web & cognitive authority
need to carefully asses (always, web or no web):
document, author,
institution & affiliation
on criteria of:
authority
accuracy
currency
objectivity
coverage
© Tefko Saracevic, Rutgers
University
13
specific evaluation criteria
• what & why? - documents, objects
– content? purpose? scope? viewpoint?
• by whom? - creators, authors, institutions
– identity? authority? credibility?
reputation? qualification? refereeing?
• where? - affiliation, connections
– identity? overt? covert? authority?
credibility? reputation?
© Tefko Saracevic, Rutgers
University
14
criteria ...
• for whom? -
orientation
– intended audience? needs satisfied?
fit with user community?
• when? - timeliness
– currency? up-to-date? revisions?
persistence estimate?
• how? - treatment, coverage
– accuracy? credibility? objectivity?
style? clarity? organization? usability?
© Tefko Saracevic, Rutgers
University
15
criteria ...
• in what way? - presentation
– format? layout? interface? search
capabilities? access?
• how much? - economics
– effort? price? cost-benefits? license?
• in comparison to? - competition
– other similar resources?
© Tefko Saracevic, Rutgers
University
16
model for evaluating
information on the web
• INPUT: filter & assessment for
– document
– author
– institution
– affiliation
• OUTPUT: combined assessment &
ascription of cognitive authority
© Tefko Saracevic, Rutgers
University
17
model ...
input from the web
assess document
assess author
filter
assess institution
assess affiliation
ascription of cognitive authority
© Tefko Saracevic, Rutgers
University
18
examples
• newspapers
– New York Times; many others
• governments
– in the US: Census Bureau; State
Department; Nat. Inst. of Standards
• organizations
– www consortium (w3c)
© Tefko Saracevic, Rutgers
University
19
examples ...
• international
– UN, European Union agencies
• professional
– Assoc. for Computing Machinery
• health
– Mayo Clinic; Rx list for pharmaceuticals
© Tefko Saracevic, Rutgers
University
20
examples ...
• science
– national academies of many countries
• commercial
– encyclopedias, reference sources
– Britannica has evaluated web sites
• publishers
– evaluated sites e.g. Choice
© Tefko Saracevic, Rutgers
University
21
libraries & web authority
• emerged as an important source
– many provide evaluated sites & links
• judicial & trusted selection:
– a key value-added contribution by
libraries internationally
– trust extends to digital collections
– makes all the difference between a
library & other collections
© Tefko Saracevic, Rutgers
University
22
library examples
• national libraries: many links
– Library of Congress; UKOLN (UK)
• academic libraries: great many
– U of Michigan: law
– U of California Berkeley: many domains
– Virtual Library (Switzerland)
– etc. etc. etc. GREAT sources!
© Tefko Saracevic, Rutgers
University
23
reference, journals
• elaborate online reference sites
– Martindale’s reference desk
– some commercial e.g. Ask Jeeves
– reference questions answered online
• pathways, guides
• publications - some free other
licensed - licensing now a big deal
© Tefko Saracevic, Rutgers
University
24
specialized
• digital libraries emerged in great
many domains, fields
– history, national memories
– arts, museums, music, poetry ...
– science, technology
– geography, climate, weather
– cooking, stamp collecting, sports ...
© Tefko Saracevic, Rutgers
University
25
conclusions
• web sources HAVE to be evaluated
• many evaluations follow traditional
criteria e.g. as for news accounts
• many new criteria evolved
• many tools already there
• hard but possible!
© Tefko Saracevic, Rutgers
University
26
sources
• URLs for the mentioned & many
other sites can be found on:
http://scils.rutgers.edu/~tefko/D-Lib_Edu/
© Tefko Saracevic, Rutgers
University
27
© Tefko Saracevic, Rutgers
University
28