Understanding Differences
Between Federated Search and
Discovery Services
By Abe Lederman
President and CTO
June 26, 2011
© 2011 Deep Web Technologies, Inc.
About Deep Web Technologies...
• Founded by Abe Lederman in 2002
–A co-founder of Verity
–20+ years in information business
• 24 person company based in Santa
Fe, New Mexico
• Over $5M in R&D
• Pioneered federated search
• Developed high profile applications
© 2011 Deep Web Technologies, Inc.
2
What Is Federated Search?
Federated Search allows users to
submit a real-time search in
parallel to multiple information
sources and retrieve aggregated,
ranked and de-duplicated results.
© 2011 Deep Web Technologies, Inc.
3
One Search, Many Sources
Blogs
eBooks
Enter Your Search…
Subscription
Sources
Begin Search
Wikis
Internal
Databases
© 2011 Deep Web Technologies, Inc.
Public Web
Sources
4
Journals
Federated Search Has Gotten A Bad Rap
• It is too slow
• Connectors break
• Brings back too few results from
each source
• Brings back too many results
• Unable to rank results well (metadata differences, lack of info)
© 2011 Deep Web Technologies, Inc.
5
Federated Search vs. Discovery
Services
Federated
Search
Discovery
Service
Index
No Index
One unified index
Sources
No limit
Must have
agreements
Content Bias
None
Possibly
Information
Currency
Real-time return
Depends on index
update
Preparation
Connector
Development
Index
Development
Speed
2 – 30 seconds
1 – 2 seconds
Control
Select sources
No ability to select
Alerts
Yes
Some
© 2011 Deep Web Technologies, Inc.
6
Landscape is Not So Clear
• Summon (ProQuest)
– Discovery Service
• EDS (EBSCO)
– Discovery Service + Federated Search
• WorldCat Local (OCLC)
– Discovery Service + Federated Search
• Primo (Ex Libris)
– Discovery Service + Federated Search
• Encore Synergy (Innovative Interfaces)
– Limited Discovery Service + Federated Search
• Explorit (Deep Web Technologies)
– Federated Search
© 2011 Deep Web Technologies, Inc.
7
Content Neutrality
Do you comply with the International Coalition
of Library Consortium (ICOLC) Statement, Principle 3?
“We encourage publishers to allow their content to be
made available through numerous vendors
appropriate for their subject matter. We also
encourage online providers and aggregators to allow
their metadata to be included in emerging discovery
layer services on a non-exclusive basis.”
--Carl Grant, "Gladiators" to perform sleight-of-hand at Charleston
Conference.” Commentary from Carl Grant. October 30, 2010.
© 2011 Deep Web Technologies, Inc.
8
Content Neutrality
-Points by Carl Grant
• Is there unbiased ranking of
content?
• Does competition have access to
content?
• Can you control the ranking of
results?
• Does the library have control over
their content?
© 2011 Deep Web Technologies, Inc.
9
Lack of Transparency of
Discovery Service Vendors
• What is being indexed?
–Which Journals/Databases
–What period is covered
• Currency of information
• Indexing full-text vs.
meta-data only
© 2011 Deep Web Technologies, Inc.
10
Ability to Integrate
• How easy/hard/expensive/time
consuming is it to add my catalog
and other internal/special sources to
index?
• How well do you authenticate/
integrate with link resolvers?
© 2011 Deep Web Technologies, Inc.
11
When Should You Choose
Federated Search?
• Access to up-to-date information is
important.
• You want control of your sources.
• You want to search internal/nonmainstream sources
• Your research is specialized (ex.
medical/legal)
• You have a wide range of subscribed
content (ex. EBSCO and ProQuest)
© 2011 Deep Web Technologies, Inc.
12
Getting Federated Search Right
• Display results incrementally
• Proactively monitor information sources
• Optimized connectors bring back
greater number of high quality results
• Do relevance ranking well
• Present best results through filters and
clusters
© 2011 Deep Web Technologies, Inc.
13
© 2011 Deep Web Technologies, Inc.
14
© 2011 Deep Web Technologies, Inc.
© 2011 Deep Web Technologies, Inc.
© 2011 Deep Web Technologies, Inc.
© 2011 Deep Web Technologies, Inc.
More Reading…
• Federated Search Primer
• Quality, Not Quantity Whitepaper
• Blog articles
–Discovery Services: Over-Hyped and UnderPerformed
–Preparing for ALA Panel and Federated
Search Neutrality
–Discovering the need for discovery solutions
that also support meta/federated searching
(Carl Grant)
© 2011 Deep Web Technologies, Inc.
19
Thank you!
Questions?
Please email me:
Abe Lederman
abe@deepwebtech.com
This Powerpoint is located here:
www.deepwebtech.com/ala2011.ppt
© 2011 Deep Web Technologies, Inc.
20