Enterprise Search and its
Evolution
Dusan Rnic
VP EMEA
Motivation
Why are YOU here ?
 Efficiency
– Metrics from Butler Group …
 Knowledge workers are 60% of labour
 Up to 20 hours a week spent ‘searching’
 Compliance: control & audit (who saw/sees what)
 Rationalisation of existing Search Engines
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Growth is phenomenal ...
Information Created, Captured, and Replicated
57% Compound Annual Growth Rate (CAGR)
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Source: IDC, 2007 “The Expanding Digital Universe”
(1 Exabyte = 1 Million Terabytes)
Heterogeneity of Information Compounds The Problem

Emails

File Systems

Content Management Systems

Purpose Built Databases

Intranet Portals

Collaboration Tools
…….and how and when people want to view the information
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
The genesis of today’s information management
IBM Fellow,
Turing Award winner (1981)
1970 "A Relational Model of Data for
Large Shared Data Banks"
Edgar Codd
ACM Fellow,
First recipient of the
SIGIR Award for Information Retrieval (1983)
1964 SMART
(Salton’s Mechanical Analysis & Retriever of Text)
Gerard Salton
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
The User Interface 40 years ago
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Implementing Search and Discovering The Long Tail
Popular Searches … those that attract attention
Buried Information
Future of Business Is Selling Less of More
Chris Anderson, 2004
More Buried Information
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Volume and Variety of Information …
What information consumes is rather obvious: it consumes
the attention of its recipients. Hence a wealth of
information creates a poverty of attention, and a
need to allocate that attention efficiently among the
overabundance of information sources that might consume it.
Herbert Simon
Nobel Prizewinner &
Turing Award recipient
Information overload is not the problem. A scarcity of attention
to find the right information in it is the problem.
…… We need better Information Visibility
Source: Computers, Communications and the Public Interest, pages 40-41, The Johns Hopkins Press, 1971.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Enterprises can learn from eCommerce
Popular Searches
Dynamically Classify
Summarise & Aggregate
Spotlight Related Content
Suggest
Visualise
AllLanding
these Pages
techniques are required to deliver Information Visibility
Buried Information
Cross Content Navigation
eCommerce historically viewed Search as the key to users finding products / information
Realisation that enabling a Discovery based User Experience driven by
increased Information Visibility is key.
Inc. All rights reserved.
... Search is just oneCopyright©2009
part of Endeca
thisTechnologies,
Proprietary
and confidential.
Discovering the “Black Swan”
Nassim Nicholas Taleb : The Black Swan
(Published 2007)
…. regards many Discoveries as Black Swans
“Undirected" and “Unpredicted”
Comes from the ancient belief that:
'All swans are white'.
Black Swan is a metaphor for
something not known to exist.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
If I didn’t predict and didn’t direct
If I did not know Black Swans existed – how could I predict
Therefore why would I direct the search to a Black Swan
More information needs to be visible to discover Black Swans
E.g.
Dynamic Classification:
Colour:
White
Black
Black & White
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Facilitating Discovery
Search
Banners
Features
All these elements work in Harmony to make information visible
Notices
They are not discrete pieces of logic working autonomously
A user action in one place – causes a contextual response in another
Facilitating Discovery that is Undirected & Unpredicted by the user
Classify
The overall User Experience is the Design Point
Spotlight
Categorise
Relate &
Aggregate
Merchandise
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Does making information visible matter ?

70 executives from leading enterprises attended a Strategy Day

Event facilitated by Michael E Porter
– World renowned business professor from Harvard
 Competitive Strategy
 The Five Forces
 The Value Chain

Concluded that …
– Individual discretion
 is essential to improve top-line revenue and bottom-line profit
 drives operational efficiency & strategic differentiation
– Daily decisions require clearer visibility into information – this requires
a ‘disruptive’ approach
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Driving Competitive Advantage - Highlights

Strategy means competing on the basis of what makes a firm
different
– Improving operational efficiency - IS NOT strategy
– Conducting activities that competitors don’t - IS strategy

Activities in the value chain contribute to differentiation or a low–
cost position

Activities can be optimized by
– Standardization
– Individual Discretion
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Driving Competitive Advantage - Highlights

Strategy means competing on the basis of what makes a firm
different
– Improving operational efficiency - IS NOT strategy
– Conducting activities that competitors don’t - IS strategy

Activities in the value chain contribute to differentiation or a low–
cost position

Activities can be optimized by
– Standardization
– Individual Discretion

Technology supporting individual discretion enables strategy
discretionary decision-making is unpredictable it happens to take
advantage of a specific context
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
A new way for IT to deliver competitive advantage
Source Of Optimization
Process
Standardization
Predictable and routine
Individual
Discretion
Contextual and tailored
• Formalize the sequence of steps
• Allow spontaneous sequences
• Standardize the tasks
• Encourage choice and creativity
• Provide tools to execute tasks
• Provide tools for exploration
Defined by the firm
At the Discretion of the Individual
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
A new way of thinking about information
First …
Arranged by the constraints of physical objects
Then …
Arranged by the constraints of anticipated use
Now …
Arranged at the point of use, at the moment of use
Actionable intelligence : Just in Time not Just in Case
- David Weinberger, “Everything Is Miscellaneous”, 2007.
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
The Analysts …
Information Access
Today is a recasting of search
Future – we believe it will
encompass ad hoc BI and more
Information Fabric
Two components:
Interactive query over customizable web services, API
emulation, etc.
Advanced federation, similar to EII
Converged Index
A hybrid index of structured and unstructured to support
information access and analysis
Information Visibility
Viewed through the lenses of the business benefit for the
verticals they cover
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Takeaways

Search alone is not likely to meet your strategic objectives

Search alone will require follow on projects to expose the long tail
– are you prepared for that additional investment ?

Learn from the parallels in eCommerce

Think in terms of Information Visibility
– Search is just one part of that

Think in terms of the User Experience

Think in terms of Discovery
Think in terms of Enabling Strategy
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.
Thank You
Copyright©2009 Endeca Technologies, Inc. All rights reserved.
Proprietary and confidential.