Enterprise Search and its Evolution Dusan Rnic VP EMEA Motivation Why are YOU here ? Efficiency – Metrics from Butler Group … Knowledge workers are 60% of labour Up to 20 hours a week spent ‘searching’ Compliance: control & audit (who saw/sees what) Rationalisation of existing Search Engines Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Growth is phenomenal ... Information Created, Captured, and Replicated 57% Compound Annual Growth Rate (CAGR) Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Source: IDC, 2007 “The Expanding Digital Universe” (1 Exabyte = 1 Million Terabytes) Heterogeneity of Information Compounds The Problem Emails File Systems Content Management Systems Purpose Built Databases Intranet Portals Collaboration Tools …….and how and when people want to view the information Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. The genesis of today’s information management IBM Fellow, Turing Award winner (1981) 1970 "A Relational Model of Data for Large Shared Data Banks" Edgar Codd ACM Fellow, First recipient of the SIGIR Award for Information Retrieval (1983) 1964 SMART (Salton’s Mechanical Analysis & Retriever of Text) Gerard Salton Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. The User Interface 40 years ago Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Implementing Search and Discovering The Long Tail Popular Searches … those that attract attention Buried Information Future of Business Is Selling Less of More Chris Anderson, 2004 More Buried Information Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Volume and Variety of Information … What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it. Herbert Simon Nobel Prizewinner & Turing Award recipient Information overload is not the problem. A scarcity of attention to find the right information in it is the problem. …… We need better Information Visibility Source: Computers, Communications and the Public Interest, pages 40-41, The Johns Hopkins Press, 1971. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Enterprises can learn from eCommerce Popular Searches Dynamically Classify Summarise & Aggregate Spotlight Related Content Suggest Visualise AllLanding these Pages techniques are required to deliver Information Visibility Buried Information Cross Content Navigation eCommerce historically viewed Search as the key to users finding products / information Realisation that enabling a Discovery based User Experience driven by increased Information Visibility is key. Inc. All rights reserved. ... Search is just oneCopyright©2009 part of Endeca thisTechnologies, Proprietary and confidential. Discovering the “Black Swan” Nassim Nicholas Taleb : The Black Swan (Published 2007) …. regards many Discoveries as Black Swans “Undirected" and “Unpredicted” Comes from the ancient belief that: 'All swans are white'. Black Swan is a metaphor for something not known to exist. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. If I didn’t predict and didn’t direct If I did not know Black Swans existed – how could I predict Therefore why would I direct the search to a Black Swan More information needs to be visible to discover Black Swans E.g. Dynamic Classification: Colour: White Black Black & White Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Facilitating Discovery Search Banners Features All these elements work in Harmony to make information visible Notices They are not discrete pieces of logic working autonomously A user action in one place – causes a contextual response in another Facilitating Discovery that is Undirected & Unpredicted by the user Classify The overall User Experience is the Design Point Spotlight Categorise Relate & Aggregate Merchandise Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Does making information visible matter ? 70 executives from leading enterprises attended a Strategy Day Event facilitated by Michael E Porter – World renowned business professor from Harvard Competitive Strategy The Five Forces The Value Chain Concluded that … – Individual discretion is essential to improve top-line revenue and bottom-line profit drives operational efficiency & strategic differentiation – Daily decisions require clearer visibility into information – this requires a ‘disruptive’ approach Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Driving Competitive Advantage - Highlights Strategy means competing on the basis of what makes a firm different – Improving operational efficiency - IS NOT strategy – Conducting activities that competitors don’t - IS strategy Activities in the value chain contribute to differentiation or a low– cost position Activities can be optimized by – Standardization – Individual Discretion Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Driving Competitive Advantage - Highlights Strategy means competing on the basis of what makes a firm different – Improving operational efficiency - IS NOT strategy – Conducting activities that competitors don’t - IS strategy Activities in the value chain contribute to differentiation or a low– cost position Activities can be optimized by – Standardization – Individual Discretion Technology supporting individual discretion enables strategy discretionary decision-making is unpredictable it happens to take advantage of a specific context Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. A new way for IT to deliver competitive advantage Source Of Optimization Process Standardization Predictable and routine Individual Discretion Contextual and tailored • Formalize the sequence of steps • Allow spontaneous sequences • Standardize the tasks • Encourage choice and creativity • Provide tools to execute tasks • Provide tools for exploration Defined by the firm At the Discretion of the Individual Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. A new way of thinking about information First … Arranged by the constraints of physical objects Then … Arranged by the constraints of anticipated use Now … Arranged at the point of use, at the moment of use Actionable intelligence : Just in Time not Just in Case - David Weinberger, “Everything Is Miscellaneous”, 2007. Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. The Analysts … Information Access Today is a recasting of search Future – we believe it will encompass ad hoc BI and more Information Fabric Two components: Interactive query over customizable web services, API emulation, etc. Advanced federation, similar to EII Converged Index A hybrid index of structured and unstructured to support information access and analysis Information Visibility Viewed through the lenses of the business benefit for the verticals they cover Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Takeaways Search alone is not likely to meet your strategic objectives Search alone will require follow on projects to expose the long tail – are you prepared for that additional investment ? Learn from the parallels in eCommerce Think in terms of Information Visibility – Search is just one part of that Think in terms of the User Experience Think in terms of Discovery Think in terms of Enabling Strategy Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential. Thank You Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential.