Personalized marketing using recommendation systems Peter A. Csikos 10 February 2009 SOME FACTS ABOUT THE HUNGARIAN E-COMMERCE SPACE… Although the total population is 10 million… … the number of people actually doing anything on the net is low Population over 14 years or older: 8.515.000 Number of regular internet users (14 years or older): 3.596.000 Number of people actually doing ecommerce 1.075.000 Source: GKINET / Hungarian Association for e-commerce Based on 2005 Microcensus Number of people actually using the internet for at least one hour per month (2010 Q1) People who have bought anything over the internet in 2009 ONLINE SHOPPING AND E-COMMERCE BETWEEN 2001 AND 2010 Ratio of people actually doing e-commerce and those who are planning to do so and those who are even not planning to do it… 1.075.000 Not even planning to Nem tervezi use e-commerce Source: Hungarian Association of e-commerce Planning to do e- Tervezi commerce in the future Vásárolt Did e-commerce 2010.I. 2009.IV. 2009. III. 2009. II. 2008. IV. 2008. II. 2007. IV. 2007. III. 2007. II. 2007. I. 2006. IV. 2006. III. 2006. II. 2006. I. 2005. IV. 2005. III. 2005. II. 2005. I. 2004. IV. 2004. III. 2004. II. 2004. I. 2003. IV. 2003. III. 2003. II. 2003. I. 2002. IV. 2002. III. 2002. II. 2002. I. 2001. IV. 2001. III. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% WHAT ARE THESE PEOPLE ACTUALLY DOING ON THE INTERNET? Number of people actually doing online shopping last year online banking Online Banking Doing administrative things ügyintézés Gathering information vásárlás előtti tájékozódás before regular shopping General Information általános információk 0% 20% 40% Gyakran veszi igénybe Regularly © GKINET / Association of e-commerce 60% Igénybe veszi sometimes 80% 100% TURNOVER OF ONLINE SHOPS IN HUNGARY BETWEEN 2001 AND 2010 EUR 500million EUR 350 million Planned © GKINET / Association of e-commerce Actual Key players on the Hungarian ecommerce market Sanoma CEMP Index Velvet Totalcar Inda Portfólió Etarget InfoRádió BOOKLINE DVD Rent C-TRAVEL OnGo Honfoglaló Mobile24 Burek ArtOffice [origo] Origo Freemail IWIW [ORIGO] TÉKA Videa Megálla-podások más szolgáltatókkal ORIGOVÁSÁRLÁS Startlap Hotdog PROFESSION JOBMONITOR Vezess.hu Figyelőnet Citromail Videoplayer STARTAPRO Mobilport Wellnesscafe Hírstart Nők Lapja Café Geographic Cosmopolitan Storyonline HÁZIPATIKA.COM csoport TECHNET SPEEDSHOP.HU Holtzbrinck Networks Randivonal.hu Brands.hu Habostorta.hu Funpic.hu Gyaloglo.hu Gumicsizma Allegro Hungary Izé.hu Lira.hu Metnet.hu NB1.hu Minimax Vatera.hu TeszVesz.hu Képzésinfo.hu KutatóCentrum Matton Images Maxima Monddmeg.hu Terminal.hu Netpincer.hu Sportolj.hu Extreme Digital EXTREMEDIGITAL.HU Klikkmánia.hu Nethajó.hu Ebolt Kft. EBOLT.HU Lapcom HASZNALTAUTO.HU Garanciális.hu SZALONAUTÓ.HU BAZÁR.HU AUTÓBAZAR.HU GRoby Groby.hu Jobline Ingatlamenedzser Autómenedzser Adózóna Ecoline Techline Medizóna Eduline URLguru Travelline Aprónet Life & Style Zöldtér Hírszerző General Media Arukereso.hu Otomoto.hu Viala Kft. Hírek Média DEPO.HU Egyperces.hu Terminál CTnetwork Hírek.hu JOBINFO.HU HVG (WAZ) INGATLANBAZAR.HU Workania.hu Delmagyar.hu Kisalfold.hu Hahu.hu MyVIP.hu MYCLUB.HU Donna.hu DonnaLight myMAP.hu SportHÍRADÓ Olimpia.hu Nascar.hu Data.hu INGATLANOK.HU Hirposta.hu Love.hu TALALKA.HU JOJATEK.HU MultiPlay Chat.hu Mellesleg.hu Farm.Hu Postafiok.hu Fashion.hu Divat.hu BOOK.HU Freeweb.hu Arkon Zrt. Ingatlan.com (B2B) Koponyeg.hu Utcakereso.hu Alaprajz.hu Linkcenter.hu Blogol.hu Telepiuleskereso.hu TYPICAL HUNGARIAN PROBLEM FOR E-COMMERCE STARTUPS: GETTING THE DEVELOPERS CLOSE TO THE INDUSTRY Focus, Focus, Focus Differences in motivation freedom of development fame; IT cups, contests Market share, market size IT excellence Publication, open source © Norbert, Buzas, ValDeal / University of Szeged New Products ROI, NPV, IRR official regulations Business Card – Gravity R&D • Domain: Personalized Recommendation Solutions for ECommerce, Digital Media and IPTV/Digital Cable Service Providers • Our offering: Delivering real time, personalized context-based product recommendations to each and every user, the Gravity solution produces measurable increase in revenues from existing customer bases and improves customer satisfaction • A fast-growing company: significant number of customers and several pilot projects running including ALEXA top35 video live streaming site, 5 to 30 employees within 12 months • Investors: US and Hungarian investors, offices in Budapest, London, Utrecht and Berlin, sales representatives in Romania, Russia and Israel and China 8 E-commerce market trends 9 The problem 10 Market trends • Personal product recommendations are one of the TOP5 most important emerging technology priorities in e-commerce (Internet Retailer, „Emerging Technology” conducted by Vovici Corp, Sept 2008) • 50% of online retail stores want to introduce personal recommendations (Internet Retailer, „Emerging Technology” conducted by Vovici Corp, Sept 2008) • 40% of US e-commerce sites plan to adopt customer reviews and ratings (The Kelsey Group and ConStat Inc, „Local commerce monitor”, Aug 2008) • 77% of retail executives said that online behavior tracking is the most promising website technology for online customer engagement (Retail Systems Research, „Playing Well with Others: eCommerce’s Evolving Role in the Customer Experience”, Aug 2008) • 76% of US e-commerce executives said the personalized recommendation may or definitely increase loyalty (The Executive Guide to Captivating Customers, June 2008) • Social shopping sites and collaborative technologies are deemed currently as leading shopping innovations and expected to be widespread by 2015 (US household shoppers prediction) (TNS Retail Forward, „New Future in Store”, May 2008) 11 THE EMERGENCE OF RECOMMENDATIONS Internet-based shopping and media consumption differs from the classical one In-store, personal shopping experience Customers know how to find the desired products Assistances help the customer: show them the desired products & recommend Internet shopping experience Web portals offers much more products, customers get lost easily Recommendation systems helps the customer to find & discover products The key is to keep the customers active & increase conversion 20% of the customers is accountable for 80% of the profits To gain new customers, companies have to spend 3-4X more than to keep the existing ones Solution: Change the way you personalize 13 WHAT IS A RECOMMENDATION ENGINE? Recommender systems or recommendation engines form a specific type of information filtering system technique that attempts to present information items (films, television, video on demand, music, books, news, images, web pages, etc.) that are likely of interest to the user. 14 HOW DOES IT WORK? User registers or starts using services User browses, watches and optionally rates content System builds a preference profile System provides presonalized recommendations 15 WHAT IT DOES? Recommender logic • • • • Data collection and processing Relevance & preference ordering Display recommendations Self-learning & improving capabilities • • Mathematical models Information systematization 16 THE RECOMMENDATIONS Customer is looking for a product Receive tips: Receive personal offerings: 17 SHORT SCIENCE: RECOMMENDATION ALGORITHMS Recommendation in general: Gravity is using a wide palette of recommendation algorithms The best fitting algorithms are selected – after careful analysis of the data – to the given recommendation problem and the corresponding optimization task. Overview of recommendation algorithms: 1. Collaborative filtering (CF): • Based on events generated in your service (Vod purchase, Live channel watching event), finds similar behavior on users, and similarity on items (VoD content, live schedule, etc.) 2. Content based-filtering (CBF): • Using only user/item metadata. Recommendations are based on matching keywords. Measuring Recommendation Quality: • Average Relative Position (ARP): The distance between the prediction and the user’s choice • Top 10 Recall: the probability of hitting the chosen item from the top 10 items of the personalized list 18 Early generation recommendation solutions… … Did not offer really personalized recommendations for each and every user… • Not personalized • Only based on part of the available information • Low customer retention (if any) •Minimal revenue increase • Lower conversion rate • Increase of customer satisfaction is questionnable 3 THE NETFLIX COMPETITION – RECOMMENDATION QUALITY MATTERS • Largest DVD rental company in the US • Oct 2006: Netflix Prize Award competition starts • Goal: Improve the recommendation performance of Netflix’s in-house developed Cinematch system by 10% • Prize: 1.000.000 US$ • The benefits of quality recommendations: • Increase customer life-time value – 1 months increase results in 50-60 million USD/year average extra income • Customers upgrade their plan – 5 USD/month/customer extra income USD 80 Million + additional revenue due to quality recommendation Gravity & The Ensemble Team has been able to improve Netflix recommendation system by 10.1% 20 NEW GENERATIONAL RECOMMENDATION ENGINES: RELEVANT RECOMMENDATION BASED ON THE ANALYSIS OF ALL SOURCES • Duration of Page Views • Order of Page Views • Clickstream • Searches • etc. • Geography • Visitors’s Past Searches • Past Shopping Behavior • Aggregated Past Behavior • etc. VISITOR BEHAVIOR PRODUCT DETAILS HISTORICAL DATA SESSION STATS • Product Viewed • Product in Catalog Location • Brand & Manufacturer • Descriptions • Ratings • etc. • Type of URL Page • Refer URL • Broadband Speed • IP Address • etc. 21 THE PERFORMANCE BENEFIT OF NEW GENERATIONAL RECOMMENDATION SYSTEMS *Based on Netflix competition results Even a slight change in recommendation accuracy results a significant increase in business potential! Increase in revenues +$80,000,000 Increase in accuracy +10% 22 BENEFITS User Service provider • Receives personalized services • Faster product search • Easier product and content discovery, „find” such products he/she wasn’t aware • Higher sales per visit • Increased time spent on site value • More non-best-selling (long tail) products sold • Strenghtened loyalty and stickyness • Decreased churn rate GRAVITY RECO Gravity Reco is a presonalization solution that continuously tracks and learns user behavior and recommends the most relevant items to the client at any given time. Gravity’s recommendations don’t just pull matches expected by anyone but our highly sophisticated algorithms dig deeper to delight the customer and to offer products that are more lucrative for the provider. Gravity Reco Key features: •Custom tailored to fit your exact needs •Real-time •No limitations on collecting data •Proactive & accurate •Fully customizable business & merchandising rules Benefits: •boosted sales •more cross- and up-selling •more returning customers •better conversion COMPTETITIVE ADVANTAGES OF GRAVITY Seamless learning from all available user actions, and information sources Tailored and personalized recommendations for each individual user real-time reconfiguration & adaptation to user action trends manages various item classes and provides cross-class recommendations (optimal for companies with various portfolios) recommendations are optimized to meet real customer’s goals (not optimized to hypothetic behavioral models) best-in-class mixed strategy based collaborative filtering based, and context aware recommendation techniques advanced reporting capabilities 25 CASE STUDY - VATERA.COM • Vatera.com – most dynamically growing auction portal in CEE region, member of Naspers Group (about 10+ auction and media sites in the portfolio) – 16 million page impressions of front page per week (in average) – 500.000+ actively listed items for bidding & sale • Key objectives of recommendation solution: – increasing the bid value of goods – easing the discovery and findings of goods Results of the pilot: •better banner conversion: 217+ % more clicks on the recommendation banner •better product discovery: 293+ % more bids •higher hammer price: 524+ % increase in bidding values 26 CASE STUDY - GROBY Groby.hu – the largest online grocery shop in Hungary • • • ~5000 item/day sold ~15000 different items available ~45000 registered users Key objective – personalize the shop • • • Personalize the customer experience during the browsing session Facilitate discovery of the usually bought items using recommendations Increase cart value The recommendation solution – provided by Gravity • • • Implemented during the re-design of the Groby website, hosted at Gravity HQ Displays 4-8 personalized recommendations on each page ~12000 recommended items/day displayed Results of the pilot: • • • Over 12% of the income is generated by the recommendation widgets Daily ~7% of the displayed recommendations is successfully converted into a purchase Daily ~50% of the customers purchase a recommended item 27 CASE STUDY - RANDIVONAL PILOT • Randivonal.hu – Hungary’s leading online dating portal – 1.000.000+ registered users (freemium model) – 800.000 daily pageviews • Pilot environment – All users are divided into two statistically significant and equal groups (A and B) – Group A receives Randivonal’s own recommendation, Group B receives Gravity’s one – Goal: improve the discovery, increase the conversion of “Personal tips” box – 200+ % improvement on conversion to control group 30000 25000 Weekly conversion on "Personal Tips" Gravity free account Randivonal free account Gravity premium account Randivonal premium account 20000 15000 10000 5000 0 28 CASE STUDY – DIGITAL MEDIA PROVIDER Our client has been the 26th most visited website of the world in January, 2010. It is a constant Alexa TOP 50 website. It is is an adult video chat site with thousands of chat hosts online in several categories. Key objectives of the pilot •Increase per show length •Increase number of performances •Increase conversion rate The recommendation solution – provided by Gravity •24 200 000 recommendations generated /day •System has been configured to generate up to 4000 recommendations/sec Results of the pilot: Income uplift generated by Gravity’s recommendation engine: +9% Registration rate increase: +5% 29 CASE STUDY - INTERACTIVE FOOTBALL PORTAL FOR VODAFONE Company: Beeweeb is an Italian software design company focused on convergent and mobile solutions. Beeweeb partners with leading technology and system integrators companies and handset manufacturers (RIM, Samsung, NokiaSiemensNetworks, Ericsson, PacketVideo, Vidiator…) to offer end-to-end solutions and to distribute its applications worldwide. Customer: Vodafone Italia Requirement: in collaboration with Beeweeb, Gravity developed the recommendation engine for a thematic portal (Vodafone calcio). Vodafone Italia wanted to increase customer engagement and provide a unique mobile experience with the personalization feature. Solution: Gravity’s recommendation engine provided personalized, football related content that is relevant to the user. It facilitates advanced content discovery of news, media and other content. Recommendations are pulled from different content types and are based on the personal preferences. 30 GRAVITARG CASE STUDY - TARGETED ADVERTISEMENTS • GraviTarg for E-commerce: Personalized Before After & targeted marketing campaings of Internet companies to drive sales – using user behavior actions and previous purchase information – provides targeted ad messages and – combines with personal offerings – seamlessly integrates into your ad display delivery platform and into 3rd party systems • Business benefits – Increase ad click-thru rates & conversions – Increase your sales and average order size – Maximize your media placements with personalized ads – Increase the ROI of your ad campaigns 31 CASE STUDY - VELTI PRODUCT DESCRIPTION Company: Velti is a leading global provider of mobile marketing and advertising technology. Requirement: Behaviour and taste based advertisement targeting Problem: Velti wanted to introduce an updated service, with increasing the average ad impression value. They chose Gravity as the partner for providing personalized advertisements. Solution: •Displays only relevant ads •Achieve same click-thru & conversion performance with less impression •Better monatization of ad display assets •Recommends other relevant ads using user behavior actions and previous purchase information on portfolio sites builds user preference and ad relevance profiles for targeting displays ads with highest user preference relevancy to drive successful ad conversions integrates into existing ad campaign management & reporting environment efficiently serves ad displays on pages with “long tail” contents ultra-scalable, high performance architecture, easy to integrate APIs 32 CASE STUDY – IPTV / CABLE TV Based on a real-life customer • 1M active subscribers • 400K active VOD users, purchasing 3,5M VODs per month • Revenue generated from VOD purchases: € 63,000,000 / year • Gross margin on VOD sales: 35% • Gravity IMPRESS (IPTV recommender) increased both the VOD customer base and the purchase activity, resulting in an additional direct revenue of €2,600,000+ / year Offered business models: a) Revenue share based on income uplift Click downloadof theactive productsubscribers brochure b) Volume based licensing, based onhere thetonumber 33 COOPERATION PROCESS Business aspect NDA 1 week Business goals & targets 2 weeks Agreement 2 weeks Pilot 1 months Launch 2 months Technical aspects Data gathering Data analysis Integration Fine-tuning Pilot 34 CASE STUDY – INTEGRATION TIMELINE • Business aspects – defining the scope of the pilot project (1 weeks) – deciding the business drivers, success factors, and key performance indicators (KPIs) (2 weeks) – integrating a recommendation solution into the customers system (2-3 weeks) – running the pilot and evaluating the KPIs (4-8 weeks based on the scope of the pilot) • Information requirements for the pilot – detailed technical description of partner’s system environment – detailed description of the use cases how the end user accessing the application – meta information about the end users involved in the pilot – if available 35 RISK FREE BUSINESS MODEL Gravity Reco – Software as a service: Success fee based on the sales increase generated by the Gravity engine / monthly fee – Appliance • One-time set up fee (covers system integration) • Monthly licensing fee based on performance • no costly upfront license fee • only recovery of integration cost • then monthly success fee that is proportional to the extra value generated 36 SUMMARY - VALUE PROPOSITION OF RECOMMENDATION ENGINES • Increased customer lifetime • Drive revenue , new users to premium services • Significant ARPU increase • Maximize user engagement, time spent on site, CTR • Recommendation system fine-tuned to your business’ exact needs • Customizable business rules • Earn incremental revenues from relevant and targeted advertisements … and And change the way your customers search… 37 38 Can you guarantee increase in your sales? Because Gravity can. For more information contact our business development team at sales@gravityrd.com 10 February 2009 39 Backup Slides 10 February 2009 08/04/2015 Content based filtering (CBF) Content based recommendations: recommendations based on metadata catalogues – uses EPG, VOD catalog metadata – weak personal taste / preference prediction power – only those items can be recommended that has a contextual similarity (e.g. titles are similar, actors are same, genre or category is same, etc.) • Solaris by Trakovski from 1972 and Solaris by Sorderbergh from 2002 would be linked incorrectly • Monster Inc and Shrek would likely not linked, because no overlapping metadata only the genre – does not follow trends - users watching behavior changes continuously, especially with live recommendations when habits changes easily (e.g. effect of reality shows / new series) – greatly depends on the quality of the metadata (metadata is hard to maintain and expensive) – popularity factors, ratings cannot be considered 40 Collaborative filtering (CF) Collaborative filtering based recommendations: recommendation based on usage activities – accurate precision technique – users’ actions do not lie, users’ preferences are naturally encoded in their actions – recommendations are done by clustering people into interest groups based on their consumption – items are recommended based on the consumption trends of the clusters – no metadata info is needed for recommendation – considers popularity factors, ratings, and accommodate to trends – the importance of VOD rental and live watching events: “Recommending New Movies: Even a Few Ratings Are More Valuable Than Metadata”, a scientific paper by Gravity chief data miners (can be downloaded from http://www.gravityrd.com/gravity/download/recsys2009pila_draft.pdf) 41 Recall precision of VOD • Recall precision of VOD tells the likeliness of VOD purchase from the recommended items 42 Scenario & Business Rule UI Demo 43 Scenario & Business Rule UI Demo 44 CUSTOMIZATION OF RECOMMENDATION • IMPRESS contains an advanced business rule & scenario editor – Complex templates can be composed from the following general building blocks: • • • • • • filter block union block recommender block emphasize block order block blender block – Internal A/B testing capability – Time-based recommendation capability – Version control & test before launch capability – Currently we provide scenario templates on operators request – Jan 2011: user-friendly UI 45 GRAVITARG EXAMPLE Normal ad serving approach • • • Display ads without relevance Wasted ad impressions Inefficient click-thru & conversion performance Ad impressions: 16, Relevant ads: 8, CTR: 3 GraviTarg approach • • Display only relevant ads Achieve same click-thru & conversion performance with fewer impressions • Better monatization of ad display assets • Recommends other relevant ads Ad impressions: 8, Relevant ads: 8, CTR: 3 Displayed ad: brown Relevant ad: Irrelevant ad: Clicked ad: 46 GRAVITARG - EASY INTEGRATION TO EXISTING AD MANAGERS AD request Targeted AD More value for the money 47