Personalized marketing using
recommendation systems
Peter A. Csikos
10 February 2009
SOME FACTS ABOUT THE HUNGARIAN E-COMMERCE
SPACE…
Although the total population is 10 million…
… the number of people actually doing
anything on the net is low
Population over 14
years or older:
8.515.000
Number of regular
internet users (14
years or older):
3.596.000
Number of people
actually doing ecommerce
1.075.000
Source: GKINET / Hungarian Association for e-commerce
Based on 2005
Microcensus
Number of people
actually using the
internet for at least one
hour per month (2010
Q1)
People who have bought
anything over the
internet in 2009
ONLINE SHOPPING AND E-COMMERCE BETWEEN 2001
AND 2010
Ratio of people actually doing e-commerce
and those who are planning to do so and
those who are even not planning to do it…
1.075.000
Not even planning to
Nem
tervezi
use e-commerce
Source: Hungarian Association of e-commerce
Planning to do e-
Tervezi
commerce in the
future
Vásárolt
Did e-commerce
2010.I.
2009.IV.
2009. III.
2009. II.
2008. IV.
2008. II.
2007. IV.
2007. III.
2007. II.
2007. I.
2006. IV.
2006. III.
2006. II.
2006. I.
2005. IV.
2005. III.
2005. II.
2005. I.
2004. IV.
2004. III.
2004. II.
2004. I.
2003. IV.
2003. III.
2003. II.
2003. I.
2002. IV.
2002. III.
2002. II.
2002. I.
2001. IV.
2001. III.
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
WHAT ARE THESE PEOPLE ACTUALLY DOING ON THE
INTERNET?
Number of people actually doing online
shopping last year
online banking
Online Banking
Doing administrative
things ügyintézés
Gathering information
vásárlás előtti tájékozódás
before regular shopping
General
Information
általános
információk
0%
20%
40%
Gyakran
veszi igénybe
Regularly
© GKINET / Association of e-commerce
60%
Igénybe veszi
sometimes
80%
100%
TURNOVER OF ONLINE SHOPS IN HUNGARY BETWEEN
2001 AND 2010
EUR 500million
EUR 350 million
Planned
© GKINET / Association of e-commerce
Actual
Key players on the Hungarian ecommerce market
Sanoma
CEMP
Index
Velvet
Totalcar
Inda
Portfólió
Etarget
InfoRádió
BOOKLINE
DVD Rent
C-TRAVEL
OnGo
Honfoglaló
Mobile24
Burek
ArtOffice
[origo]
Origo
Freemail
IWIW
[ORIGO] TÉKA
Videa
Megálla-podások
más szolgáltatókkal
ORIGOVÁSÁRLÁS
Startlap
Hotdog
PROFESSION
JOBMONITOR
Vezess.hu
Figyelőnet
Citromail
Videoplayer
STARTAPRO
Mobilport
Wellnesscafe
Hírstart
Nők Lapja Café
Geographic
Cosmopolitan
Storyonline
HÁZIPATIKA.COM
csoport
TECHNET
SPEEDSHOP.HU
Holtzbrinck Networks
Randivonal.hu
Brands.hu
Habostorta.hu
Funpic.hu
Gyaloglo.hu
Gumicsizma
Allegro Hungary
Izé.hu
Lira.hu
Metnet.hu
NB1.hu
Minimax
Vatera.hu
TeszVesz.hu
Képzésinfo.hu
KutatóCentrum
Matton Images
Maxima
Monddmeg.hu
Terminal.hu
Netpincer.hu
Sportolj.hu
Extreme
Digital
EXTREMEDIGITAL.HU
Klikkmánia.hu
Nethajó.hu
Ebolt Kft.
EBOLT.HU
Lapcom
HASZNALTAUTO.HU
Garanciális.hu
SZALONAUTÓ.HU
BAZÁR.HU
AUTÓBAZAR.HU
GRoby
Groby.hu
Jobline
Ingatlamenedzser
Autómenedzser
Adózóna
Ecoline
Techline
Medizóna
Eduline
URLguru
Travelline
Aprónet
Life & Style
Zöldtér
Hírszerző
General Media
Arukereso.hu
Otomoto.hu
Viala Kft.
Hírek Média
DEPO.HU
Egyperces.hu
Terminál
CTnetwork
Hírek.hu
JOBINFO.HU
HVG (WAZ)
INGATLANBAZAR.HU
Workania.hu
Delmagyar.hu
Kisalfold.hu
Hahu.hu
MyVIP.hu
MYCLUB.HU
Donna.hu
DonnaLight
myMAP.hu
SportHÍRADÓ
Olimpia.hu
Nascar.hu
Data.hu
INGATLANOK.HU
Hirposta.hu
Love.hu
TALALKA.HU
JOJATEK.HU
MultiPlay
Chat.hu
Mellesleg.hu
Farm.Hu
Postafiok.hu
Fashion.hu
Divat.hu
BOOK.HU
Freeweb.hu
Arkon Zrt.
Ingatlan.com (B2B) Koponyeg.hu
Utcakereso.hu
Alaprajz.hu
Linkcenter.hu
Blogol.hu
Telepiuleskereso.hu
TYPICAL HUNGARIAN PROBLEM FOR E-COMMERCE STARTUPS:
GETTING THE DEVELOPERS CLOSE TO THE INDUSTRY
Focus, Focus, Focus
Differences in motivation
freedom
of development
fame;
IT cups,
contests
Market share,
market size
IT excellence
Publication,
open source
© Norbert, Buzas, ValDeal / University of Szeged
New Products
ROI, NPV, IRR
official
regulations
Business Card – Gravity R&D
• Domain: Personalized Recommendation Solutions for ECommerce, Digital Media and IPTV/Digital Cable Service
Providers
• Our offering: Delivering real time, personalized context-based
product recommendations to each and every user, the Gravity
solution produces measurable increase in revenues from
existing customer bases and improves customer satisfaction
• A fast-growing company: significant number of customers
and several pilot projects running including ALEXA top35
video live streaming site, 5 to 30 employees within 12 months
• Investors: US and Hungarian investors, offices in Budapest,
London, Utrecht and Berlin, sales representatives in Romania,
Russia and Israel and China
8
E-commerce market trends
9
The problem
10
Market trends
• Personal product recommendations are one of the TOP5 most important emerging
technology priorities in e-commerce (Internet Retailer, „Emerging Technology”
conducted by Vovici Corp, Sept 2008)
• 50% of online retail stores want to introduce personal recommendations (Internet
Retailer, „Emerging Technology” conducted by Vovici Corp, Sept 2008)
• 40% of US e-commerce sites plan to adopt customer reviews and ratings (The
Kelsey Group and ConStat Inc, „Local commerce monitor”, Aug 2008)
• 77% of retail executives said that online behavior tracking is the most promising
website technology for online customer engagement (Retail Systems Research,
„Playing Well with Others: eCommerce’s Evolving Role in the Customer
Experience”, Aug 2008)
• 76% of US e-commerce executives said the personalized recommendation may or
definitely increase loyalty (The Executive Guide to Captivating Customers, June
2008)
• Social shopping sites and collaborative technologies are deemed currently as
leading shopping innovations and expected to be widespread by 2015 (US
household shoppers prediction) (TNS Retail Forward, „New Future in Store”, May
2008)
11
THE EMERGENCE OF RECOMMENDATIONS
Internet-based shopping and media consumption differs from the
classical one
In-store, personal shopping experience
 Customers know how to find the desired products
 Assistances help the customer: show them the desired products &
recommend
Internet shopping experience
 Web portals offers much more products, customers get lost easily
 Recommendation systems helps the customer to find & discover products
The key is to keep the customers active & increase conversion
 20% of the customers is accountable for 80% of the profits
 To gain new customers, companies have to spend 3-4X more than to keep
the existing ones
Solution: Change the way you personalize
13
WHAT IS A RECOMMENDATION ENGINE?
Recommender systems or recommendation
engines form a specific type of information
filtering system technique that attempts to
present information items (films, television,
video on demand, music, books, news, images,
web pages, etc.) that are likely of interest to
the user.
14
HOW DOES IT WORK?
User registers or
starts using
services
User browses,
watches and
optionally rates
content
System builds a
preference profile
System provides
presonalized
recommendations
15
WHAT IT DOES?
Recommender
logic
•
•
•
•
Data collection and processing
Relevance & preference ordering
Display recommendations
Self-learning & improving
capabilities
•
•
Mathematical models
Information systematization
16
THE RECOMMENDATIONS
Customer is looking for a product
Receive tips:
Receive
personal
offerings:
17
SHORT SCIENCE: RECOMMENDATION
ALGORITHMS
Recommendation in general:
 Gravity is using a wide palette of recommendation algorithms
 The best fitting algorithms are selected – after careful analysis of the data – to
the given recommendation problem and the corresponding optimization task.
Overview of recommendation algorithms:
1. Collaborative filtering (CF):
• Based on events generated in your service (Vod purchase, Live channel watching event), finds
similar behavior on users, and similarity on items (VoD content, live schedule, etc.)
2. Content based-filtering (CBF):
• Using only user/item metadata. Recommendations are based on matching keywords.
Measuring Recommendation Quality:
• Average Relative Position (ARP): The distance between the prediction and the user’s choice
• Top 10 Recall: the probability of hitting the chosen item from the top 10 items of the
personalized list
18
Early generation recommendation solutions…
… Did not offer really personalized recommendations for each and every user…
• Not personalized
• Only based on part of
the available information
• Low customer retention
(if any)
•Minimal revenue
increase
• Lower conversion
rate
• Increase of customer
satisfaction is
questionnable
3
THE NETFLIX COMPETITION –
RECOMMENDATION QUALITY MATTERS
• Largest DVD rental company in the US
• Oct 2006: Netflix Prize Award competition starts
• Goal: Improve the recommendation performance of Netflix’s in-house developed
Cinematch system by 10%
• Prize: 1.000.000 US$
• The benefits of quality recommendations:
• Increase customer life-time value – 1 months increase results in 50-60 million
USD/year average extra income
• Customers upgrade their plan – 5 USD/month/customer extra income
USD 80 Million + additional revenue due to quality recommendation
Gravity & The Ensemble Team has been able to improve Netflix
recommendation system by 10.1%
20
NEW GENERATIONAL RECOMMENDATION ENGINES: RELEVANT
RECOMMENDATION BASED ON THE ANALYSIS OF ALL SOURCES
• Duration of Page
Views
• Order of Page Views
• Clickstream
• Searches
• etc.
• Geography
• Visitors’s Past Searches
• Past Shopping
Behavior
• Aggregated Past
Behavior
• etc.
VISITOR
BEHAVIOR
PRODUCT
DETAILS
HISTORICAL
DATA
SESSION
STATS
• Product Viewed
• Product in Catalog
Location
• Brand & Manufacturer
• Descriptions
• Ratings
• etc.
• Type of URL Page
• Refer URL
• Broadband Speed
• IP Address
• etc.
21
THE PERFORMANCE BENEFIT OF NEW
GENERATIONAL RECOMMENDATION SYSTEMS
*Based on Netflix competition results
Even a slight change in
recommendation
accuracy results a
significant increase in
business potential!
Increase in revenues
+$80,000,000
Increase in accuracy +10%
22
BENEFITS
User
Service provider
• Receives personalized
services
• Faster product search
• Easier product and content
discovery, „find” such
products he/she wasn’t
aware
• Higher sales per visit
• Increased time spent on
site value
• More non-best-selling (long
tail) products sold
• Strenghtened loyalty and
stickyness
• Decreased churn rate
GRAVITY RECO
Gravity Reco is a presonalization solution that continuously tracks and learns user
behavior and recommends the most relevant items to the client at any given time.
Gravity’s recommendations don’t just pull matches expected by anyone but our
highly sophisticated algorithms dig deeper to delight the customer and to offer
products that are more lucrative for the provider.
Gravity Reco
Key features:
•Custom tailored to fit your exact needs
•Real-time
•No limitations on collecting data
•Proactive & accurate
•Fully customizable business & merchandising rules
Benefits:
•boosted sales
•more cross- and up-selling
•more returning customers
•better conversion
COMPTETITIVE ADVANTAGES OF GRAVITY
Seamless learning from all available user actions,
and information sources
Tailored and personalized recommendations for
each individual user
real-time reconfiguration & adaptation to user
action trends
manages various item classes and provides
cross-class recommendations (optimal for
companies with various portfolios)
recommendations are optimized to meet real
customer’s goals (not optimized to hypothetic
behavioral models)
best-in-class mixed strategy based collaborative
filtering based, and context aware
recommendation techniques
advanced reporting capabilities
25
CASE STUDY - VATERA.COM
• Vatera.com – most dynamically growing auction portal in CEE region, member of
Naspers Group (about 10+ auction and media sites in the portfolio)
– 16 million page impressions of front page per week (in average)
– 500.000+ actively listed items for bidding & sale
• Key objectives of recommendation solution:
– increasing the bid value of goods
– easing the discovery and findings of goods
Results of the pilot:
•better banner conversion: 217+ % more clicks on the recommendation banner
•better product discovery: 293+ % more bids
•higher hammer price: 524+ % increase in bidding values
26
CASE STUDY - GROBY
Groby.hu – the largest online grocery shop in Hungary
•
•
•
~5000 item/day sold
~15000 different items available
~45000 registered users
Key objective – personalize the shop
•
•
•
Personalize the customer experience during the browsing session
Facilitate discovery of the usually bought items using recommendations
Increase cart value
The recommendation solution – provided by Gravity
•
•
•
Implemented during the re-design of the Groby website, hosted at Gravity HQ
Displays 4-8 personalized recommendations on each page
~12000 recommended items/day displayed
Results of the pilot:
•
•
•
Over 12% of the income is generated by the recommendation widgets
Daily ~7% of the displayed recommendations is successfully converted into a purchase
Daily ~50% of the customers purchase a recommended item
27
CASE STUDY - RANDIVONAL PILOT
• Randivonal.hu – Hungary’s leading online dating portal
– 1.000.000+ registered users (freemium model)
– 800.000 daily pageviews
• Pilot environment
– All users are divided into two statistically significant and equal groups (A and B)
– Group A receives Randivonal’s own recommendation, Group B receives Gravity’s one
– Goal: improve the discovery, increase the conversion of “Personal tips” box
– 200+ % improvement on conversion to control group
30000
25000
Weekly conversion on "Personal Tips"
Gravity free account
Randivonal free account
Gravity premium account
Randivonal premium account
20000
15000
10000
5000
0
28
CASE STUDY – DIGITAL MEDIA PROVIDER
Our client has been the 26th most visited website of the world in January, 2010. It is a
constant Alexa TOP 50 website. It is is an adult video chat site with thousands of chat hosts
online in several categories.
Key objectives of the pilot
•Increase per show length
•Increase number of performances
•Increase conversion rate
The recommendation solution – provided by Gravity
•24 200 000 recommendations generated /day
•System has been configured to generate up to 4000 recommendations/sec
Results of the pilot:
Income uplift generated by Gravity’s recommendation engine: +9%
Registration rate increase: +5%
29
CASE STUDY - INTERACTIVE FOOTBALL PORTAL FOR
VODAFONE
Company: Beeweeb is an Italian software design company focused on convergent
and mobile solutions. Beeweeb partners with leading technology and system
integrators companies and handset manufacturers (RIM, Samsung,
NokiaSiemensNetworks, Ericsson, PacketVideo, Vidiator…) to offer end-to-end
solutions and to distribute its applications worldwide.
Customer: Vodafone Italia
Requirement: in collaboration with Beeweeb, Gravity developed the
recommendation engine for a thematic portal (Vodafone calcio). Vodafone Italia
wanted to increase customer engagement and provide a unique mobile
experience with the personalization feature.
Solution: Gravity’s recommendation engine provided personalized, football
related content that is relevant to the user. It facilitates advanced content
discovery of news, media and other content. Recommendations are pulled from
different content types and are based on the personal preferences.
30
GRAVITARG CASE STUDY - TARGETED
ADVERTISEMENTS
• GraviTarg for E-commerce: Personalized
Before
After
& targeted marketing campaings of Internet
companies to drive sales
– using user behavior actions and previous
purchase information
– provides targeted ad messages and
– combines with personal offerings
– seamlessly integrates into your ad display
delivery platform and into 3rd party systems
• Business benefits
– Increase ad click-thru rates & conversions
– Increase your sales and average order size
– Maximize your media placements with
personalized ads
– Increase the ROI of your ad campaigns
31
CASE STUDY - VELTI PRODUCT DESCRIPTION
Company: Velti is a leading global provider of mobile marketing and
advertising technology.
Requirement: Behaviour and taste based advertisement targeting
Problem: Velti wanted to introduce an updated service, with increasing
the average ad impression value. They chose Gravity as the partner for
providing personalized advertisements.
Solution:
•Displays only relevant ads
•Achieve same click-thru & conversion
performance with less impression
•Better monatization of ad display assets
•Recommends other relevant ads
using user behavior actions and previous
purchase information on portfolio sites
builds user preference and ad relevance profiles
for targeting
displays ads with highest user preference
relevancy to drive successful ad conversions
integrates into existing ad campaign
management & reporting environment
efficiently serves ad displays on pages with “long
tail” contents
ultra-scalable, high performance architecture,
easy to integrate APIs
32
CASE STUDY – IPTV / CABLE TV
Based on a real-life customer
• 1M active subscribers
• 400K active VOD users, purchasing 3,5M VODs per month
• Revenue generated from VOD purchases: € 63,000,000 / year
• Gross margin on VOD sales: 35%
• Gravity IMPRESS (IPTV recommender) increased both the VOD customer base and
the purchase activity, resulting in an additional direct revenue of €2,600,000+ /
year
Offered business models:
a) Revenue share based on income uplift
Click
downloadof
theactive
productsubscribers
brochure
b) Volume based licensing, based
onhere
thetonumber
33
COOPERATION PROCESS
Business aspect
NDA
1 week
Business
goals &
targets
2 weeks
Agreement
2 weeks
Pilot
1 months
Launch
2 months
Technical aspects
Data
gathering
Data
analysis
Integration
Fine-tuning
Pilot
34
CASE STUDY – INTEGRATION TIMELINE
• Business aspects
– defining the scope of the pilot project (1 weeks)
– deciding the business drivers, success factors, and key performance indicators (KPIs)
(2 weeks)
– integrating a recommendation solution into the customers system (2-3 weeks)
– running the pilot and evaluating the KPIs (4-8 weeks based on the scope of the pilot)
• Information requirements for the pilot
– detailed technical description of partner’s system environment
– detailed description of the use cases how the end user accessing the application
– meta information about the end users involved in the pilot – if available
35
RISK FREE BUSINESS MODEL
Gravity Reco
– Software as a service: Success fee based on the sales increase
generated by the Gravity engine / monthly fee
– Appliance
• One-time set up fee (covers system integration)
• Monthly licensing fee based on performance
• no costly upfront license fee
• only recovery of integration cost
• then monthly success fee that is proportional to the extra value generated
36
SUMMARY - VALUE PROPOSITION OF
RECOMMENDATION ENGINES
• Increased customer lifetime
• Drive revenue , new users to premium services
• Significant ARPU increase
• Maximize user engagement, time spent on site, CTR
• Recommendation system fine-tuned to your business’ exact needs
• Customizable business rules
• Earn incremental revenues from relevant and targeted advertisements … and
And change the way your customers search…
37
38
Can you guarantee increase in
your sales?
Because Gravity can.
For more information contact our business development team at
sales@gravityrd.com
10 February 2009
39
Backup Slides
10 February 2009
08/04/2015
Content based filtering (CBF)
Content based recommendations: recommendations based on metadata catalogues
– uses EPG, VOD catalog metadata
– weak personal taste / preference prediction power
– only those items can be recommended that has a contextual similarity (e.g. titles are
similar, actors are same, genre or category is same, etc.)
• Solaris by Trakovski from 1972 and Solaris by Sorderbergh from 2002 would be
linked incorrectly
• Monster Inc and Shrek would likely not linked, because no overlapping metadata
only the genre
– does not follow trends - users watching behavior changes continuously, especially
with live recommendations when habits changes easily (e.g. effect of reality shows /
new series)
– greatly depends on the quality of the metadata (metadata is hard to maintain and
expensive)
– popularity factors, ratings cannot be considered
40
Collaborative filtering (CF)
Collaborative filtering based recommendations: recommendation based on usage activities
– accurate precision technique
– users’ actions do not lie, users’ preferences are naturally encoded in their actions
– recommendations are done by clustering people into interest groups based on their
consumption
– items are recommended based on the consumption trends of the clusters
– no metadata info is needed for recommendation
– considers popularity factors, ratings, and accommodate to trends
– the importance of VOD rental and live watching events: “Recommending New
Movies: Even a Few Ratings Are More Valuable Than Metadata”, a scientific paper by
Gravity chief data miners (can be downloaded from
http://www.gravityrd.com/gravity/download/recsys2009pila_draft.pdf)
41
Recall precision of VOD
•
Recall precision of VOD tells the likeliness of VOD purchase from the recommended
items
42
Scenario & Business Rule UI Demo
43
Scenario & Business Rule UI Demo
44
CUSTOMIZATION OF
RECOMMENDATION
•
IMPRESS contains an advanced business rule & scenario editor
– Complex templates can be composed from the following general building blocks:
•
•
•
•
•
•
filter block
union block
recommender block
emphasize block
order block
blender block
– Internal A/B testing capability
– Time-based recommendation capability
– Version control & test before launch capability
– Currently we provide scenario templates
on operators request
– Jan 2011: user-friendly UI
45
GRAVITARG EXAMPLE
Normal ad serving approach
•
•
•
Display ads without relevance
Wasted ad impressions
Inefficient click-thru & conversion
performance
Ad impressions: 16, Relevant ads: 8, CTR: 3
GraviTarg approach
•
•
Display only relevant ads
Achieve same click-thru & conversion
performance with fewer impressions
• Better monatization of ad display assets
• Recommends other relevant ads
Ad impressions: 8, Relevant ads: 8, CTR: 3
Displayed ad: brown
Relevant ad:
Irrelevant ad:
Clicked ad:
46
GRAVITARG - EASY INTEGRATION TO EXISTING
AD MANAGERS
AD request
Targeted AD
More value for the
money
47