Efficient Behavior Targeting Using SVM
Ensemble Indexing
Jun Li, Peng Zhang, Yanan Cao, Ping Liu, Li Guo
Chinese Academy of Sciences
State Grid Energy Institute, China
Behavior targeting

Behavior Targeting (BT) uses users’ historical behavior data to
select the most relevant ads for display.
ads

User behavior data
Example from Yahoo! Research
Targeted users
Regression for BT

Poisson Regression model (Ye Chen, eBay, 2009).
 x:
ad clicks and views, page views, search queries and
clicks.
 y: click-through rate (CTR).
Poisson dis.
Poisson reg.
on view
View
data
ad
category
Click
data
Poisson dis.
Poisson reg.
on click
Ye Chen et al., Large-scale behavior targeting (KDD’09 best paper award)
Limitations

Limitations:
 parameter tuning is very difficult.
 the Poisson assumption is not always true for real-world behavior data.
 Clicks are typically several orders of magnitude fewer than views.
 User interests are not always fixed, but rather transient.
Classification for BT

SVM for classification
View
data
ad
category
Click
data
View and
click data(+)
View but no
click data(-)
SVM for
classification
Challenges 1,2,3
 Example
1: 3 users on Nikon (www.nikon.com)’s ad a
Classification for BT

Ensemble SVM on data streams

Merits
 no
complicated parameters
 no statistical assumptions
 Dynamic model on data streams
Challenge 4
Limitations

Time cost is heavy for online computing

ensemble prediction
 time
cost: A (advertisers)*W(ensemble size)*N(support
vectors)*T(features)
Example 2: We collect
2 million behavior
events (W = 10) in 1
minute, and prediction
result costs 53 minutes.
Solutions

Construct Index structure for Ensemble SVM.
Features
map
Text terms
Support vector
Document
Ensemble SVM
Document set
Why the index work ?
 Trade space for time.
 shared features among multiple support vectors
Zhang et al., of
knowledge
index
for online data streams
 the sparseP. structure
support
vectors
( KDD 2011 & ICDM 2011)
The index structure

The SVM-index structure
 Example 3: based on example 1, consider a SVM
with 3 support vectors
Inverted
hashing table
Support vectors
Ensemble
information
Time complexity O(T)
The index structure

Operations
–
Search: Predict the label of each incoming user data x,
• Step 1: searches support vectors in the left inverted indexes
• Step 2: calculate x’s class label
See our source
codes.
–
–

Insert: Integrate new classifiers into ensemble
Delete: Drop outdated classifiers from ensemble
Memory
Experiments

Data sets
 Search
•
engine data
Comparisons
–
–
–
Possion
E-SVM
E-Index (our method)
Comparisons

Observations
E-index has sub-linear
prediction time
E-SVM consumes more
memory
Comparisons
Ensemble models are more accurate than Poisson regression model
Comparisons
The index method can significantly improve the efficiency, especially
when the ensemble size is large.
Related Work



Behavior targeting
 Regression models vs. classification models
Stream indexing
 Boolean expression indexing in Publish/subscribe systems
Ensemble models
 Concept drifting
Conclusions


Contributions
 Identify and address the prediction efficiency problem for
ensemble models for behavior targeting.
 Convert ensemble SVM model to a document set, and
propose a new type of invert text index structure to achieve
sub-linear prediction time.
Future work
 Index more complicated SVM models with non-linear kernels.
Questions?
For source code, visit our website
streamming.org/homepages/lijun.html