Fan Guo
Chao Liu
Carnegie Mellon University
Microsoft Research-Redmond

Search Results for “CIKM”
# of clicks received
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
2

Adapt ranking to user clicks?
# of clicks received
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
3

Tools needed for non-trivial cases
# of clicks received
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
4


One of the most extensive (yet indirect) surveys
of user experience.
For researchers:
 Help understand human interaction with IR results
 Design and calibrate novel models and hypotheses

For practitioners:
 Measure, monitor and improve search engine
performance.
 Attract more page views and clicks, boost profit
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
5

Introduce problems and applications in web
search click modeling.

Present latest development of click models in
web search.

Provide examples and discuss trade-offs for
model design, implementation and evaluation.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
6
Ph.D. Student (exp. 2011),
Computer Science Department,
Carnegie Mellon University
 Advisor: Christos Faloutsos
 Dissertation topic: graph mining
for large bioinformatics image
databases
 2008, M.S., CMU
 2005, B.E., Tsinghua University,
Beijing, China

3/22/2016
CIKM'09 Tutorial, Hong Kong, China
7
Researcher, Internet Services
Research Center (ISRC), MSRRedmond.
 Research focus: large-scale
search/browsing log analysis for
effective Web information access.
 2007, Ph.D., UIUC
2005, M.S., UIUC

 Advisor: Jiawei Han
 Dissertation on statistical debugging
and automated failure analysis

3/22/2016
2003, B.S., Peking University, China
CIKM'09 Tutorial, Hong Kong, China
8





Introduction
Designing click models
Bayesian click models
Selected topics on click models
Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
9

Introduction
 Web search click logs
 Interpret clicks as relevance feedback
 Building statistical models for clicks
 Applications of click models




Designing click models
Bayesian click models
Selected topics on click models
Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
10

Click-through

Browser action

Dwelling time

Explicit judgment

Other page elements
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
11

Auto-generated data keeping important
information about search activity.
Query
Position
cikm
Session ID
URL
f851c5af178384d12f3d
Click
1
cikm2008.org
1
2
www.cikm.org
0
3
www.cikm.org/2002
0
4
www.fc.ul.pt/cikm2007
0
5
www.comp.polyu.edu.hk/conference/cikm2009
1
6
cikmconference.org
0
7
Ir.iit.edu/cikm2004
0
8
www.informatik.uni-trier.de/~ley/db/conf/cikm/index.html
0
9
www.tzi.de/CIKM2005
0
10
www.cikm.com
0
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
12

A real world example
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
13

How large is the click log?

search logs: 10+ TB/day
 In existing publications:
▪ [Craswell+08]: 108k sessions
▪ [Dupret+08] : 4.5M sessions (21 subsets * 216k sessions)
▪ [Guo +09a] : 8.8M sessions from 110k unique queries
▪ [Guo+09b]: 8.8M sessions from 110k unique queries
▪ [Chapelle+09]: 58M sessions from 682k unique queries
▪ [Liu+09a]: 0.26PB data from 103M unique queries
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
14

How large is one
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
?
15

Introduction
 Web search click logs
 Interpret clicks as relevance feedback
 Building statistical models for clicks
 Applications of click models




Designing click models
Bayesian click models
Selected topics on click models
Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
16

Clicks are good…
 Are these two clicks
equally “good”?

Non-clicks may have
excuses:
 Not relevant
 Not examined
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
17
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
18
Higher positions receive
more user attention
(eye fixation) and clicks
than lower positions.

This is true even in the
extreme setting where
the order of positions is
reversed.

“Clicks are informative
but biased”.
Percentage

Percentage
Normal Position
[Joachims+07]
Reversed Impression
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
19

“Clicked > Skipped Above” [Joachims02]
1
2
3
4
5

Preference pairs:
#5>#2, #5>#3, #5>#4.

Use Rank SVM to optimize
the retrieval function.

Limitation:
6
7
 Confidence of judgments
 Little implication to user
modeling
8
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
20

Introduction
 Web search click logs
 Interpret clicks as relevance feedback
 Building statistical models for clicks
 Applications of click models




Designing click models
Bayesian click models
Selected topics on click models
Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
21

Given a set of web search click logs:
 Predict clicks: output the probability
of click vectors given a new order of
URLs.
210
possibilities!
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
22

Given a set of web search click logs:
 Estimate relevance: measures how
good a URL is with regard to the
information need of the query/user.
Relevance score = 0.5
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
23

The probability of a click if the document
appears at the top position.
 Relevance score = 0.5 indicates that on average,
Density function
the document will be clicked once per 2 sessions.
 Bayesian click models characterize relevance
using a probability distribution
3/22/2016
Relevance score
CIKM'09 Tutorial, Hong Kong, China
24

Effective: aware of the position-bias and
address it properly

Scalable: linear complexity for both time and
space, easy to parallel

Incremental: flexible for model update based
on new data
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
25

Introduction
 Web search click logs
 Interpret clicks as relevance feedback
 Building statistical models for clicks
 Applications of click models




Designing click models
Bayesian click models
Selected topics on click models
Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
26

Optimizing the retrieval function
 Ranking alternation based on clicks [Liu+09b]
0.72
0.20
0.05
0.08
0.90
0.10
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
27

Optimizing the retrieval function
 Ranking alternation based on clicks
 As a feature to a learning-to-rank system
(e.g., RankNet [Burges+05] )
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
28

Online advertising
 User model for sponsored search auctions
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
29

Online advertising
 User model for sponsored search auctions
 Click through rate (CTR) prediction [Zhu+10]
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
30

Search engine evaluation
 Pskip [Wang+09]:
click-through-rate above last clicks;
dwelling time features could also be incorporated.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
31

Search engine evaluation
 Pskip [Wang+09]: click-through-rate above last clicks;
 Search relevance score [Guo+09c]:
average relevance score weighted by chance of examination
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
32

User behavior analysis
 A preliminary work showing different user
behavior patterns for navigational and
informational queries [Guo+09c]
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
33

Introduction

Designing click models
 Basic user hypotheses
 Modeling the first click
 Extending to multiple clicks
 Summary of model design
 Bayesian click models
 Selected topics on click models
 Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
34

A document must be examined before a click.

The (conditional) probability of click upon
examination depends on document relevance.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
35

The click probability could be decomposed:
 Global component: the examination probability
which reflects the position-bias
 Local component: depends on the (query, URL)
pair only
 The building block for every existing model!
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
36

The first document is always examined.

First-order Markov property:
 Examination at position (i+1) depends on
examination and click at position i only

Examination follows a strict linear order:
Position i
3/22/2016
Position (i+1)
CIKM'09 Tutorial, Hong Kong, China
37

The first document is always examined.

First-order Markov property:
 Examination at position (i+1) depends on
examination and click at position i only

Examination follows a strict linear order:
Position i
3/22/2016
Position (i+1)
CIKM'09 Tutorial, Hong Kong, China
38

Limitation: examination/click rate monotonically
decreases with rank, which is not always true.
Web search data in [Guo+09a]

Ads click data in [Zhu+10]
Some models do not follow this hypothesis (e.g., UBM)
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
39

Introduction

Designing click models
 Basic user hypotheses
 Modeling the first click
 Extending to multiple clicks
 Summary of model design
 Bayesian click models
 Selected topics on click models
 Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
40

Put together two hypotheses:
Cascade Model =
[Craswell+08]

Formal model specification:
 P(Ci=1|Ei=0) = 0, P(Ci=1|Ei=1) = rui
examination hypothesis
 P(E1=1) =1, P(Ei+1=1|Ei=0) = 0
cascade hypothesis
 P(Ei+1=1|Ei=1, Ci=0)=1
modeling a single click
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
41

The user behavior chart:
Examine the
URL
Click?
No
See Next
URL?
Yes
Yes
Done
3/22/2016
Index for URL at position i
CIKM'09 Tutorial, Hong Kong, China
42

First click in Click Chain Model [Guo+09b] as well as
Dynamic Bayesian Network model [Chapelle+09]
Examine the
URL
Click?
No
Yes
No
Done
3/22/2016
See Next
URL?
Done
CIKM'09 Tutorial, Hong Kong, China
Yes
The chance that user
may immediately
abandon examination
w/o a click.
43

First click in User Browsing Model [Dupret+08]
Examine the
URL
Click?
Yes
Done
3/22/2016
No
See Next
URL?
No
i ←i+1
CIKM'09 Tutorial, Hong Kong, China
Yes
Position-dependent
parameters
44

Introduction

Designing click models
 Basic user hypotheses
 Modeling the first click
 Extending to multiple clicks
 Summary of model design
 Bayesian click models
 Selected topics on click models
 Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
45

Generalize the cascade model to 1+ clicks:
 P(Ci=1|Ei=0) = 0, P(Ci=1|Ei=1) = rui
 P(E1=1) =1, P(Ei+1=1|Ei=0) = 0
 P(Ei+1=1|Ei=1, Ci=0)=1
 P(Ei+1=1|Ei=1, Ci=1)= λi
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
λ:global parameters characterizing
user browsing behavior
46

Generalize the cascade model to 1+ clicks:
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
47

DCM Algorithms:
 Input: for each query session, the query term, with
(URL, clicked) tuple for all top-10 positions.
 Output: relevance for each (query, URL) pair;
global parameters for user behavior
 Method: approximate* maximum-likelihood
estimation.
3/22/2016 the algorithmCIKM'09
Tutorial, Hong
Kong, China
*Footnote:
maximizes
a lower
bound of log-likelihood function.
48
Query
Session ID
Position
Last
clicked
position
3/22/2016
1
2
3
4
5
6
7
8
9
10
cikm
f851c5af178384d12f3d
URL
Click
cikm2008.org
www.cikm.org
www.cikm.org/2002
www.fc.ul.pt/cikm2007
www.comp.polyu.edu.hk/...
cikmconference.org
Ir.iit.edu/cikm2004
www.informatik.uni-trier.de...
www.tzi.de/CIKM2005
www.cikm.com
1
0
0
0
1
0
0
0
0
0
CIKM'09 Tutorial, Hong Kong, China
49
Query
Session ID
Position
Last
clicked
position
3/22/2016
1
2
3
4
5
6
7
8
9
10
cikm
ab8dee4c4dd21e6aaf03
URL
Click
cikm2008.org
www.cikm.org
www.cikm.org/2002
www.fc.ul.pt/cikm2007
cikmconference.org
www.comp.polyu.edu.hk/...
Ir.iit.edu/cikm2004
www.informatik.uni-trier.de...
www.tzi.de/CIKM2005
www.cikm.com
0
1
0
0
0
1
0
0
1
0
CIKM'09 Tutorial, Hong Kong, China
50

The estimation formula for relevance:
empirical CTR measured before last clicked position

The estimation formula for global (user
behavior) parameters:
empirical probability of “clicked-but-not-last”
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
51
Details

Keep 3 counts for each (query, URL) pair

Then
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
52

The examine-next probability depends on the
relevance of the URL clicked:
Not what I want,
go to examine
the next
Aha, this is the
right one, and I’m
done!
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
53

The examine-next probability depends on the
relevance of the URL clicked:
 P(Ei+1=1|Ei=1, Ci=1)= α2(1-rui) + α3rui
 P(Ei+1=1|Ei=1, Ci=0)= α1
where 0 < α1 ≤ 1, 0 ≤ α3< α2≤ 1
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
54

The full picture:
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
55

There is a subtle difference between the relevance
of the URL snippet and the landing page.
hmmm…, this
looks pretty nice
errr…, it’s way
out of date
Conclusion: attractive, but not satisfactory.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
56

The examine-next probability depends on the
“satisfaction score”:
 P(Ei+1=1|Ei=1, Ci=1)= γ(1-sui) + 0sui
 P(Ei+1=1|Ei=1, Ci=0)= γ
where 0 < γ ≤1

The click probability is associated with
“attractiveness score”:
 P(Ci=1|Ei=1)= aui
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
57

The full picture:
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
58

The examine-next probability depends on
both the preceding clicked position r, and the
distance to this position d.
r=0
d=1
3/22/2016
Position
URL
Click
1
2
3
4
5
6
…
cikm2008.org
www.cikm.org
www.cikm.org/2002
www.fc.ul.pt/cikm2007
cikmconference.org
www.comp.polyu.edu.hk/...
…
0
1
0
0
0
1
…
CIKM'09 Tutorial, Hong Kong, China
59

The examine-next probability depends on
both the preceding clicked position r, and the
distance to this position d.
r=0
d=2
3/22/2016
Position
URL
Click
1
2
3
4
5
6
…
cikm2008.org
www.cikm.org
www.cikm.org/2002
www.fc.ul.pt/cikm2007
cikmconference.org
www.comp.polyu.edu.hk/...
…
0
1
0
0
0
1
…
CIKM'09 Tutorial, Hong Kong, China
60

The examine-next probability depends on
both the preceding clicked position r, and the
distance to this position d.
r=2
d=1
3/22/2016
Position
URL
Click
1
2
3
4
5
6
…
cikm2008.org
www.cikm.org
www.cikm.org/2002
www.fc.ul.pt/cikm2007
cikmconference.org
www.comp.polyu.edu.hk/...
…
0
1
0
0
0
1
…
CIKM'09 Tutorial, Hong Kong, China
61

The examine-next probability depends on
both the preceding clicked position r, and the
distance to this position d.
r=2
d=2
3/22/2016
Position
URL
Click
1
2
3
4
5
6
…
cikm2008.org
www.cikm.org
www.cikm.org/2002
www.fc.ul.pt/cikm2007
cikmconference.org
www.comp.polyu.edu.hk/...
…
0
1
0
0
0
1
…
CIKM'09 Tutorial, Hong Kong, China
62

The examine-next probability depends on
both the preceding clicked position r, and the
distance to this position d.
r=2
d=3
3/22/2016
Position
URL
Click
1
2
3
4
5
6
…
cikm2008.org
www.cikm.org
www.cikm.org/2002
www.fc.ul.pt/cikm2007
cikmconference.org
www.comp.polyu.edu.hk/...
…
0
1
0
0
0
1
…
CIKM'09 Tutorial, Hong Kong, China
63

The examine-next probability depends on
both the preceding clicked position r, and the
distance to this position d.
 Users would lose patience when they browse
through without issuing a click.
 The probability monotonically drops as d
increases and r remains the same.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
64

The examine-next probability depends on
both the preceding clicked position r, and the
distance to this position d.
 P(Ei=1|C1:i-1)= βri,d
where ri = max{j| j <i , Cj=1}, di = i - ri
 55 parameters are needed for top-10 positions
i
(0≤r<r+d≤10).
 Cascade hypothesis is not assumed.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
65

The full picture:
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
66

Introduction

Designing click models
 Basic user hypotheses
 Modeling the first click
 Extending to multiple clicks
 Summary of model design
 Bayesian click models
 Selected topics on click models
 Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
67

Probability of examine the first URL
* Footnote:
3/22/2016 it
Model
Cascade
P(E1)
1
DCM
CCM
DBN
1
1*
1*
UBM
β0,1
Hongparameter
Kong, China
is flexible toCIKM'09
add Tutorial,
another
to specify this probability.
68

Probability of click upon examination
*Footnote:
3/22/2016
Model
Cascade
P(Ci=1|Ei=1)
rd
DCM
CCM
DBN
rd
rd *
ad
UBM
rd
i
i
i
i
i
CIKM'09
Tutorial, Hongdistribution,
Kong, China
the mean of the
relevance
detailed in the next part
69

Probability of examine-next w/o a click
Model
Cascade
P(Ei+1=1|Ei=1,Ci=0)
1
DCM
CCM
DBN
1
α1
γ
UBM
*Footnote:
3/22/2016
CIKM'09
Tutorial,
Kong, China
the probability
does
notHong
depend
on Ei
βr
i+1
,di+1
*
70

Probability of examine-next after a click
3/22/2016
Model
Cascade
P(Ei+1=1|Ei=1,Ci=1)
--
DCM
CCM
DBN
αi
α2(1-rd ) + α3rdi
γ(1-sd )
UBM
βi,1
CIKM'09 Tutorial, Hong Kong, China
i
i
71

Probability of examine-next after a click
3/22/2016
Model
Cascade
P(Ei+1=1|Ei=1,Ci=1)
--
DCM
CCM
DBN
αi
α2(1-rd ) + α3rdi
γ(1-sd )
UBM
βi,1
CIKM'09 Tutorial, Hong Kong, China
i
i
72

Size of parameter sets
3/22/2016
Model
Cascade
# of global params
0
DCM
CCM
DBN
9
3
1
UBM
55
CIKM'09 Tutorial, Hong Kong, China
73

Inference and estimation algorithms
Model Single-Pass
3/22/2016
Details
DCM
Maximizing a lower bound of LL,
fastest
CCM
No iteration needed, thanks to
the Bayesian framework
DBN
EM-based, iterative algorithms
UBM
EM-based, usually takes ~30
iterations to converge
CIKM'09 Tutorial, Hong Kong, China
74

Inference and estimation algorithms
Model Single-Pass
3/22/2016
Details
DCM
Maximizing a lower bound of LL,
fastest
CCM
No iteration needed, thanks to
the Bayesian framework
DBN
EM-based, iterative algorithms
UBM
EM-based, usually takes ~30
iterations to converge
CIKM'09 Tutorial, Hong Kong, China
75

Introduction
Designing click models

Bayesian click models

 Bayesian framework and the rationale
 Bayesian Browsing Model: a case study
 Click Chain Model in a nutshell


Selected topics on click models
Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
76
“probability” of p(H)
p(H)=0.8
0 p(H) 1
Posterior
0 p(H) 1
Bayesian
Frequentist
3/22/2016
Prior
CIKM'09 Tutorial, Hong Kong, China
77
Posterior
Prior
Density Function
(not normalized)
3/22/2016
x
x2
CIKM'09 Tutorial, Hong Kong, China
x3
x3(1-x)
x4(1-x)
78
Posterior
Prior
Density Function
(not normalized)
3/22/2016
x1(1-x)0
x2(1-x)0
CIKM'09 Tutorial, Hong Kong, China
x3(1-x)0
x3(1-x)1
x4(1-x)1
79

The graphical model for coin-toss
X
C1
3/22/2016
C2
C3
CIKM'09 Tutorial, Hong Kong, China
C4
C5
80

The graphical model for coin-toss
X
C1
3/22/2016
C2
C3
CIKM'09 Tutorial, Hong Kong, China
C4
C5
81
x1
(1-x)0
Density Function (1-0.6x)0
(not normalized) (1+0.3x)1
(1-0.5x)0
(1-0.2x)0
…
Prior
3/22/2016
x1
(1-x)1
(1-0.6x)0
(1+0.3x)1
(1-0.5x)0
(1-0.2x)0
…
x2
(1-x)1
(1-0.6x)0
(1+0.3x)2
(1-0.5x)0
(1-0.2x)0
…
CIKM'09 Tutorial, Hong Kong, China
x3
(1-x)1
(1-0.6x)1
(1+0.3x)2
(1-0.5x)0
(1-0.2x)0
…
x3
(1-x)1
(1-0.6x)1
(1+0.3x)2
(1-0.5x)1
(1-0.2x)0
…
82

Representation of relevance
 A probability distribution on
[0,1] for each (query, URL) pair
 The density function is in a
polynomial form over a small
set of linear factors.
 The coefficients of such linear
factors are shared between
different (query, URL) pairs.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
x3
(1-1x)1
(1-0.6x)1
(1+0.3x)2
(1-0.5x)1
(1-0.2x)0
…
83

Inference:
 Go over each query session once, update
the exponents for corresponding (query,
URL) pair impressed*
 Analytical or numerical integration may
be needed to compute the
normalization constant.
*Footnote:
3/22/2016
by virtue of the
CIKM'09
Tutorial,and
Hongconditional
Kong, China independence relationship/assumption
Bayes
theorem
84

Key problems:
 Which is the right factor to update?
 How to estimate all the coefficients?
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
85

Modeling Benefits:
 Confidence for the URL relevance estimate
 Relative judgments: probability of URL i is more
relevant to the query than URL j
 Easy to interpret: coefficients in linear factors
reflect position-bias and user browsing patterns

Computational Benefits:
 Single-pass, linear algorithms; no iterations
 Paralleled version is easy to implement
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
86

Introduction
Designing click models

Bayesian click models

 Bayesian framework and the rationale
 Bayesian Browsing Model: a case study
 Click Chain Model in a nutshell


Selected topics on click models
Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
87

For a specific query session, let
where 1 ≤ i ≤ M=10.
3/22/2016
S1
S2
S3
…
SM
E1
E2
E3
…
EM
C1
C2
C3
…
CM
CIKM'09 Tutorial, Hong Kong, China
88
Relevance
S1
S2
S3
…
SM
Examination
E1
E2
E3
…
EM
Click
C1
C2
C3
…
CM
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
89
Details


Compute the posterior distribution
Conditional independence relationship
induced from the graphical model
How many times the
URL j was clicked
3/22/2016
How many times URLj was not
clicked when it is at position (r + d)
with the preceding click at position r
CIKM'09 Tutorial, Hong Kong, China
90

Only top M=3 positions are shown, 3 query
sessions and 4 distinct URLs.
3/22/2016
Position
1
2
3
Query Session 1
1
2
3
Query Session 2
1
3
4
Query Session 3
1
3
4
CIKM'09 Tutorial, Hong Kong, China
91

Initialize M(M+1)/2+1 counts for each URL
URL
Clicks
r=0
d=1
r=0
d=2
r=0
d=3
r=1
d=1
r=1
d=2
r=2
d=1
4
0
0
0
0
0
0
0
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
92

Update counts for URL 4
 If not impressed, do nothing;
 If clicked, increment “clicks” by 1;
 Otherwise, locate the right r and d to increment.
URL
Clicks
r=0
d=1
r=0
d=2
r=0
d=3
r=1
d=1
r=1
d=2
r=2
d=1
4
0
0
0
0
0
0
0
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
93

Update counts for URL 4
 If not impressed, do nothing;
 If clicked, increment “clicks” by 1;
 Otherwise, locate the right r and d to increment.
URL
Clicks
r=0
d=1
r=0
d=2
r=0
d=3
r=1
d=1
r=1
d=2
r=2
d=1
4
0
0
0
0
0
0
1
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
94

Update counts for URL 4
 If not impressed, do nothing;
 If clicked, increment “clicks” by 1;
 Otherwise, locate the right r and d to increment.
URL
Clicks
r=0
d=1
r=0
d=2
r=0
d=3
r=1
d=1
r=1
d=2
r=2
d=1
4
1
0
0
0
0
0
1
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
95


The posterior for URL 4
URL
Clicks
r=0
d=1
r=0
d=2
r=0
d=3
r=1
d=1
r=1
d=2
r=2
d=1
4
1
0
0
0
0
0
1
Interpretation:
 The larger the probability of examination, the
stronger the penalty for a non-click.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
96

Keep 2 counts for each parameter (one for
click, and the other one for non-click)
Parameter
Click
Non-click
Parameter
Click
Non-Click
β0,1
0
0
β1,1
0
0
β0,2
β0,3
0
0
0
0
β1,2
β2,1
0
0
0
0
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
97

For each position in a query session, locate
the right r and d to increment.
Parameter
Click
Non-click
Parameter
Click
Non-Click
β0,1
1
0
β1,1
0
1
β0,2
β0,3
0
0
0
0
β1,2
β2,1
0
0
1
0
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
98

For each position in a query session, locate
the right r and d to increment.
Parameter
Click
Non-click
Parameter
Click
Non-Click
β0,1
1
1
β1,1
0
1
β0,2
β0,3
1
0
0
0
β1,2
β2,1
0
0
1
1
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
99

For each position in a query session, locate
the right r and d to increment.
Parameter
Click
Non-click
Parameter
Click
Non-Click
β0,1
1
2
β1,1
1
1
β0,2
β0,3
1
0
0
0
β1,2
β2,1
0
1
1
1
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
100

Maximum-Likelihood Estimate:
Parameter
Click
Non-click
Parameter
Click
Non-Click
β0,1
1
2
β1,1
1
1
β0,2
β0,3
1
0
0
0
β1,2
β2,1
0
1
1
1
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
101
Details

Let

Initializing and updating the counts:
 Time:
Space:
Linear to the size
of the click log
3/22/2016
Almost constant
storage required
CIKM'09 Tutorial, Hong Kong, China
102
Details

Let
Initializing and updating the counts:
 Time:
Space:
 Computing relevance scores using numerical
integration with B bins:
 Time:
Space:

3/22/2016
CIKM'09 Tutorial, Hong Kong, China
103





Step 1: initialize counting statistics;
Step 2: scan through the click log once and
update the counts for both inference and
estimation
Step 3: compute parameter values;
Step 4: use numerical integration to obtain
relevance scores.
Step 2 also applies for (linear) incremental
computation!
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
104

Introduction
Designing click models

Bayesian click models

 Bayesian framework and the rationale
 Bayesian Browsing Model: a case study
 Click Chain Model in a nutshell


Selected topics on click models
Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
105

The user behavior model:
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
106

Graphical model:
Relevance
S1
S2
S3
…
SM
E1
E2
E3
…
EM
C1
C2
C3
…
CM
Examination
Click
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
107
Details
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
108

Number of user behavior parameters
CCM
3

Number of distinct factors for (query, URL)
CCM
22

UBM
55
UBM
56
Number of counts needed for parameters
CCM
5
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
UBM
110
109

Introduction
Designing click models
Bayesian click models

Selected topics on click models


 Scaling click models for Petabyte-scale data
 Click model evaluation
 Tailoring user goals to click models

Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
110

Data collected in 8 weeks
 Job k includes data between week 1 and k
 Both time and space costs are prohibitive for a
single node.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
111

A Simple Task: counting # impression
for each (query, URL) pair
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
112
Machine #1
Machine #2
Machine #3
Machine #4
Extent
Extent
Extent
Extent
GetPairs
GetPairs
GetPairs
GetPairs
Map
Map
Map
Map
Sort
Sort
Sort
Sort
Count
Count
Count
Count
Output
Machine #1
Machine #2
Machine #3
Machine #4
Extent
Extent
Extent
Extent
GetPairs
GetPairs
GetPairs
GetPairs
Map
Map
Map
Map
Sort
Count
“Map” putsSort
all of the same Sort
Pairs onto one
machine. This allows you to group by
Countprocesses.
various Count
fields in subsequent
Output
Sort
Count

A Simple Task: counting # impression
for each (query, URL) pair

Map = Bucket: the intermediate key is
(query, URL) pair
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
115
Machine #1
Extent
Machine #2
Machine #3
Machine #4
Extent
Extent
Extent
GetPairs“Count”
GetPairs
carries
out standardGetPairs
increment-by-1GetPairs
over each distinct Pair.
Map
Sort
Count
Map
Map
Map
“Count” REDUCES the amount of data since
each Pair has
Sort only one output
Sort value
Sort
Count
Count
Output
Count

A Simple Task: counting # impression
for each (query, URL) pair

Map = Bucket: the intermediate key is
(query, URL) pair

Reduce = Count: it accepts a list of (key,
value) tuple, and outputs the final result
for each distinct key
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
117
Machine #1
Machine #2
Machine #3
Machine #4
Extent
Extent
Extent
Extent
GetPairs
GetPairs
GetPairs
GetPairs
Map
Map
Map
Map
Sort
Sort
Sort
Sort
Count
MAP
REDUCE
Count
Count
Output
Count
0 for clicks
3
2 5
1 4 6
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
119

Map: scan the click log
 Intermediate key: (query, URL)
 Value: the index of linear factors
(0~55 for top-10 positions)

Reduce: scan the list of (key, value)
 The key indicates which exponent vector to
update
 The value indicates the index of the element in
the exponent vector to increment
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
120


Linearly increasing computation load
Near-constant elapsed time
• 3 hours
• 265 TB log data
• 1.15 billion (query, url)
pairs
Single machine computation load
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
Elapse time on SCOPE
121

Introduction
Designing click models
Bayesian click models

Selected topics on click models


 Scaling click models for Petabyte-scale data
 Click model evaluation
 Tailoring user goals to click models

Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
122
Impression Data
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
Click Data
123
Impression Data
Click Data
Global
Parameters
Relevance Scores
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
M=10
124
New Impression
Vector from an
Existing Query
Global
params
Relevance
3/22/2016
Predicted Examination
Clicks
CIKM'09 Tutorial, Hong Kong, Predicted
China
125

Data are collected from a commercial search
engine after query term normalization and
spam removal.

For each query term, split query sessions
evenly into training and test sets according to
the timestamp.

Top frequent/infrequent query terms are
removed.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
126

Most popular metrics:
 Average test data log-likelihood (LL)
(probability of accurately predicting the click vector, 2^10
possibilities)
[Guo+09a, Guo+09b, Liu+09a, Zhu+10]
 Perplexity of prediction for each position
(2^{average entropy} of click/no-click binary prediction
for each position independently)
[Dupret+08, Guo+09a, Guo+09b, Zhu+10]
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
127

Other Metrics:
 Click-through-rate (CTR) prediction
(Especially for predicting CTR@1)
[Chapelle+09, Zhu+10]
 Predicting first/last clicked positions
[Guo+09a, Guo+09b]
 Position-bias sanity check
(plot the click rate curve for top-10 positions v.s. the
ground truth)
[Guo+09a, Guo+09b]
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
128

Average Log-likelihood
 Random guess: log(2-10) = -3.01
 Optimal value: 0
Model
CCM
UBM
DCM
LL
Improvement Ratio
-1.171
-1.264
-1.302
9.7%
14%
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
129
Better
Worse
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
130
Better
Worse
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
131

Average Perplexity over top 10 positions
 Random guess: 2
 Optimal value: 1
Model
CCM
UBM
DCM
Perplexity
Improvement Ratio
-1.1479
1.1577
1.1590
7.5%
8.3%
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
132
Worse
Better
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
133
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
134

For 1M query sessions, the estimated time in
seconds:
DCM
80
CCM*
150
BBM*
165
UBM**
5,000
* Time for CCM and BBM includes computing posterior mean
and variance using numerical integration w/ 100 bins.
** UBM converges in 34 iterations.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
135

Introduction
Designing click models
Bayesian click models

Selected topics on click models


 Scaling click models for Petabyte-scale data
 Click model evaluation
 Tailoring user goals to click models

Conclusion
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
136

Queries could be categorized into 2 sets:
 Navigational: to find the link to an existing
website, e.g., bing;
 Informational: more exploration, multiple clicks
may arise, e.g., iron man.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
137



Different user goals result in different
browsing and click patterns.
The straightforward mixture-modeling
approach is not practical. [Dupret+08]
Solution:
 Classify query terms a priori based on user goals.
 Fitting and learning 2 sets of model parameters
for navigational and informational queries.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
138

Two-way classification for query terms based
on click data using…
 Median position of click distribution
 Mean position of click distribution
 Average # clicks per query session
…

Pick the one which has best click prediction
 If a position receives 50% of the click,
then navigational, else informational
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
139

Improvement of click prediction for DCM:
 Log-Likelihood: 4.0%
 Perplexity: 1.3%

Examination/Click position-bias:
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
140
Introduction
 Designing click models
 Bayesian click models
 Selected topics on click models
 Conclusion

3/22/2016
CIKM'09 Tutorial, Hong Kong, China
141

Click models
 A statistical tool to leverage valuable user implicit
feedback in terabyte/petabyte search logs.
 Provide click prediction as well as relevance
estimates.
 Application domains include learning to rank,
measuring search performance, online
advertising, user behavior analysis…
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
142

Click models
 Different model designs reflect various
assumption of user behaviors to explain the
position-bias.
 The modeling choice may depend on the
application scenario.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
143

Click models
 Efficient, single-pass, parallelizable algorithms are
desired in real-world applications.
 Bayesian framework could be applied to click
models for both modeling benefits and
computational benefits.
 Click Chain Model and Bayesian Browsing Model
represent state-of-the-art examples.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
144

Bigger Context
 Query reformulations
 Personalization

Richer inputs
 Universal search
 Diverse user feedback

Click model v.s. Human judgments
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
145





[Burges+05]: C. Burges, T. Shaked, E. Renshaw, A. Lazier, M.
Deeds, N. Hamilton, and G. Hullender. Learning to rank using
gradient descent. ICML’05.
[Chapelle+09]: O. Chapelle and Y. Zhang. A dynamic Bayesian
network click model for web search ranking. WWW’09.
[Craswell+08]: N. Craswell, O. Zoeter, M. Taylor, and B. Ramsey. An
experimental comparison of click position-bias models. WSDM
’08.
[Dean+04]: J. Dean and S. Ghemawat. MapReduce: Simplified data
processing on large clusters. OSDI’04.
[Dupret+08]: G. Dupret and B. Piwowarski. A user browsing model
to predict search engine click data from past observations.
SIGIR’08.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
146





[Guo+09a]: F. Guo, C. Liu, and Y.-M. Wang. Efficient multiple-click
models in web search. WSDM’09.
[Guo+09b]: F. Guo, C. Liu, A. Kannan, T. Minka, M. Taylor, Y.-M.
Wang, and C. Faloutsos. Click chain model in web search.
WWW’09.
[Guo+09c]: F. Guo, L. Li, and C. Faloutsos. Tailoring click models to
user goals. WSCD’09.
[Joachims02]: T. Joachims. Optimizing search engines using
clickthrough data. KDD’02.
[Joachims+07]: T. Joachims, L. Granka, B. Pan, H. Hembrooke, F.
Radlinski, and G. Gay. Accurately interpreting clickthrough data as
implicit feedback, ACMTOIS, 25(2), 2007.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
147





[Lee+05]: U. Lee, Z. Liu, and J. Cho. Automatic identification
ofuser goals in web search. WWW’05.
[Liu+09a]: C. Liu, F. Guo, and C. Faloutsos. BBM: Deriving click
models from petabyte-scale data. KDD’09.
[Liu+09b]: C. Liu, M. Li, and Y.-M. Wang. Post-rank reordering:
resolving preference misalignments between search engines and
end users. CIKM’09.
[Richardson+07]: M. Richardson, E. Dominowska, and R. Ragno.
Predicting clicks: estimating the click-through rate for new ads.
WWW’07.
[Zhu+10]: Z. Zhu, W. Chen, T. Minka, C. Zhu and Z. Chen. A novel
click model and its applications to online advertising. To appear in
WSDM’10.
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
148
Nick Craswell
MSR, Cambridge
Christos Faloutsos
Carnegie Mellon University
Anitha Kannan
Li-Wei He
MSR, ISRC-Redmond
Tom Minka
MSR, Cambridge
MSR, Search Lab
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
149
Mike Taylor
MSR, Cambridge
3/22/2016
Ethan Tu
Yi-Min Wang
MSR, ISRC-Redmond
MSR, ISRC-Redmond
CIKM'09 Tutorial, Hong Kong, China
150
3/22/2016
CIKM'09 Tutorial, Hong Kong, China
151