Context-Aware Ranking in
Web Search
(SIGIR 10’) Biao Xiang, Daxin
Jiang, Jian Pei, Xiaohui Sun,
Enhong Chen, Hang Li
2010/10/26
1
Outline
Introduction
Ranking Principles
Context-aware Ranking Principles
Effectiveness of Principles
Context-Aware Ranking
Experimental Results
Conclusions
2
Introduction
almost all the existing ranking models
consider only the current query and the
documents
 do not take into account any context
information
the previous queries in the same session
the answers clicked on
skipped by the user to the previous queries
3
 two critical problems about context-aware
ranking for Web search
How can we take advantage of different types of
contexts in ranking?
How can we integrate context information into a ranking
model?
 discuss the types of contexts and propose
ranking principles
 evaluate the effectiveness of the principles
 incorporate context information into a learningto-rank model
4
Ranking Principles
*Context-Aware Ranking Principles
Reformulation
C
S
C
S
 Principle 1 (Reformulation). For consecutive queries
in a
session such that qt reformulates
, if a search result d for
is clicked on or skipped,
as a result for
is unlikely to be
clicked on and thus should be demoted.
5
 Specialization
 Principle 2 (Specialization). For consecutive queries
in a session such that qt specializes
, the user
likely prefers the search results specifically focusing on .
6
Generalization
 Principle 3 (Generalization). For consecutive queries
in a session such that
generalizes
, the
user would likely not prefer the search results specifically
focusing on qt−1.
7
 General Association
 Principle 4 (General association). For consecutive queries
in a session such that
and
are generally associated, the
user likely prefers the search results related to both
and qt.
Such results should be promoted for qt.
8
How to import the principles
Principle2

:the set of terms appearing in query
but not in query .

promote the results matching
in the set of answers to .
Principle3

:the set of terms appearing in query
but not in query .

promote the results matching
in the set of answers to
.
9
Principle4
choose any topic taxonomy such as the Open
Directory Project (ODP)

( ): the sets of topics of
( )

: the set of common topics between
and

promote a search result u if the set of
topics of u shares at least one topic with .
10
Ranking Principles
*Effectiveness of Principles
37,320 user sessions
successive query pairs within the same
sessions
manually labeled the relations
10, 000 randomly selected successive
query pairs.
11
a search result
satisfies the principle if
should be promoted(2,4) or not(1,3)

:the set of search results that satisfy
the principle

:consists of the search results that
were clicked on by the users
estimated
Δ = P(c = 1|h = 1)− P(c = 1|h =0)
12
Evaluation in Different Types of Contexts
 the two successive queries match the relation of the principle
Evaluation in All Contexts
13
Context-Aware Ranking
RankSVM : an SVM model for
classification on the preference between a
pair of document.
the original ranking list from the
search engine
the list from the RankSVM
RankSVM-R0
RankSVM-R1
RankSVM-F
re- ranking
14
Experimental Results
 1500 cases, 1000 for training,500 for validation

,
15
Conclusions
 human labeled data and user click data
complementary to each other
 the four context-aware methods are better than
the search engine
the effectiveness of contextaware ranking.
 all three RankSVM methods perform better
than the baseline in context-aware ranking.
consider different types of contexts in Web search.
 the RankSVM-F and RankSVM-R1 methods
show larger improvements than RankSVM-R0
the usefulness of considering the original ranking of the
search engine.
16