CONTEXTUAL QUERY USING BELL TESTS
JOAO BARROS, ZENO TOFFANO, YOUSSEF MEGUEBLI AND BICH-LIÊN DOAN
SUPELEC (ÉCOLE SUPÉRIEURE D’ÉLECTRICITÉ)
FRANCE
Quantum Interaction 2013
Leicester
July 25-27, 2013
QUANTUM INTERACTION RESEARCH AT SUPELEC

Research activity initiated in 2011 under the impulse of
 Bich-Liên DOAN (Computer Science: Information Retrieval, Semantic
Web), Dep. Of Computer Science.

Zeno TOFFANO (Physicist, lectures Quantum Mechanics. Research: Solid
State Physics, NMR, Lasers, Fiber Optics Telécommunications), Dep. Of
Telecommunications.
2 PhDs
 Joao BARROS (MsC: Theoretical Physics) at the heart of this research
(funding from « Fondation SUPELEC »).




Youssef MEGUEBLI (MsC: Computer Science) : Opinion based
Information retrieval.
We undertook some preliminary investigations in the form of tests.
 (arXiv:1207.4328)
We emphasize on the experimental « Quantum-like » approach.
PRELIMINARY INVESTIGATIONS :
POLL TESTS ON POLYSEMY IN FOREIGN LANGUAGE:



The test assesses correlations in a foreign language (Chinese here).
It aims to show the role of polysemy of words. The question was to
quantify the correlation with different meanings of the proposed words.
4 people were interviewed (all Chinese) to give their opinion scores.
POLL TEST
chinese word
笔记本
性
生
清
出入
4 persons
polysemy
Scores
laptop computer
9964
paper notebook
4667
sex
9956
character
2665
life
5585
to be born
9967
Qing dynasty
8648
fair end honest
2563
go in and out
8666
the failure to agree
4696
PRELIMINARY INVESTIGATIONS :
POLL TESTS ON HETEROGENEOUS MEDIA


The test proposes nine musical excerpts.
The question is to rate from 0 to 10 whether these excerpts fall under the category "rock" or
"blues ". 4 persons were interviewed.
POLL TEST
“Music Excerpts belonging to”
“Blues”
“Rock” + “Blues”
Interference
5.5
4.5
10
=
8
2.5
10.5
over
4.5
5.75
10.25
over
« Folsom prison blues »
4
4
8
under
« The wind cries Mary »
4.75
6.25
11
over
« Don't let me down »
8.25
3.25
11
over
« Tenth avenue freeze out »
3.5
3.75
7.25
under
2
8
10
=
7.5
3.25
10.75
over
« Dazed and confused »
« Susie Q »
« That's all right »
« Since I've been loving you »
« I heard it through the grapevine»


“Rock”
The sum of the results for both categories is very rarely equal to 10 indicating that the
chosen categories are certainly not mutually exclusive.
Interference effects between concepts of different media (over-extension/under-extension).
PRELIMINARY
INVESTIGATIONS
WORDS BELONGING TO 2
CATEGORIES : CORRELATION


Tests on words belonging to two categories
Fruit, Vegetable or to both (questions are
independent)
We define a « Bell-like » correlation
parameter
𝑡𝑒𝑠𝑡
𝑆𝐵𝑒𝑙𝑙
= 𝜇 𝐹 𝜇 𝑉 +𝜇 𝐹 1−𝜇 𝑉
+ 1 − 𝜇 𝐹 𝑜𝑟 𝑉

−
1−𝜇 𝐹 𝜇 𝑉
In this analysis we observed no violation of
the Bell Inequality (<2)
POLL TEST
“Word belonging
to”
max: 1
max: 1
max: 1
max: 4
µ(Fruit)
µ(Veg.)
µ(F or V)
SBell
garlic
.16
.52
.33
0.39
almond
.68
.07
.83
0.82
beet
.2
.4
.53
0.35
broccoli
.04
.92
1
0.93
mushroom
.06
.75
.6
0.36
cauliflower
.06
.92
.93
0.86
cucumber
.18
.72
.83
0.61
gherkin
.26
.47
.8
0.41
spinach
.04
.95
.8
0.75
bean
.1
.97
.87
0.84
coco nut
.96
.02
.6
1.36
olive
.6
.4
.67
0.77
parsley
.14
.35
.47
0.37
pepper
.08
.05
0
1.03
potato
.16
.45
.17
0.62
apple
.94
.03
1
0.94
ginger
.1
.22
.26
0.63
grapes
1
.02
1
1
tomato
.8
.6
1
0.92
PRELIMINARY INVESTIGATIONS
FIRST APPROACH : « HEURISTIC QUANTUM-LIKE HAL MODEL »

Our goal here is to classify the texts according to a context-related search criteria using the HAL
algorithm (Hyperspace Analog to Language).

We create a symmetrical HAL matrix (more discussions hereafter).

A query has been undertaken: "A AND B".



Texts referring to the word Tomato (can be considered either as Fruit or as Vegetable). Three
documents were selected from the Internet using related keywords (for complete discussion see
arXiv:1207.4328)
The score p given by the algorithm corresponds to the probability for this text to be in first
position in our request. The average value of the test is defined as E = 2p1 (E from 1 to +1).
A Bell parameter (CHSH type) is defined and calculated as follows
𝐻𝐴𝐿
𝑆𝐵𝑒𝑙𝑙
= 𝐸 𝑝𝑇𝐹 − 𝐸 𝑝𝑇𝑉

We observe Bell
inequality violation
in document 2
𝐻𝐴𝐿
𝑆𝐵𝑒𝑙𝑙
>2
+ 𝐸 𝑝𝑃𝐹 − 𝐸 𝑝𝑃𝑉
Document n°
1
2
3
Tomato AND Fruit
0.788
0.581
0.373
score
pTF
Tomato AND Vegetable
0.349
0.469
0.213
score
pTV
Plant AND Fruit
0.651
0
0.385
score
pPF
Plant AND Vegetable
0.315
0
0.223
score
pPV
0.947
2.23
1.105
Bell param. S




Long story:
 The field of Bell inequality violations (Bell 1964) and
entanglement has fascinated many scientists
throughout the last decades. An interesting historical
narrative is in “How the Hippies Saved Physics” by
David Kaiser, Ed. W. W. Norton (Physics World 2012
Book of the Year ).
Much debate
 classical and non-classical behaviour
 entanglement
 local and non-local, contextual and non-contextual
 more than Quantum, non-local boxes…
Experiments demonstrating Bell inequality
violation
 1969 Clauser: first experiment
 1982 A. Aspect (Orsay France) on polarized photons:
definitive proof
 Entanglement with Spins (NMR, Rydberg atoms…)
 Towards the realization of a Quantum Computer
A new field: Quantum Information
 Entanglement is at the heart of this field because it is
seen as a potential “resource” for computing (lower
complexity) and coding (secure cryptography)
BELL
INEQUALITIES
THE BELL CHSH INEQUALITY CASES

The CHSH (Clauser, Horne, Shimony, Holt)-Bell parameter SBell form is proposed for tests with two
binary outcomes, +1 or 1, adapted to query answers (YES/NO), can be defined as follows:
𝑆𝐶𝐻𝑆𝐻−𝐵𝑒𝑙𝑙 = 𝐸 𝐴, 𝐵 − 𝐸 𝐴, 𝐶 + 𝐸 𝐵, 𝐷 + 𝐸(𝐶, 𝐷)

where 𝐴, 𝐵, 𝐶 and 𝐷 are tests and 𝐸 𝑋, 𝑌 stands for the expectation value of the outcome of mutual
tests 𝑋 and 𝑌. 𝑆𝐵𝑒𝑙𝑙 can never exceed 4.

Classical, local, separable: 𝑆𝐵𝑒𝑙𝑙
lies between 0 and 2. We could write
𝐸 𝑋, 𝑌 = 𝐸 𝑋 𝐸 𝑌 .

Quantum: The case 2 ≤ 𝑆𝐵𝑒𝑙𝑙 ≤ 2 2
achieved with bipartite quantum
entangled states.
𝑆𝐵𝑒𝑙𝑙 = 2 2 is
called the Tsirelson’s bound and is a
limit for Quantum systems.

No-signalling. The case between
2 2 and 4 is called the “nosignalling” region. The maximum
value 𝑆𝐵𝑒𝑙𝑙 = 4 can be attained with
logical probabilistic constructions
often named non-local PR boxes.
HAL AND QI RESEARCH






We investigate the relationships between words within a document; these relationships can be
formed by creating a “semantic space” using the Hyperspace Analogue Language (HAL)
introduced by Lund and Burgess (1996).
The HAL algorithm does not require any explicit human a-priori judgment. In the procedure a
HAL lexical co-occurence matrix is built with a "window," representing a span of words
passed over the corpus being analyzed.
Operationally, two words are considered as co-occurring when they appear in the same floating
window. The size of this window is a few hits left and right of the word in question.
Similar approach: LSA (Latent Semantic Analysis) also builds matrices in semantic space.
Darányi, Wittek,
 Physical analogy between semantic space of HAL and Quantum Theory, where at each
word can be associated a given energy (in analogy with spectral emission lines in atoms
corresponding to transition energies)
Bruza
 HAL used for analogies with Quantum Theory for activating associations of concepts.
THE HAL MATRIX SEMANTIC SPACE





The matrix is built with a "window" representing a span of words passed over the
corpus being analyzed. The width of this window can be varied.
Words within the window are recorded as co-occurring with weight inversely
proportional to the number of other words separating them within the window (word
distance measure).
The information contained in a line is the sum of co-occurrences for words appearing
before the word, the information contained in a column represents the sum of cooccurrence for the words appearing after the word.
We used a symmetric real positive matrix obtained by the sum of the HAL
matrix and its transpose (equivalent to run HAL backwards).
All words are considered and simple plurals are treated as singular words. Lower
and upper case letters are not distinguished. Words having the same origin are
treated differently (for example “battle” and “battling” are distinct).
DOCUMENT « ORANGE »
CONSTRUCTION OF THE




HAL MATRIX
Symmetric matrix sum of two HAL matrices (forward and backward).
Repeated words contribute to strengthen the associated vector (see “orange” and “the” in the
example below).
The rows and columns of the symmetric co-occurrence matrix constitute vectors in a highdimensional space.
The dimensionality of the space is determined by the number of columns in the matrix (context
vectors).
TEXT example with a window spanning on 3 words (l = 3)
"THE COLOUR ORANGE TAKES ITS NAME FROM THE ORANGE FRUIT"
Matrix
(l=3)
M+M^T
THE
THE
COLOUR ORANGE TAKES
ITS
NAME
FROM
FRUIT
16
3
5
1
1
2
3
2
COLOUR
3
8
3
2
1
0
0
0
ORANGE
5
3
16
3
2
2
2
3
TAKES
1
2
3
8
3
2
1
0
ITS
1
1
2
3
8
3
2
0
NAME
2
0
2
2
3
8
3
0
FROM
3
0
2
1
2
3
8
1
FRUIT
2
0
3
0
0
0
1
8
QUANTUM MODEL FOR HAL :
VECTOR DEFINITION


We attribute to each document an associated vector.
The vector state of the document is the linear sum of all the word vectors |𝒘𝒊 it
contains. Each word vector state is extracted from the lines of the symmetric HAL
matrix.
𝑁
|Ψ =
|𝑤𝑖
𝑖




We are interested in analyzing how two words are connected within a document, namely
word 𝑨 and word 𝑩.
The two associated word vectors |𝑤𝐴 and |𝑤𝐵 define a plane on the semantic space.
We will consider the projection of the document vector state |Ψ on the plane
spanned by |𝑤𝐴 and |𝑤𝐵 .
This resulting normalized state vector, represents the reduced document state
vector |𝝍 .
QUANTUM MODEL FOR HAL :
VECTOR ORTHONORMALIZATION




To obtain |𝜓 we take the vectors |𝑤𝐴 and |𝑤𝐵 and normalize them obtaining two new
vectors: |𝑢𝐴 and |𝑢𝐵 forming a non-orthogonal basis |𝒖𝑨 , |𝒖𝑩 (in general).
We apply the Gram-Schmidt orthogonalization process to the basis {|𝑢𝐴 , |𝑢𝐵 }, and
we obtain two new orthonormalized basis that describe the plane formed by the original
vectors |𝑤𝐴 and |𝑤𝐵 : the basis {|𝒖𝑨 , |𝒖𝑨⊥ } and {|𝒖𝑩 , |𝒖𝑩⊥ }.
By projecting the vector |Ψ on one of these basis, we obtain its projection onto this plane.
Taking this vector and renormalizing gives us the desired vector |𝝍 .
The vector |𝜓 can be decomposed on both orthogonal basis.
|𝜓 = α|𝑢𝐴 + α⊥ |𝑢𝐴⊥ = β|𝑢𝐵 + β⊥ |𝑢𝐵⊥

The coefficients α, α⊥ , β and β⊥ are obtained by projecting the state vector |Ψ on both
basis vectors and then normalizing to unity.
QUANTUM MODEL FOR HAL :
QUERY OPERATORS



We want to define Query operators.
The query operators 𝐴 and 𝐵 are defined
 +1 state that corresponds to the word meaning we are interested.
 −1 in the orthogonal direction.
When applied to the document state vector |𝜓 defined before :
𝐴|𝜓 = α|𝑢𝐴 − α⊥ |𝑢𝐴⊥

𝐵 |𝜓 = β|𝑢𝐵 − β⊥ |𝑢𝐵⊥
This action is analogous to the spin Pauli matrix 𝜎𝑧 , and we can associate it to 𝐴 in the
basis {|𝑢𝐴 , |𝑢𝐴⊥ }
𝐴 = 𝜎𝑧 =

1
0
0
−1
Other query operators can be defined. We choose 𝐴𝑥 and 𝐵𝑥 . The action of 𝐴𝑥 on |𝜓
gives on the {|𝑢𝐴 , |𝑢𝐴⊥ } basis:
𝐴𝑥 α|𝑢𝐴 + α⊥ |𝑢𝐴⊥

= α⊥ |𝑢𝐴 + α|𝑢𝐴⊥
This action corresponds to switching the components and is equivalent to the spin Pauli
matrix 𝜎𝑥
𝐴𝑥 = 𝜎𝑥 =
0
1
1
0
QUANTUM MODEL FOR HAL :
QUERY OPERATOR BASIS REPRESENTATION


We choose the basis associated to word 𝐴 , {|𝑢𝐴 , |𝑢𝐴⊥ } , and write the operators with respect
to this basis.
Using the transformation matrix 𝑀 from the 𝐴 basis to the 𝐵 basis
𝑀=
𝑢𝐵 𝑢𝐴
𝑢𝐵⊥ 𝑢𝐴
𝑢𝐵 𝑢𝐴⊥
𝑢𝐵⊥ 𝑢𝐴⊥
=
𝑝
− 1 − 𝑝2
1 − 𝑝2
𝑝
where 𝑝 = 𝑢𝐵 𝑢𝐴 is the scalar product here a positive number smaller than 1

We obtain the 𝐵 matrix form in the {|𝑢𝐴 , |𝑢𝐴⊥ } basis associated :
𝐵=
2𝑝2 − 1
2𝑝 1 − 𝑝2
2𝑝 1 − 𝑝2
1 − 2𝑝2
BELL PARAMETER CALCULATION
USING QUERY OPERATORS



Bell tests are usually a proof of a non-separability of the combination of two different
systems.
We define a parameter 𝑆𝑞𝑢𝑒𝑟𝑦 that can be understood as the sense associated to a word A in a
document in correlation with the sense of another word B.
The Bell parameter 𝑆𝑞𝑢𝑒𝑟𝑦 is the combination quantum mean values with different query
operators which can be considered as measuring devices:
𝑆𝑞𝑢𝑒𝑟𝑦 = 𝐴𝐵+ + 𝐴𝑥 𝐵+ + 𝐴𝐵− − 𝐴𝑥 𝐵−

Using specific operators associated to words A and B .
𝐴 ; 𝐴𝑥 ; 𝐵+ = −


1
2
𝐵 + 𝐵𝑥
; 𝐵− =
1
2
𝐵 − 𝐵𝑥
This particular operator choice is inspired from the usual example that maximizes
the violation of the Bell inequalities .
We calculate quantum mechanical mean values over vector |𝜓 using the Born rule.
For example:
𝐴𝐵
𝜓
= 𝜓 𝐴𝐵 𝜓
BELL PARAMETER
CALCULATION:
Q-HAL
ALGORITHM
Input
Document
Construction of a “clean” (no punctuation
marks) sequence of words, including
eventual repeated words: Doc list.
Construction of the “Dictionary”:
sequence of non repeated words: Dic list.
Window size l
Construction of a primitive HAL matrix: for each word of the Doc list a window of length l is
associated and all the scores of the words within it are collected in a matrix. The entry for each
score is determined by the position of the words in the Dic list. Complete HAL matrix is obtained
by summing this matrix with its transpose.
Normalization of each row vector. Determination of the state of the system by summing over all
vectors and normalizing.
Calculation of the expected values of the defined operators and the Bell parameter.
New window size
l+1
Plot



Flow diagram of the Quantum HAL algorithm.
The algorithm was implemented using Python programming language along with the
string module and pylab.
Our approach presented here can be perceived as an experiment done on objects
outside the domain of physics.
QUERY 1 :
WORD “REAGAN” IN THE CONTEXT OF WORD “IRAN”.




The Bell parameter function of the window
size starts from zero and increases until it
reaches a maximum (the Tsirelson's bound
2 2) then drops again. (for 3 documents)
The document “Iran” always gives a
constant value of 2. Here one of the words
(“Reagan”) is missing.
This suggests that each document has an optimal HAL window size that maximizes the
parameter 𝑆𝑞𝑢𝑒𝑟𝑦 .
The “sooner” a peak appears the less interaction, in the sense of window length, is needed to
get higher correlation between the two words. Bearing this in mind, the document “IranContra affair” is clearly the one selected by the model.
QUERY 2: TEST ON THE POLYSEMY OF THE
WORD “ORANGE”



Test on the polysemy of the word “orange” and associated concepts. In this example we
are interested in the ambiguity between the meanings color and fruit. We also associate
the concept of juice.
The query “Orange - Fruit” presents the first peak around 𝑙 = 22 for the document
“Orange color”, the second in 𝑙 = 29 for the document “Orange fruit”, then for 𝑙 = 40 the
document “Orange juice” and very far away the document “Juice”.
Regarding the peaks we find the expected order for the documents: “Orange juice”,
“Orange Fruit”, “Juice” and “Orange Color”.
QUERY: PATHOLOGICAL TEXT EXAMPLE
Queries on words A1 and A100 in a
text of 5000 words, for text
repetition periodicities of 100, 150
and 200.


We made 2 word queries on « pathological » documents consisting in texts with
repeating periodic structure based on the same original document.
The curves still peak at the Tsirelson’s bound and also present other effects probably
due to the repetition period.
COMMENTS ON RESULTS




The results show Bell parameter that peaks up to the maximal value of
Sbell = 2√2, (the Tsirelson’s bound).
We found that the Bell parameter is strongly dependent on the HAL
window size. There is an optimal window size that maximizes Sbell.
Reminiscent of what was already noticed (Bruza) a possible explanation :
 if the window size is set too large, spurious co-occurrence associations
are represented in the matrix
 if the window size is too small, relevant associations may be missed.
Comparing different documents, the one with the first appearing peak
seems to be the more relevant.
SOME CONCLUSIONS AND PERSPECTIVES







The main feature in relation to Quantum Theory explored in this work is
the violation of the Bell inequalities which can be related to entanglement
and nonlocality.
The results show always correlation on two words due to Bell inequality
violation up to the maximal value of 𝑆𝐵𝑒𝑙𝑙 = 2 2, (the Tsirelson’s bound).
We introduced a new tool which has connections with the Quantum
Theory: Query Operators.
It is not clear how to interpret the Bell inequality violation here and what
is the meaning of the optimal length that maximizes the Bell parameter.
HAL constitutes a good « playground » for doing Quantum-like
experiments.
We believe that it should be possible, after much experimentation on
different documents, to introduce new families of query observables
adapted to different purposes and contexts in Information Retrieval.
Can entanglement give a measure of query relevance?
Thank You