From: AAAI Technical Report SS-03-03. Compilation copyright © 2003, AAAI (www.aaai.org). All rights reserved.
STCSP: A Representation Model for Sequential Patterns
Aomar Osmani
Laboratoire LIPN-CNRS UMR7030
99, Avenue J.-B. Clément,
F-93430 Villetaneuse.
{ao@lipn.univ-paris13.fr}
Abstract
Temporal constraint satisfaction problem is a useful formalism for representing and reasoning about temporal
occurrences and their temporal relationships. In this
paper, we present a new temporal constraint satisfaction problem to deal with sequential patterns (STCSP).
The problem is represented as a constraint satisfaction problem with sequential valuated variables, where
multiple sequential values are attributed for each variable. Several classes of STCSP can be defined: qualitative/quantitative intervals/points and any combination
of them.
We will introduce the STCSP in the case of qualitative
points. The main results are: (i) a new formalization
of the sequential patterns problems, (ii) the definition of
the usual problems: finding solution, checking the consistency, answering queries about scenarios that satisfy
some classes of constraints, etc., (iii) the algorithm for
reasoning about qualitative point STCSP with a particular evaluation function.
Introduction
The serial behavior is essential in many fields of
the artificial intelligence, particularly in planning
(Moi98),
reasoning (Kui94),
robotics (PFL+ 00;
natural
language
(Pro01;
SWea96),
SOS+ 99),
speech
recognition
(JUA93),
adaptive
control
(Mee95), prediction of temporal series (HBM01;
BJGG+ 02), biological applications (Pil00; JbAK01;
OHTT01; MA01),
weather prediction (Hop01;
Hop02),
telecommunication
networks
(ORC+ 99;
CD99; Osm99; Rie96), market places (Tes92), multiagents systems (Moi98; RPBA02), training of navigation (SP99), e-commerce (Tes92), anomalies detection (LC98), data mining in great databases (AS95;
D.J01).
The treatment of the sequences or the sequence-mining
is the discovery of sets of characteristics shared through
time by a great number of objects ordered in time (D.J01).
It is an interdisciplinary problem which uses in particular
artificial intelligence, databases, cognitive sciences, and
c 2003, American Association for Artificial IntelliCopyright gence (www.aaai.org). All rights reserved.
engineering techniques.
There are several approaches for sequences representation and sequences reasoning. These approaches depend
on the problem formulation (Example possibility or not
to take into account actions), the type of considered
objects: points or/and intervals, the type of reasoning:
qualitative or quantitative and the type of the considered
problem: prediction, recognition, or sequences generation. Several types of formalisms are currently used
like Markov’s models (Bau72; FAN97; JBS99), symbolic planning systems (AC90), neural networks (FGS95;
GHL95), associative networks (ea88; KS88), dynamic programming (BT96; BC96), reinforcement learning (Bel57;
Lin92), probabilistic prediction (OH97), graphs models,
interval algebra, temporal difference approaches (SB97),
linear regression (BER90), chronicles (DG94), inductive
logical programming (aSD94), etc. Some works combine
various approaches for the sequences learning problem:
symbolic rules systems and neural networks (SP98), symbolic planning and reinforcement learning (DB97), etc.
The work published by Ron Sun and Lee Giles (Se02)
gives a progress report on the sequence learning problem
formulation. It gives a comparison of various models, their
capacities and their limits, the implementations, the empirical comparisons and their uses in concrete applications.
Our interest is to use a Constraint Satisfaction Problem
(CSP) approach to deal with sequence representation and
sequence reasoning. A classical CSP problem is specified
by providing a set of variables, the corresponding domain
values for each variable, and a set of constraints on the
variables. Solving an instance of a CSP problem consists
in assigning values to variables such that all constraints are
satisfied simultaneously (Tsa93). Each constraint is defined
over some subset of the original set of variables and limits
the combinations of values that the variables can take in
this subset. The number of affected variables is the arity
of the constraint. Any n-ary constraint can be expressed
as a binary constraint. Hence, binary CSPs are, in sense,
representative of all CSP problems. For a comprehensive
overview on the CSP see (Kum92). An exhaustive study
can also be found in (Tsa93).
The Temporal CSP (TCSP) (SV98) is a particular class of
CSP where variables represent times and constraints represent sets of possible temporal relations between them. The
purpose of this paper is to present a TCSP approach to deal
with sequential patterns. One of the originalities of this
framework is the possibility to represent and to reason about
time evolution and multiple variables values. Let us consider
the following examples:
Example 1 I read the newspaper (I1 ) when I take my breakfast (I2 ). After breakfast I will take the bus to go to my office
(I3 ).
Example 2 I read the newspaper when I take my breakfast.
After breakfast I take the bus to go to my office. Before starting my work (I4 ) I read my newspaper.
In example 1, each variable has only one instantiation. If
we consider interval algebra representation (All83), we will
have the following expression:
(I2 during I1 ) ∧ (I1 bef ore I3 )
In example 2, variable I1 has two instantiations among the
time:
(I2 during I1 ) ∧ (I1 bef ore I3 )∧
(I3 bef ore I2 ) ∧ (I2 bef ore I4 )
If we consider the classical TCSP consistency algorithms,
this situation is wrong. This type of situation is essential
in several applications like alarm diagnosis (Osm99;
OL00). To present this problem, we will consider in this
paper a particular STCSP: qualitative point STCSP.
The paper is organized as follows: Section 2 presents
the usual problems concerned with sequences. Section 3
introduces the model. Section 4 defines some fundamental concepts. In section 5, we describe the qualitative point
STCSP with the frequency evaluation function. Conclusions
are given in section 5.
STCSP problems
These are various sequences patterns problems: looking for
more frequent solutions, verifying the frequency of a given
sequence, etc. The main studied ones are (Se02):
• Prediction problem: given a pattern [(a1 , . . . , an ), R], we
want to predict the object an+j , j > 0 such that (∀i ∈
1..n) ai < an+1 . When j=1, we make predictions based
only on the immediately preceding object;
• Sequence generation. This task has the same formulation as the prediction problem but consider the following
situation: given a pattern [(a1 , . . . , an ), R], we want to
generate an object an+j such that (∀i ∈ 1..n) ai < an+1 ;
• Sequence recognition. This problem consists in verifying
the consistency of a given pattern [(a1 , . . . , an ), R]. The
response will be either the pattern is valid or not, according to a set of pre-defined criteria.
Example 3 To illustrate the differences between the prediction and the recognition problems, let us consider the following sequence: (a1 , a2 , a1 , a2 , a1 , a3 , a1 , a1 ). If we consider the recognition problem with a frequency evaluation
function (see section ) and a threshold=2 then the response
for the question: find all the maximal frequent subsequences
will be the sequence (a1 , a2 ). If the question is the following prediction question: if we observe object a1 which
is the possible immediate observation, the response will be
a1 ∨ a2 ∨ a3 . In other word we can not predict anything.
To reason about STCSP, we need to define different kinds
of objects. Our first work is to define the following classes:
• Qualitative point reasoning about patterns. In this case
only Vilain and Kautz (VK86; VKV90) point relations are
defined between objects;
• Quantitative point reasoning about patterns;
• Qualitative intervals reasoning about patterns. In this case
interval algebra is used;
• Quantitative point reasoning about patterns with interval
of occurrence. In this case objects are temporal points but
the occurrence period is an interval;
• Quantitative intervals reasoning about patterns.
Note that it is possible to consider any combination of these
cases.
The second consideration is the definition of the evaluation function. This function depends on the considered goal.
The most used one is the sequence frequency. But, there exists several other evaluation functions like prediction function, similarity function.
When evaluation functions are used, different types of
constraints may be used to select significant sequences. For
instance, the most used constraint for frequency evaluation
function is a fixed threshold. In a given application we
consider that a sequence is frequent if and only if the
number of its occurrence is greater then an a priori fixed
threshold.
For each considered object expressed in a given language
and for each given evaluation function with predefined constraints, there exists a set of solvers or algorithms to solve
considered problems. The next section gives the generic
model to describe sequential patterns.
STCSP model
We define a Sequence Temporal Constraint Satisfaction
Problem (STCSP) as a 5-uplet
M =< O, L, f, C, S >
Where:
O: describes the considered objects (for example, qualitative/quantitative points and/or intervals). In this paper we
define O as a set of temporal points;
L: defines the considered sequence language. For the qualitative point sequences case, we use Vilain and Kautz point
algebra. A sequence will be defined as a couple (vector of
objects, matrix of relations). Each element of the matrix
will be a disjunction of the following qualitative relations
between points: {<, >, =}. The algebraic operations in
the point algebra are defined in (VKV90).
We will use in this paper the following notations:
(a1 , a2 , . . . , an ) indicates a sequence a1 < a2 < . . . <
an . {a1 , a2 , . . . , an } indicates that for all event ai and
aj , if ai is the same event as aj then if i < j then
ai < aj , otherwise ai may have any other relation with
aj . [(a1 , . . . , an ), R] defines any pattern (a1 , . . . , an )
such that:
• if i < j and ai is the same event as aj then ai < aj ;
• R defines the matrix relations between objects ai for
i ∈ 1..n.
f : f defines the evaluation function. The evaluation functions define the applicable operations on the temporal sequences. The most used one is the computation of the
sequences frequency:
f (M, M1 ) defines the frequency of the subFor
sequence M1 in the sequence M .
=
2.
instance, f ( (a1 , a2 , a1 , a2 ), a1 )
f ( (a1 , a2 , a1 , a2 ), (a1 , a2 )) = 3 if we accept
redundancy, and f ( (a1 , a2 , a1 , a2 ), (a1 , a2 )) = 2
otherwise.
Other useful functions in sequential patterns reasoning
may be defined. For instance, we can consider the ”prediction function” defined as follows:
f (M, M1 , n) defines what sequence with the length n
we can predict in the sequence M after the observation
M1 .
For example, f ( (a1 , a2 , a3 , a1 , a2 , a3 ), (a1 , a2 ), 1) =
a3 .
C defines the constraints applied to defined functions. C is
used to select the significant patterns. For instance, if we
consider the function computing the frequency, the constraint C may contains fixed or dynamic threshold which
indicates that the sequence is frequent or no.
S: Solver. There is at least one solver for each evaluation
function. S defines algorithms applied on the evaluation
function.
Definitions and properties
In this section we will introduce some fundamental concepts. Some concepts are not completely defined because
they depends on the kind of model M. First, we present
poset, which is the basic concept to reason about sequences
then we introduce some relations and some operations on
sequences.
Poset : partially ordered sets
A partially ordered set (A, ) consists of a nonempty set A
and a binary relation on A such that satisfy properties
(P 1)−(P 3). Sets equipped with a relation such are called
partially ordered sets (poset).
For all s,p,q, we have:
(P1) Reflexivity: s s
(P2) Antisymmetry: s p and p s imply that s = p
(P3) Transitivity: s p and p q imply that s q
(P4) Linearity: for all p, q ∈ S p q or q p
When elements are not comparable, we use the symbol k to
denote the non-comparability; we write s k p iff s 6 p and
p 6 s.
Let S be an ordered set. S is a sequence if, for all p, q ∈ S,
p and q are comparable. An alternative name is a linearly
ordered set (property (P4)).
Lemma 1 With the induced order, any subset of a sequence
is a sequence.
Relations between sequences
Order-isomorphisms This property is used to determine
when two ordered sets are the same. Two poset S and
P are order-isomorphic denoted P ∼
= S if there exists a
map φ from P onto S such that x y in P if and only if
φ(x) φ(y) in S. Then φ is called an order-isomorphism.
φ is necessary bijective 1 : using reflexivity and antisymmetry of in Q then in P, φ(x) = φ(y) ⇔ φ(x) φ(y) and
φ(y) φ(x) ⇔ x and y x ⇔ x = y
Lemma 2 On the other hand, not every bijective map between ordered sets is an order-isomorphism.
Theorem 1 Let S be a set of objects and R a set of relations
on S. If objects of S are comparables with R objects and R
is a linear order set then (S, R) is a linear order.
Example 4 if we consider S as a qualitative points, R =
{<, =, >} as possible relations between objects of S, and
(topologic relation) linear order defined between elements
of R, then (S, R) is a linear order.
Definition 1 (Subsequence) A partial order set S is a subsequence of a poset P, denoted S ⊆ P , if
• there exists an injective application φ from P onto S such
that if x y in S then φ(x) φ(y) in P;
• each object of S and P is represented by pair (identifier,
time). For all x = (ix , tx ), φ(x) = (ix , φ(tx )).
Example 5 if we consider the model presented in section ,
then [(a, c), (<, <)] ⊆ [(a, b, c, a), (<, =, <, <)]
The subsequence concept is very important to deal with
sequences but this concept is still ambiguous. According to
the considered model M and the kind of application, several constraints maybe added to refine the definition of the
subsequence. The most used constraints are: subsequences
without gap or with bounded gaps.
• subsequence without gap: S v P if S ⊆ P , and for all
x, y ∈ S such that x y, if 6 ∃z ∈ S such that x z y then 6 ∃t ∈ P such that φ(x) t φ(y). In
the case where is a poset (= {α1 , . . . , αn }), the
last condition becomes: x αi y, if 6 ∃z ∈ S such that
x αj z αk y then 6 ∃t ∈ P such that φ(x) αl t αm φ(y)
1
It’s inverse φ−1 is also an order-isomorphism.
for all i, j, k, l, m ∈ {1..n} (gaps in objects and gaps in
relations are prohibited).
• subsequence with bounded gap: the bounds formulation
depends on the type of the considered model M. For instance, if we consider qualitative points model then possible bound maybe the maximal number of objects in the
sequence P separating two contiguous objects in the sequence S: for all x, y ∈ S such that x y, if 6 ∃z ∈ S
such that x z y then there exists at the most
ti,i∈{1,...,θ} ∈ P such that φ(x) ti φ(y).
Definition 2 (order-isomorphic subsequence)A partial order set S is an order isomorphic subsequence of a poset
P, denoted S ∼
=⊆ P , if there exists a poset S 0 such that:
S∼
= S 0 , and S 0 ⊆ P .
The order-isomorphic subsequence is useful, for instance,
to express that a sequence S is quasi-subsequence of a sequence P : if there exists a similarity function between objects of S ∪ P , then φ maybe used to substitute similar objects. This substitution must guaranty that S is similar to S 0
and S 0 ⊆ P , in which case S becomes quasi-subsequence
of P .
Occurence This relation, denoted C(s, s1 ), gives the number of all possible instances of the sequence s1 in the sequence s. For instance, if we consider the quelitative point
model, C( (a, b, c, b, a), (a, b) ) = 2.
It also useful to define the number of all possible instances of
the sequence s1 in the sequence s without redundency. It’s
denoted C⊥ (s, s1 ) and it is mean that each object of s appear
at most in one instance s1 . For instance, if we consider the
quelitative point model, C⊥ ( (a, b, c, b, a), (a, b) ) = 1.
Similarity between sequences This relation, denoted S,
quantify the similarity between pair of sequences. This comparison concerns the local structure of considered primitives
(it’s studied in many domains (Bis95)), and the relational
structure between them. The function S
S: sequences X sequences→ [0..1]
(s1 , s2 ) 7→ (s1 , s2 )
has the following properties:
• S(s, s) = 1;
• antisymetric. From geometric model point of view, the
similarity relation is usually symmetric, assume minimality (S(A, B) ≥ S(A, A) = 0) and the triangle inequality (S(A, B) + S(B, C) ≥ S(A, C)). But, we use a
more general meaning of this word (Tve77). Tversky
criticized geometric models on the grounds that violations of all three assumptions are empirically observed
(WFCK99).One alternative formula proposed by Tversky
is S(A, B) = θf (A∩B)−αf (A−B)−βf (B −A). The
similarity is defined as a linear combination of the common and distinctive features. The terms α, β and θ reflect
the weights given to the common and distinctive features.
• Some works consider that S(s1 , s2 ) = 1 means : s1 is
perfectly included in s2 . If S(s1 , s2 ) = S(s1 , s2 ) = 1
then s1 is equivalent to s2 . In this case, the similarity
function can be seen as an extension of the subsumption
function.
Operations
Intersection: ∩. The intersection between sequences is
defined as follows: s1 ∩ s2 = s3 such that s3 ⊆ s1 and
s3 ⊆ s2 and there exists no s 6= s3 such that s3 ⊆ s, s ⊆ s1 ,
and s ⊆ s2 .
Other forms of intersections maybe defined according to the
type of the considered subsequence concept. For instance,
if we consider the subsequence without gap (v), then the
previous definition of the intersection will be the same by
replacing the standard subsequence(⊆) by the subsequence
without gap (v).
Example 6 Let us consider the qualitative point model:
• (a, b, c, a, f ) ∩ (d, b, h, t, a, d) = (b, a)
• = [(a, b, a, b), (<, <, <)] ∩ [(a, b, c, a, g), (=, <, <, =
)] = [(b, a), (<)],
• = [(a, b, c), (<, <)] ∩ [(f, a, b, g, c(<, =, =, =)] =
[(), ()],
Union: ∪. disjoint union is defined as follows:
s1 ∪s2 is the ordered set formed by defining x ⊆ y in s1 ∪s2
if and only if:
x, y ∈ s1 and x ⊆ y in s1 or
x, y ∈ s2 and x ⊆ y in s2 .
Linear sum: ⊕. Linear sum is defined as follows:
s1 ⊕s2 is the ordered set formed by defining x ⊆ y in s1 ⊕s2
if and only if:
x, y ∈ s1 and x ⊆ y in s1 or
x, y ∈ s2 and x ⊆ y in s2 or
x, ∈ s1 and y ∈ s2 and x ⊆ y.
∪ and ⊕ are associative.
Projection: π . Let us consider Ps1 (s) the ordered
set containing only all possible instances of s1 in s.
For instance, if we consider the qualitative point model,
P(a,b) (a, b, c, b, a) = (a, b, b).
The projection of s1 into s2 , denoted π(s1 , s2 ), if defined as
follows:
π(s1 , s2 ) = Ps2 (s1 )
Example 7 The projection of the sequence (a, b, c, b, a) in
the sequence (a, b) in the qualitative point model gives all
possible instances of (a, b) in the sequence (a, b, c, b, a):
π(a, b, c, b, a), (a, b)) = (a, b, b)
In some applications, we need also the projection without
redundancy π⊥ . Let us consider P⊥s1 (s) the ordered set
containing only all possible instances of s1 in s such that
each object of s appears at most in one instance s1 .
π⊥ (s, s1 ) = P⊥s1 (s)
For instance, π⊥ (a, b, c, b, a), (a, b)) = (a, b).
Composition: ◦. Let us consider C(s1 , s2 ) the common
part of the sequences s1 and s2 . For instance, if we consider
the maximal prefix matching between s1 = ((a, b, f, b), (<
, <, =, <)) and s2 = ((a, d, b, a, b, d), (<, <, <, <, =))
then C(s1 , s2 ) = (a, b, b) ((a,b,f,b) with (a, d, b, a, b, d)).
The composition operation between s1 and s2 is defined
as a composition relation between objets of s1 − C(s1 , s2 )
and s2 − C(s1 , s2 ) as defined in the model M with the
respect to the relations between objects defined in s1 and
s2 .
Example 8 If we consider the qualitative point model
(see section ), s1 ((a, b, f, b), (<, =, <), and s2 =
((a, d, b, a, b, d), (<, <, <, <, =)) then
Constraint Network
A qualitative point STCSP network of M
=<
O, L, f, C, S > is a multi-valuated graph G = (N, V )
where each node n ∈ N represents an object of O and V is
a set of couple (time point relation, position).
Example 9 Let us consider the following temporal pattern
[(a1 , a2 , a3 , a3 , a4 ), (=, <, <, =)]. Figure gives the corresponding STCSP network.
s1 ◦ s2 = ((a, d, b, f, a, b, d), (<, <, <, <, <, =))
NB : we use qualitative point composition table (VK86) with
the restrictions induced by the previous relations between
objects in s1 and s2 .
Qualitative point STCSP
In temporal patterns problems, the patterns database is the
main source of information shared between objects. This
situation is not the case with classical TCSP. In the STCSP
usually all relationships between objects are extracted from
an Apriori given instance of the problem. In fact, in this kind
of applications only simple relations are defined between objects. Let us consider the sequence [(a1 , . . . , an ), R] such
that ri,j is the relation between ai and aj .
To simplify the representation we may omit the relationship
in the arcs: an oriented arc corresponds to the relation ”<”.
The non oriented arc refers to the relation ”=” (see figure
(b) ).
Qualitative point STCSP algorithms
(∀i)(∀j) ri,j ∈ {<, =, >}
R is a symmetric matrix, where ri,i = {=}. If we organize
the pattern [(a1 , . . . , an ), R] such that:
(∀i) ri,i+1 ∈ {=, <}
Then:
Property 1 The definition of only the relations ri,i+1 in the
matrix R for the temporal pattern [(a1 , . . . , an ), R] is sufficient to generate all the atomic relations between objects.
Prove: For all i, if we know the relations between ai , ai+1
and ai+1 , ai+2 then the possible relations between ai , ai+2
will be: ri,i+2 = {=}, if ri,i+1 = {=} and ri+1,i+2 = {=},
ri,i+2 = {<} if ((ri,i+1 = {=} and ri+1,i+2 = {<}) or
ri,i+1 = {<} and ri+1,i+2 = {=})).
If we consider that the elements ri,i+1 (i≥0) of the matrix
R are defined then assume that if the relations rj,j+1 for
j ∈ i..i + n are atomic then ri,i+n is atomic. If the relations
ri,i+n and ri+n,i+n+1 are atomic then using the first part of
the prove the relation ri,i+n+1 exists and is atomic a.
Theorem 2 The definition of the matrix R, in the case of
qualitative points STCSP network, assures that the cloture
is consistent and that the cloture network is also an atomic
network.
The
representation
of
temporal
[(a1 , . . . , an ), R] will be simplified as
[(a1 , . . . , an ), (r1,2 , . . . , ri,i+1 , . . . , rn−1,n )]
Figure 1: The qualitative point TSCSP network.
pattern
follows:
The type of the selected algorithms depends on the considered application and the kind of problem to solve. Let us
consider the following model: M =( qualitative point, Vilain
and Kautz algebra, frequency evaluation function, threshold,
S).
In the addition to the problems presented in section , the
solver S of our given problem contains deux classes of algorithms: generic ones, like:
• checking the θ-consistency of a given pattern,
• checking the θn -consistency of the network,
• finding all solutions,
• finding minimal set of solutions.
used in most applications and specific ones related to
particular situations.
For specific algorithms let us consider the telecommunication network monitoring real problem. The telecommunication network equipments generate an infinite sequences of
alarms2 . These alarms arrive to the supervision center. One
considered problem is the detection of ”regular” sequences
of alarms received by the supervisor center when a known
breakdown situation is happened.
2
To simplify the problem, we define an alarm as an event in the
network received by the supervisor center identified by the alarm
name, equipement identifiers, and the time when the event is happening.
Definition 3 A pattern p = [(a1 , . . . , an ), R] is θconsistent in the network G if there exists at least θ instances
”without redundancy” 3 of the pattern p in G.
The algorithm allN extGen(G uses the point algebra
composition operation to generate for each value of all variables in the STCSP, possible successors).
Example 10 In the following network G = [(a, b, a, b), (<
, <, <)], the pattern p = [(a, b), (<)] is 2-consistant:
(a,b,a,b), and (a,b,a,b). But, if we consider the redundancy
there are three instances of p in G: (a,b,a,b), (a,b,a,b), and
(a,b,a,b).
Example 12 Let us consider the following network: G =
[(a1 , a2 , a3 , a3 , a4 ), (=, <, <, =)]. After the application of
the allN extGen() algorithm, the resulting network will be:
Ga = [(a1 , a2 , a3 , a3 , a4 ), (r1,2 = r3,4 = {=}, r3,3 = {<
}, r2,3 = {=}, r1,3 = r1,4 = r2,4 = {<})] (see figure 12).
Definition 4 A network G is θ-consistent if there is at last
one θ-consistent pattern p = [(a1 , . . . , an ), R] in the network G. G is θn -consistent if there exist at last one θconsistent pattern p of the length n in G 4 .
Example 11 In the following network G = [(a, b, a, b), (<
, <, <)], the pattern p = [(a, b), (<)] is 2-consistent. But it
is not 3-consistant. However, if we consider the redundancy
there are three instances of p in G.
Checking the θ-consistency algorithm A temporal
pattern s is θ-consistent on the network G if there are more
that θ instances of s in G such that the intersection between
instances of s is empty.
Figure 2: (a) Example of STCSP network (b) application of
the algorithm AllNextGen() on (a).
Algorithm 1 Checking the θ-consistency of a given pattern
Input :
The global network:
G = [(a1 , . . . , am ), (ra1,2 , dots, ram−1,m )]
s = [(b1 , . . . , bn ), (r1,2 , . . . , rn−1,n )]
s is θ-consistent?
Output :
begin
1 result = 0;
2 Ga = AllNextGen(G);
// for all values of variable computes the possible successors
3 currentNodeG= s1 ;
// (such that aj = s1 ) starts the generation from the heat of s
4 treeSolutionsRoot = VirtualRootNode;
//TreeSolution includes the tree of potential solutions
5 treeSolutions.add(virtualNode,s1 );
6 currentSolution =s1 ;
7 currentPositionInGa = ai ;
8 while ((result 6= θ) and (currentNodeG has next Node)) do
9
for all next node nd of currentPositionGa do
10
if ((currentNodeInGa, nd)=(si , si+1 ))
11
treeSolutions.add(si , si+1 );
12 if (i=n)
13
result++;
14
for (j=n downto 1) do
15
treeSolutions.delete(sj );
16
if (the node (sj−1 , s1 ) exists)
treeSolutions.add(virtualRoot, s1 );
17
treeSolutions.delete(sj , a) for all a 6= s1 ;
18 if (result= θ) return (s is θ-consistent in G; // (end)
19 if (currentNodeG has not next Node)
20 return (s is θ-inconsistent in the G; // (end)
EndAlgorithm
3
Each label of each arc of G appears in at most in one instance
of the pattern p.
4
The pattern length is the number of objects instances in the
pattern.
Finding θ-solutions algorithm This algorithm generates
all possible θ-solutions of the network. We propose an
incremental algorithm which first computes arc θ-solutions,
then path θ-solutions, then 4th θ-solutions, etc.
Definition 5 A network G is arc θ−consistent if there is at
less one arc with θ labels.
Definition 6 A network G is path θ−consistent if there
is at less one sequence s of the length 3 such that s is
θ−consistent in G.
Definition 7 A network G is n θ−consistent if there is at less
one sequence s of the length n such that s is θ−consistent in
G.
The proposed finding θ−solutions algorithm uses the
following properties: (1) the trivial one is if the temporal
pattern s is θ−consistent then all subpatterns of s are also
θ−consistent, (2) if we have all θ−consistent solutions of
the length n then solutions of the length n + 1 may be
matching sequences of length n sharing the same prefix
subsequence of the length (n-1). The same propriety is
applied in Apriori algorithm proposed in (AS95).
Algorithm 2 Finding θ-solutions
Input :
network G
Output : all θ− solutions
begin
1 solutions = ∅ ; stepSolutions =0; l =1 ;
2 for all arc in G if (θ ≤ nbLabels of (si , sj , l) then
3 solutions =solutions ∪(si , sj , l); stepSolutions++;
4 while (stepSolutions 6= 0) do
5 stepSolutions =0; l++;
6 for all couple (si , sj ) such that
si = ((a1 , . . . , al−1 , a), (r1,1 , . . . , r1,l−1 , ra ))and
sj ((a1 , . . . , al−1 , b), (r1,1 , . . . , r1,l−1 , rb ))
check the θ− consistency of the sequences
s = ((a1 , . . . , al−1 , a, b), (r1,1 , . . . , r1,l−1 , ra , r))
such that r = {<, =, >}
8
for each case if s if θ−consistent then
9
solutions=solutions ∪s;
10
stepSolutions++;
EndAlgorithm
7
checking the minimal θ-consistency algorithm The previous algorithm for checking the θ− consistency may be
used with the following operation: after the line (9) we add
solutions = solutions/{si , sj }. Both of them may be generated from the solution s.
checking the θn -consistency algorithm A network G is
θn -consistent if there exists a θ-consistent sequence s of the
length n.
It’s possible to propose a more efficient algorithm. We can
also us the finding θ− solutions algorithm and stop the computation when the value of l is n.
Conclusion
This paper presents a new formulation of the sequential
pattern problem using constraint satisfaction problems formalism. A classification of the main kinds of considered
objects is presented. A brief introduction is proposed for
the qualitative point STCSP, and a network representation
is given. Some problems and associated resolution algorithms are proposed. This work is still in progress, and raises
many questions especially how to manage and which kind of
knowledge we can explore when we consider systems where
the values of variables change with time.
References
P. Agre and D. Chapman. What are plans for? In P.
Maes, ed. Designing Autonomous Agents. Elsevier, New
York, 1990.
James F. Allen. Maintaining knowledge about temporel
intervals. Communications of the ACM, 26(11):832–843,
1983.
R. Agrawal and R. Srikant. Mining sequentiel motifs.
In International Conference on Data Engineering (ICDE),
Taipei, Taiwan. Expanded version is available as IBM Research Report RJ9910, October 1994, 1995.
N. Lavrac an S. Dzeroski. Inductive Logic Programming.
Ellis Horword, New York, 1994.
L. E. Baum. An inequality and associated maximization
technique in statistical estimation for probabilistic functions of a markov process. In Inequalities, pages 1–8, 1972.
D.J. Berndt and J. Clifford. Finding patterns in time series: A dymanic programming approach. In Advances in
knowdelge Discovery and Data Mining. U. M. Fayyad, G.
Pietesky-Shapiro, P.Smyth, & R.Uthurusamy, editors. MIT
Press, 1996.
R. Bellman. Dynamic programming. In Princeton University Press, Princeton, NJ., 1957.
G. CASELA ET R.L. BERGER. Statistical Inference.
Brooks/cole, Pacific Grove, CA, 1990.
G. Bisson. Why and how to define a similarity measure for
object based representation systems, 1995.
Z. Bar-Joseph, G. Gerber, D.K. Gifford, , and T.S.
Jaakkola. A new approach to analyzing gene expression
time series data. In 6th Annual Int. Conf. on Research in
Comp. Molecular Biology, 2002.
D. Bertsekas and J. Tsitsiklis. Neuro-dynamic programming. Athena Scienti c, Belmont, MA., 1996.
T. V. Duong C. Dousson. Discovering chronicles with numerical time constraints from alarm logs for monitoring
dynamic systems. In IJCAI, pages 620–626, 1999.
R. Dearden and Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence,
89:pp.219–283., 1997.
C. Dousson and M. Ghallab. Suivi et reconnaissance de
chroniques. revue francaise dintelligence artificielle, 8:29–
61, 1994.
D.J.ZAKY. On spade:an efficient algorithm for mining frequent sequences. Machine Learning, 42:31–60, 2001.
I. Guyon et al. Storage and retrieval of complex sequences
in neural networks. Physical Review A,, 38:6365–6372,
1988.
Y. Fujiwara, M. Asogawa, and Kenta Nakai. Prediction of
mitochondrial targeting signals using hidden markov models. Genome Informatics, 1997.
P. Frasconi, M. Gori, and G. Soda. Recurrent neural
networks and prior knowledge for sequence processing.
Knowledge Based Systems, 8(6), 1995.
C.L. Giles, B.G. Horne, and T. Lin. Learning a class of
large unite 435 395 456 408 state machines with a recurrent
neural network. In Neural Networks 8 (9), pages 1359–
1365, 1995.
Bernard Hugueney and Bernadette Bouchon-Meunier.
Time-series segmentation and symbolic representation
from process-monitoring to data-mining. In CI01, LNCS
volume 2206, pages 118–123, 2001.
F. Hoppner. Discovery of temporal patterns - learning rules
about the qualitative behaviour of time series. In Proc. of
the 5th European Conference on Principles and Practice
of Knowledge Discovery in Databases, Lecture Notes in
Artificial Intelligence 2168, Springer. Freiburg, Germany,
pages 192–203, Sept 2001.
F. Hoppner. Learning dependencies in multivariate time
series. In Proc. of the ECAI’02 Workshop on Knowledge
Discovery in (Spatio-) Temporal Data, Lyon, France, pages
25–31, 2002.
T.K. Jenssen, L.M. berg, M.L. Andersson, and J. Komorowski. Methods for large-scale mining of networks of
human genes. In Proceedings of the SIAM Conference on
Data Mining. Chicago, IL, USA,, 2001.
S.D. Johnson, D.S. Battisti, and E.S. Sarachik. Empirically
derived markov models and prediction of tropical sea surface temperature anomalies. In Journal of Climate, 1999.
L.R. RABINER ET B.H. JUANG. Fundamentals of
Speech Recognition. Prentice Hall, Englewood Cliffs, New
Jersey, 1993.
D. Kleinfeld and H. Sompolinsky. Associative neural network models for the generation of temporal patterns. Biophysics Journal, 54:1039–1051, 1988.
B. Kuipers. Qualitative Reasoning: Modeling and Simulation with Incomplete Knowledge. Artificial Intelligence.
MIT Press, 1994.
V. Kumar. Algorithms for constraints satisfaction problems: a survey. AI Magazine, 13 1::32–44, 1992.
L.TERRAN and E. CARLA. Temporal sequence learning
and data reduction for anomaly detection. 5th ACM Conference on Computer and Communications Security, San
Francisco, California, USA, 3-5:150–158, 1998.
L. Lin. Self-improving reactive agents based on reinforcement learning, planning, and teaching. Machine Learning,
8:293–321, 1992.
L. Masotti M.A. Andrade, R. Casadio. Tools for automated
protein annotation protein sequence analysis. In in the Genomic Era. Bologna (Italia): CLUEB, 2001.
L. Meeden. An incremental approach to developping intelligent neural network controllers for robots. In Adaptive
behavior, 1995.
Katsuhiro Moizumi. The mobile agent planning problem.
PhD thesis, Thayer School of Engineering, Dartmouth College, November 1998.
B.M. OPPER and D. HAUSSLER. Worst case prediction
over sequences under log loss. In The Mathematics of Information Coding, Extraction and Distribution, Springer
Verlag, 1997.
T. Ono, H. Hishigaki, A. Tanigami, and T. Takagi. Automated extraction of information on proteinprotein interactions from the biological literature. Bioinformatics,
17:155–161, 2001.
A. Osmani and F. Lévy. A constraint-based approach
to simulate fault in telecommunication networks. In
(IEA/AIE-2000), 2000.
A. Osmani, L. Rozé, M.O. Cordier, P. Dague, F. Lévy, and
E. Mayer. Supervision of telecommunication networks. In
(ECC-99), 1999.
A. Osmani. Diagnostic à base de modèles pour la supervision de pannes dans un réseau de télécommunications &
raisonnement sur les intervalles. PhD thesis, Laboratoire
LIPN, Institut Galilée, Université de Paris XIII, 1999.
A. Popescul, G.W. Flake, S. Lawrence, L. Ungar, and C.L.
Giles. Clustering and identifying temporal trends in document databases. In IEEE Advances in Digital Libraries
Conference, Washington, D.C., May 2000.
V. Pillet.
Méthodologie d’extraction automatique
d’information à partir de la littérature scientifique en
vue d’alimenter un nouveau système d’information. Application à la génétique moléculaire pour l’extraction
d’information sur les interactions. PhD thesis, Science de
l’information et de la communication, 2000.
D. M. Proux. une stratégie d’extraction d’informations
dans des corpus spécialisés par application de méthodes
d’analyse linguistique de surface et de représentation conceptuelle des structures sémantiques. PhD thesis, faculté
des sciences et techniques, université de Bourgogne, 2001.
M. Riese. Model-Based Diagnosis of communication protocols. PhD thesis, Swiss Federal Institute of Technology,
Lausanne, 1996.
Simeon J. Simoff et Ingrid Slembek Robert P. Biuk-Aghai.
Knowledge-assisted reverse engineering of virtual work
processes. UPGRADE Vol. III, No. 1,, 2002.
R. Sutton and A. Barto. Reinforcement Learning. MIT
Press, Cambridge, MA., 1997.
R. Sun and L. Gilles editors. Introduction to sequence
Learning. (a completer), 2002.
A. Stentz, M. Ollis, S. Scheding, H. Herman, C. Fromme,
J. Pedersen, T. Hegadorn, R. McCall, J. Bares, and
R. Moore. Position measurement for automated mining
machinery. In Proceedings of the International Conference on Field and Service Robotics, pages 299–304, August 1999.
R R. Sun and T. Peterson. Autonomous learning of sequential tasks: experiments and analyses. IEEE Transactions on
Neural Networks, 6(9):1217–1234, 1998.
R. Sun and T. Peterson. Multi-agent reinforcement learning: weighting and partitioning. Neural Networks, 12(45):127–153, 1999.
Eddie Schwalb and Lluı́s Vila. Temporal constraints:
A survey. Constraints: An International Journal, 3(2–
3):129–149, 1998.
(ed.) S. Wermter et al. Connectionist and Statistical Approaches to Natural Language Processing. Springer, Heidelberg, Germany, 1996.
T. Tesauro. On practical issues in temporal difference
learning. Machine Learning, 8:257–277, 1992.
E. Tsang. Foundations of constraint satisfaction. Academic
press, Essex, 1993.
S. Tversky. Features of similarity. Psychological Review,
84:327–352, 1977.
M. Vilain and H. Kautz. Constraint propagation algorithms
for temporal reasoning. pages 377–382, 1986.
M. Vilain, H. Kautz, and P. Van Beek. Constraint propagation algorithms for temporal reasoning: A revised report.
In D. S. Weld and J. de Kleer, editors, Readings in Qualitative Reasoning about Physical Systems, pages 373–381.
Kaufmann, 1990.
Robert A. Wilson and (ed.) Franck C. Keil. The MIT Encyclopedia of the Cognitive Sciences. MIT Press, 1999.