A Subsequence Matching with
Gaps-Range-Tolerances Framework:
A Query-By-Humming Application
Alexios Kotsifakos
University of Texas at Arlington, University of Athens (Greece)
Joint work with
Panagiotis Papapetrou (Aalto University, Finland)
Jaakko Hollmen (Aalto University, Finland)
Dimitrios Gunopulos (University of Athens, Greece)
Outline

Motivation

Related Work

Problem Setting

SMBGT: The proposed method

Experimental Evaluation

Summary and Conclusions
2
Introduction – Motivation
Suppose you hear a song but cannot recall its name…

One solution: Hum a short part of it and perform a search
on a music repository to find it or songs similar to it.

Query-By-Humming (QBH): Given a hummed query song
search a music db for the K most similar songs.

Subsequence Matching: Given a query sequence and
a db of sequences, identify the subsequence in db that
best matches the query.
3
Subsequence Matching in a Database
query
What subsequence of any
database sequence is the
best match for Q ?
database
4
Converting music pieces to time series

Each music piece is a sequence of notes.
Part of the Happy Birthday music score.

Notes consist of pitch (frequency) and duration.
The 2D time series
representation of the
above music score.
Pitch interval : distance
between two pitches.
IOIR : duration ratio of
two consecutive notes
5
QBH: A real example
Hummed query:
“Yesterday”
(Beatles)
Target:
“Yesterday”
QBH: A real example
Hummed query:
“Yesterday”
(Beatles)
Matched
target
subsequence
QBH: A noisy application domain

Slight or more serious errors due to instant or temporary key or
tempo loss  error-tolerant method.

Allow skipping a bounded number of elements in both query
and target sequences due to serious humming errors.

Constrain the length of the matching subsequence.

Optional constraint to the min. number of matching elements.
8
Our contributions

A subsequence matching framework that allows for




a constrained number of gaps on both query and target sequences,
variable tolerance levels in the matching (no probabilistic model),
a matching range that constrains the max. match length,
bounding the min. # of matched elements.

SMBGT: Given a query Q and a target sequence X (with |Q| <<
|X|) finds the subsequence of X that best matches Q,
considering the framework.

Extensive comparative evaluation on QBH of several DB-based
methods and a probabilistic model-based method.
9
Outline

Motivation

Related Work

Problem Setting

SMBGT: The proposed method

Experimental Evaluation

Summary and Conclusions
10
Related Work

DP-based methods





Whole sequence matching: DTW and variants (cDTW, EDR, ERP)
robust to misalignments and warps, LCSS allows for gaps, edit
distance, Iliopoulos et al., etc.
Subsequence matching: SPRING finds the subsequences of evolving
numerical streams closest to a query, DTWc and DTWs account for
local adjustments of tempo, Han et al. (2007) performed uniform
segmentation and sliding windows for near exact matching where
user selects segments’ length.
n-gram methods for near exact matching
All account for 1D + fail to handle noise imposed by users.
Probabilistic-based methods (HMMs)
11
Outline

Motivation

Related Work

Problem Setting

SMBGT: The proposed method

Experimental Evaluation

Summary and Conclusions
12
Problem Setting: Representing Musical Pieces

Expressing pitch



Absolute pitch: frequency of the note.
Pitch interval: frequency difference between two adjacent notes.
Encoding duration



Inter-Onset-Interval (IOI): difference in time onsets of two adjacent notes.
IOI Ratio (IOIR): ratio of IOIs of two adjacent notes.
Log IOI Ratio (LogIOIR): logarithm of IOIR.

To save computational time we deal with note transitions:
<pitch interval, IOIR>, <pitch interval, LogIOIR>

Performed quantizations on pitch interval with modulo 12, and on
LogIOIR to the closest integer or closest value in [-2, 2].
13
Problem Setting: Definitions


X = {x1, …, xn} representing music piece of length |X|, x j  x jp , x rj  X
pair of real values for pitch and duration, DB={X1, …, XN }music db,
X[ts : te]={xts , …, xte} subsequence of X (not always continuous), Q = {q1, …, qn }.
Definition 1 (Variable error-tolerant match):
qi  Q and x j  X match with variable  -tolerance, qi f x j , if, for a set of
constraints  f  { pf ,  r f } :  pf ( i )  f p ( qip ) and  rf ( i , j )  f r ( qir , x rj ).
14
Problem Setting: Definitions

Definition 2 (Common bounded-gapped subsequence):
Consider Q[ ts 1 : te1 ] and X [ ts 2 : te 2 ] of equal length. Let G Q , GX the indices of
those elements in Q and X that are included in the corresponding subsequences.
If q i  x  i ,  i  G Q ,  i  GX , i  1,...,|G Q |, and π i+1 -π i   ,  i+1 -γ i   ,
then pair { Q[ ts 1 : te1 ], X [ ts 2 : te 2 ]} defines a common gapped subsequence of Q and X .
The longest such subsequence with te 2 -ts 2  r is called SMBGT ( Q , X ).
SMBGT: error-tolerant matching denoted as ε-match.
15
Problem Setting:
Example and Problem Formulation

Example



α = 2, β = 1, r = 6, ε = 1 (absolute tol.)
Q = {6, 3, 10, 5, 3, 2, 9}, X = {1, 1, 3, 4, 6, 9, 2, 3, 1}
Q[2:6] with GQ = {2, 4, 5, 6} and X[3:8] with GX = {3, 4, 7, 8}
Q = {6, 3, 10, 5, 3, 2, 9}
SMBGT(Q, X) =
{Q[2:6], X[3:8]}
X = {1, 1, 3, 4, 6, 9, 2, 3, 1}
Problem: Given a database DB with N sequences of arbitrary lengths,
a query sequence Q, and positive integers δ and r, find set
S  { X i [ts : te ] | X i  DB} of the top-K subsequences with
| SMBGT (Q, X i [ts : te ]) |  .
16
Outline

Motivation

Related Work

Problem Setting

SMBGT: The proposed method

Experimental Evaluation

Summary and Conclusions
17
SMGT


SMGT: α = β = inf.
Array a (|Q+1|)*(|X|+1). For all i in {1,…,|Q|}, j in {1,…,|X|}
a 0, j  0, a i ,0  0
ai , j




f

a

1,
if
q

 i 1, j 1
i
 xj


max{a i 1, j , a i , j 1}, otherwise.
Matrix s keeps in si,j for each ai,j the start point of its best alignment.
Online computation: two 1D arrays prev, cur, track scores and start points
(prev.value, cur.value, prev.start, cur.start) of j-1 and j columns of a.
Best solution (SMGT) in best = (bestvalue, beststart, bestend) accounting for δ.
Check subsequences with length r (Reset()). Update values:
 Match  ai,j = 1
 Mismatch  left cell can be inherited if length < r-1, top cell always.
18
SMBGT

Mismatch (propagation()): store the largest # of matched
elements that can be propagated vertically or horizontally,
not violating α and β.
prev
j  Astart
( i )   (left propagation)
cur
i  Bstart
( i  1)   (top propagation)


Astart , Bstart : store the latest match positions in X and Q
Additional propagation check in Reset().
19
SMBGT: Example - Initialization
Q = {0, -4, 1, 2, -2}, X = {0, 0, -4, 3, 0, 2, -3, 1}
α = 2, β = 1, δ = 3, r = |Q| = 5, ε = 0
Alignment array α
X
0
0
-4
3
0
2
-3
1
Q
0
0
0
0
0
0
0
0
0
0
0
-4
0
1
0
2
0
-2
0
20
SMBGT: Example - Computation
a
X
Q
0
0
Astart
X
Q
0
1
0
1
-4
1
-4
1
1
0
1
0
2
0
2
0
-2
0
-2
0
0
-4
3
s
0
2
-3
1
X
Q
0
0
1
0
1
-4
1
-4
1
1
0
1
0
2
0
2
0
-2
0
-2
0
3
3
0
2
-3
1
0
2
-3
1
X
0
-4
-4
Bstart
Q
0
0
0
-4
3
0
2
-3
1
21
SMBGT: Example - Computation
a
X
Q
0
0
0
1
-4
Astart
X
Q
0
0
1
0
1
2
1
1
-4
1
2
1
0
0
1
0
0
2
0
0
2
0
0
-2
0
0
-2
0
0
-4
3
s
0
2
-3
1
X
Q
0
0
0
0
1
2
0
1
1
-4
1
2
-4
1
1
1
0
0
1
0
0
2
0
0
2
0
0
-2
0
0
-2
0
0
0
2
-3
1
0
2
-3
1
X
0
3
3
Bstart
Q
-4
-4
-4
3
0
2
-3
1
22
SMBGT: Example - Computation
a
X
Q
0
0
-4
0
1
1
-4
1
1
Astart
X
Q
0
0
-4
1
0
1
2
2
1
2
-4
1
2
3
0
0
2
1
0
0
3
2
0
0
0
2
0
0
0
-2
0
0
0
-2
0
0
0
3
s
0
2
-3
1
X
Q
0
0
-4
0
1
2
-4
1
1
Bstart
0
0
-4
2
0
1
1
1
2
2
-4
1
1
2
0
0
2
1
0
0
2
2
0
0
0
2
0
0
0
-2
0
0
0
-2
0
0
0
0
2
-3
1
0
2
-3
1
X
Q
3
3
3
0
2
-3
1
23
SMBGT: Example - Computation
a
X
Q
0
0
-4
3
0
1
1
1
-4
1
1
1
0
2
-2
Astart
X
Q
0
0
-4
3
1
0
1
2
2
2
2
2
-4
1
2
3
3
0
2
2
1
0
0
3
3
0
0
0
0
2
0
0
0
0
0
0
0
0
-2
0
0
0
0
s
0
2
-3
1
X
Q
0
0
-4
3
0
1
2
2
-4
1
2
1
0
2
-2
Bstart
0
0
-4
3
2
0
1
1
1
1
2
2
-4
1
1
2
2
0
2
2
1
0
0
2
2
0
0
0
0
2
0
0
0
0
0
0
0
0
-2
0
0
0
0
2
-3
1
2
-3
1
X
Q
0
0
0
2
-3
1
24
SMBGT: Example - Computation
a
X
Astart
Q
0
0
-4
3
0
0
1
1
1
1
-4
1
1
2
1
0
0
2
0
-2
0
X
Q
0
0
-4
3
0
1
0
1
2
2
2
5
2
2
-4
1
2
3
3
3
2
2
2
1
0
0
3
3
3
0
0
0
0
2
0
0
0
0
0
0
0
0
0
-2
0
0
0
0
0
s
2
-3
1
X
Bstart
Q
0
0
-4
3
0
0
0
-4
3
0
0
1
2
2
2
5
0
1
1
1
1
1
-4
1
2
2
2
2
-4
1
1
2
2
2
1
0
0
2
2
2
1
0
0
2
2
2
2
0
0
0
0
0
2
0
0
0
0
0
-2
0
0
0
0
0
-2
0
0
0
0
0
-3
1
-3
1
X
Q
2
2
2
-3
1
25
SMBGT: Example - Computation
a
X
Astart
Q
0
0
-4
3
0
2
0
1
1
1
1
1
-4
1
1
2
2
1
0
0
2
2
0
0
-2
0
0
X
Q
0
0
-4
3
0
2
1
0
1
2
2
2
5
5
2
1
-4
1
2
3
3
3
5
2
2
0
1
0
0
3
3
3
0
0
0
0
3
2
0
0
0
0
0
6
0
0
0
3
-2
0
0
0
0
0
6
s
-3
1
X
Bstart
Q
0
0
-4
3
0
2
0
0
-4
3
0
2
0
1
2
2
2
5
5
0
1
1
1
1
1
1
-4
1
2
2
2
2
5
-4
1
1
2
2
2
1
1
0
0
2
2
2
0
1
0
0
2
2
2
0
2
0
0
0
0
0
2
2
0
0
0
0
0
4
-2
0
0
0
0
0
2
-2
0
0
0
0
0
4
1
1
X
Q
-3
-3
-3
1
26
SMBGT: Example - Computation
a
X
Astart
Q
0
0
-4
3
0
2
0
1
1
1
1
1
-4
1
1
2
2
1
0
0
2
2
0
0
-2
0
0
X
Q
0
0
-4
3
0
2
1
0
1
2
2
2
5
5
2
1
-4
1
2
3
3
3
5
2
2
0
1
0
0
3
3
3
0
0
0
0
3
2
0
0
0
0
0
6
0
0
0
3
-2
0
0
0
0
0
6
s
-3
1
X
Bstart
Q
0
0
-4
3
0
2
0
0
-4
3
0
2
0
1
2
2
2
5
5
0
1
1
1
1
1
1
-4
1
2
2
2
2
5
-4
1
1
2
2
2
1
1
0
0
2
2
2
0
1
0
0
2
2
2
0
2
0
0
0
0
0
2
2
0
0
0
0
0
4
-2
0
0
0
0
0
2
-2
0
0
0
0
0
4
1
1
X
Q
-3
-3
-3
1
27
SMBGT: Example - Computation
a
X
Astart
Q
0
0
-4
3
0
2
0
1
1
1
1
1
-4
1
1
2
2
1
0
0
2
2
0
0
-2
0
0
X
Q
0
0
-4
3
0
2
1
0
1
2
2
2
5
5
2
1
-4
1
2
3
3
3
5
2
2
0
1
0
0
3
3
3
0
0
0
0
1
2
0
0
0
0
0
6
0
0
0
3
-2
0
0
0
0
0
6
s
-3
1
X
Bstart
Q
0
0
-4
3
0
2
0
0
-4
3
0
2
0
1
2
2
2
5
5
0
1
1
1
1
1
1
-4
1
2
2
2
2
5
-4
1
1
2
2
2
1
1
0
0
2
2
2
0
1
0
0
2
2
2
0
2
0
0
0
0
0
6
2
0
0
0
0
0
4
-2
0
0
0
0
0
2
-2
0
0
0
0
0
4
1
1
X
Q
-3
-3
-3
1
28
SMBGT: Example - Computation
a
X
Astart
Q
0
0
-4
3
0
2
0
1
1
1
1
1
-4
1
1
2
2
1
0
0
2
2
0
0
-2
0
0
X
Q
0
0
-4
3
0
2
1
0
1
2
2
2
5
5
2
1
-4
1
2
3
3
3
5
2
2
0
1
0
0
3
3
3
0
0
0
0
1
2
0
0
0
0
0
6
0
0
0
1
-2
0
0
0
0
0
6
s
-3
1
X
Bstart
Q
0
0
-4
3
0
2
0
0
-4
3
0
2
0
1
2
2
2
5
5
0
1
1
1
1
1
1
-4
1
2
2
2
2
5
-4
1
1
2
2
2
1
1
0
0
2
2
2
0
1
0
0
2
2
2
0
2
0
0
0
0
0
6
2
0
0
0
0
0
4
-2
0
0
0
0
0
6
-2
0
0
0
0
0
4
1
1
X
Q
-3
-3
-3
1
29
SMBGT: Example - Computation
a
X
Astart
Q
0
0
-4
3
0
2
-3
0
1
1
1
1
1
1
-4
1
1
2
2
2
1
0
0
2
2
2
0
0
0
-2
0
0
0
X
Q
0
0
-4
3
0
2
-3
1
0
1
2
2
2
5
5
5
1
1
-4
1
2
3
3
3
5
5
2
0
0
1
0
0
3
3
3
0
0
0
0
1
1
2
0
0
0
0
0
6
6
0
0
1
1
-2
0
0
0
0
0
6
6
s
1
X
Bstart
Q
0
0
-4
3
0
2
-3
0
1
2
2
2
5
5
-4
1
2
2
2
2
1
0
0
2
2
2
0
0
0
-2
0
0
0
X
Q
0
0
-4
3
0
2
-3
5
0
1
1
1
1
1
1
1
5
5
-4
1
1
2
2
2
1
1
2
0
0
1
0
0
2
2
2
0
0
0
0
6
6
2
0
0
0
0
0
4
4
0
0
6
6
-2
0
0
0
0
0
4
4
1
1
1
30
SMBGT: Example - Computation
a
X
Astart
X
Q
0
0
-4
3
0
2
-3
1
Q
0
0
-4
3
0
2
-3
1
0
1
1
1
1
1
1
1
0
0
1
2
2
2
5
5
5
0
-4
1
1
2
2
2
1
1
0
-4
1
2
3
3
3
5
5
0
1
0
0
2
2
2
0
0
2
1
0
0
3
3
3
0
0
8
2
0
0
0
0
0
1
1
2
2
0
0
0
0
0
6
6
8
-2
0
0
0
0
0
1
1
1
-2
0
0
0
0
0
6
6
6
s
X
Bstart
X
Q
0
0
-4
3
0
2
-3
1
Q
0
0
-4
3
0
2
-3
1
0
1
2
2
2
5
5
5
0
0
1
1
1
1
1
1
1
0
-4
1
2
2
2
2
5
5
0
-4
1
1
2
2
2
1
1
0
1
0
0
2
2
2
0
0
5
1
0
0
2
2
2
0
0
3
2
0
0
0
0
0
6
6
5
2
0
0
0
0
0
4
4
3
-2
0
0
0
0
0
6
6
6
-2
0
0
0
0
0
4
4
4
31
SMBGT: Example - Solution
Q = 0 -4
1
2
-2
skipped
X=0 0
-4
3
0
2
-3
1
bestvalue = 3, beststart = 2, bestend = 6
a
X
Q
0
0
-4
3
0
2
-3
1
0
1
1
1
1
1
1
1
0
-4
1
1
2
2
2
1
1
0
1
0
0
2
2
2
0
0
2
2
0
0
0
0
0
1
1
2
-2
0
0
0
0
0
1
1
1
32
Outline

Motivation

Related Work

Problem Setting

SMBGT: The proposed method

Experimental Evaluation

Summary and Conclusions
33
Experimental Setup: Data & Evaluation

Music DB: 5643 freely available MIDI files.

6 synthetic query sets, 100 queries/set of lengths 13-137


Q0: exact segments of the db.
Q.10 – Q.50: randomly modified 10-50% elements (both dimensions) of
each query in Q0, with at most 3 consecutive elements changed.

100 hummed queries of lengths 14-76.

DP and Model-based competitor methods




DTW variants: DTWs, DTWc, SPRING modified to allow r.
Edit: Version for music modified for LogIOIR + quantizations
Iliopoulos et al.: Elastic version for subsequence matching supporting
constant and variable tolerance, modified to allow r.
HMMs: Each db sequence modeled by an HMM.
34
Experimental Setup: Evaluation measures



Recall: % of queries for which correct answer is in top-K.
MRR: mean inverse rank of queries in their top-K results.
AR: average rank of all queries in the set, where rank is
the # of sequences with score at least as good as that of
the correct match.
35
Experimental Setup: Variable Tolerances

Pitch

Absolute and relative tolerances tested, with  pf (i)   qip * t  , t  0.2, 0.25, 0.5
and for relative tolerance two cases considered
x jp /(1   pf )  qip  x jp *(1   pf ), qip , x jp  0
x jp /(1   pf )  qip  x jp *(1   pf ), qip , x jp  0

Duration


IOIR:  rf (i, j )  {xrj  2* qir , xrj  qir  0.5}
r
r
r
LogIOIR:


{0

log
(
x
/
q
)

1},
log
x
2
j
i
2
j  0

f
 r (i, j )  

r
r
r

{| log 2 ( x j / qi ) | 1}, log 2 x j  0


36
Experimental Results: Synthetic Queries (1/4)

Parameters


r = |Q|, δ = 0.9, 0.7. 0.6, 0.5, 0.35, 0.3 for Q0 - Q.50 , α = β = 3,
tested both tolerance schemes, K = 20.
Best accuracy





SMBGT: variable abs. tolerance, t = 0.2.
SMGT: variable abs. tolerance, t = 0.25.
Il. et al.: constant rel. tolerance with εp = 1, εr = 4.
HMM: #states = 5, Gaussian observation distribution of the states.
Representation



↑ noise: high recall for representations subset of lower noise levels’
DP methods: <mod12, IOIR>, <pitch interval, IOIR>
HMM: <mod12, LogIOIR in [-2,2]>  smallest alphabet
37
Experimental Results: Synthetic Queries (2/4)

Q0



DP-based methods:
100% recall, MRR = 1,
AR = 1 for top-5.
DTWs: 96% recall for
K = 250.
HMM: 96% recall for
K = 5, MRR=0.95, AR
= 1.22.
38
Experimental Results: Synthetic Queries (3/4)

Q0



DP-based methods:
100% recall, MRR = 1,
AR = 1 for top-5.
DTWs: 96% recall for
K = 250.
HMM: 96% recall for
K = 5, MRR=0.95, AR
= 1.22.
39
Experimental Results: Synthetic Queries (4/4)
40
Experimental Results: Hummed Queries

Tolerance




Small constant and variable t better than greater values.
Variable t better than constant.
Absolute better than relative.
SMBGT and SMGT: absolute, t = 0.2 and 0.25.
Any tol. scheme.
41
Outline

Motivation

Related Work

Problem Setting

SMBGT: The proposed method

Experimental Evaluation

Summary and Conclusions
42
Summary
1.
2.
3.
4.
5.
6.
Lack of flexibility for existing subsequence matching methods.
Edit most promising, but 30 times lower recall than SM(B)GT for
K = 50, α, β improve accuracy.
Variable ε-tolerance better than constant tolerance.
Small range for α, β tested. Other applications  cross validation.
δ tuned according to hummers’ skills, r not exceed 1.2*|Q|.
Simple representation favor DP-based methods.
DP-based methods have similar retrieval time complexity.
43
Conclusions and Future Work




We proposed a novel subsequence matching framework.
We showed that it can be applied to noisy application
domains, such as QBH.
We performed an extensive experimental evaluation on a
large collection of music songs using hummed queries.
Directions for future work include



Testing the proposed method on other noisy domains.
Developing a software application based on SMBGT.
Indexing.
44
Thank you!!!
Questions ???
46
Appendix
SMBGT

Theorem: SMBGT is not metric, as it does not follow
the triangle inequality. Consider:



1D time series X1 = {3, 2}, X2 = {4}, X3 = {2, 2, 1}
ε-tolerance = 1, r = inf, δ = 0, α = β = 1
SMBGT(X1, X2) = 1, SMBGT(X2, X3) = 0, SMBGT(X1,
X3) = 2, but
SMBGT(X1, X2) + SMBGT(X2, X3) < SMBGT(X1, X3).
48
SMBGT: Pseudocode
49
DP Methods – Edit distance



Latest version, extended for LogIOIR and quantizations.
PitchRange, DurationRange : max pitch interval, and LogIOIR range in DB.
Reports minj{α|Q|,j}.
50
DP Methods – SPRING



w(qi, xj): Lp of qi and xj
s to keep start points of best alignments. At each iteration, the start
point of the element used to produce dbest is propagated.
Reports minj{α|Q|,j}.
51
DP Methods – DTW tempo scaling



c: user-defined positive integer
w(qip, xjp) = |qip - xjp|
Report minj{α|Q|,j}.
52
DP Methods – Iliopoulos et al.




Designed for whole sequence matching.
Demands all points of Q match within constant ε-tolerance in a
subsequence of X.
Allows limited # of gaps in X.
Developed a more elastic version suitable for subsequence matching



Single db scan identifies all possible start points of candidate matches.
For each candidate, perform DP computation.
Whole query matching requirement eliminated. When non-matched query
element is met, we consider Q[1:i] candidate match.
53
HMM

Doubly stochastic process defined by





M distinct states
L distinct symbols that can be observed at each state – alphabet
T = {tij} of transition probabilities, where tij = P[st = j|st-1 = i], 1<=i,j<=M
E = {ej(k)} of the probabilities of observation symbols at j, where ej(k) =
P[ot = k|st = j]
Π = {πj} of prior probabilities, where πj = P[s1 = j], 1<=j<=M.

Reflects the probabilistic relations of symbols within sequences.

Look for the model maximizing the log-likelihood of the query.
Model each db sequence with one HMM.

54
Data

Database
 Covers various music genres, e.g., blues, rock, classical, jazz, themes from
movies and tv series, total of 40891 time series.
 Converting music pieces to time series (offline process):
 All-channels extraction: for all channels, extracted the highest pitch at
every time click.
 Converted tuples <pitch, click> to <pitch interval, IOIR>.

Synthetic Queries
 Pitch interval change by ±k in [3, 8] (error by human + audio tool).
 IOIR modified by ±k (real) in [2, 4].
 At most 3 consecutive elements were replaced by noisy values.
Hummed Queries
 4 males (2 with + 2 without musical training), 25 songs each.
 Conversion to MIDIs with Akoff.
 Several genres of music, no lyrics, much noise  several versions of each song.

55
Experimental Results: Accuracy
56
Experimental Results: Accuracy
57
Experimental Results: Representation
58
Experimental Results: Complexities



All DP-based methods: O(|Q|*|X|).
Iliopoulos et al.: O(|X|+|Q|*|X|^2).
HMM


Training for a sequence X: O(W*|X|*M^2),
W = #iterations for Baum-Welch algorithm.
Log-likelihood of Q : O(|Q|*M^2).
59