Dynamic Time Warping
Applications and Derivation
Charles Tappert
Seidenberg School of CSIS, Pace University
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm

Many Applications




Speech recognition
Speech sound alignment
Speech sound generation
Online handwriting recognition
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm



Derivation of a DTW algorithm variation (speech recognition)
A speech utterance is represented as a time sequence of
feature vectors
Example
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm


Consider a finite state machine model of a speech utterance
prototype where the observable output from transitions
between states is an acoustic feature vector which is a
probabilistic function of the origin state of each transition
Note: some transitions cause stretching and others cause
compression of the sequence of feature vectors produced
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm


Background information
Univariate (one-dimensional) normal density function
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm


Background information
Multivariate normal density function
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm



In traversing each arc in this model a feature vector is
produced with assumed underlying normal distribution
where i is the unknown, j the prototype, Vi are feature
vectors of the unknown, Mj are mean feature vectors and
sigma the covariance matrix of the prototype
This statistical characterization of prototypes would require
multiple repetitions of the vocabulary to be recognized
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm


To find the optimal overall probability of the model
(prototype) generating the candidate, we estimate the
maximum value of the cumulative probability over the
possible paths through the model
Assuming statistical independence of the feature vectors,
the best path to any point (i, j) and probability P(i, j) can be
computed, starting with P(0, 0) = Prob(0, 0) and P(i, j) = 0
elsewhere, using the recursion relation
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm


Taking the log of terms in previous equation, dropping
constant terms, multiplying by -2, and assuming zero
covariance terms yields the recursion relation
where D(i, j) is considered a cumulative distance measure
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm



Further, assume equal variances and transition probabilities,
and include an index k indicating the prototype
d(i, j; k) is the distance between feature vectors i and j
Note: since the log function is a monotonically increasing
function of its argument and changing sign converts a
maximizing relation into a minimizing one, this distance
relation leads to the same decisions as the probability
recursion relation, except for the simplifying assumptions
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm



This derivation shows the simplifying assumptions made in
going from a probabilistic model to a greatly simplified
distance model
Currently, most string matching commercial and research
systems use probabilistic models, and the Hidden Markov
Model (HMM) is probably the dominant one
As computing power has increased over the years, more
complex and primarily probabilistic models requiring large
training corpuses have been used
Dynamic Time Warping (DTW)
non-linear/elastic matching, Viterbi algorithm

In his research at IBM’s T.J. Watson Research Center,
your instructor worked in both the speech recognition
and the pen computing/handwriting recognition groups



founding member of the speech group (once over 50 workers)
spearheaded development of ThinkWrite handwriting recognizer
in IBM’s pen-enabled ThinkPad product in the early 1990s
The data in both the speech and online handwriting
problems are time sequences


Speech is recorded as a time waveform and usually transformed
via frequency analysis into a sequence of spectral time samples
Online handwriting is captured as a time sequence of x-y
coordinates describing the trajectory of the handwriting