Pronunciation Learning for Automatic Speech
Recognition
by
Ibrahim Badr
Submitted to the Department of Electrical Engineering and Computer
Science
in partial fulfillment of the requirements for the degree of
Master of Science in Electrical Engineering and Computer Science
ARCHNES
MASE
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
June 2011
@ Massachusetts Institute of Technology 2011. All rights reserved.
. ............
A uthor .............
Department of Electrical Engineering and Computer Science
May 18, 2011
Certified by.................
.. . .. . .. . .
.
James Glass
Principal Research Scientist
Thesis Supervisor
Accepted by .............
/
j)
eslie A. Kolodziejski
Chair of the Committee on Graduate Students
2
Pronunciation Learning for Automatic Speech Recognition
by
Ibrahim Badr
Submitted to the Department of Electrical Engineering and Computer Science
on May 18, 2011, in partial fulfillment of the
requirements for the degree of
Master of Science in Electrical Engineering and Computer Science
Abstract
In many ways, the lexicon remains the Achilles heel of modern automatic speech recognizers (ASRs). Unlike stochastic acoustic and language models that learn the values of their
parameters from training data, the baseform pronunciations of words in an ASR vocabulary
are typically specified manually, and do not change, unless they are edited by an expert.
Our work presents a novel generative framework that uses speech data to learn stochastic
lexicons, thereby taking a step towards alleviating the need for manual intervention and automnatically learning high-quality baseform pronunciations for words. We test our model on
a variety of domains: an isolated-word telephone speech corpus, a weather query corpus and
an academic lecture corpus. We show significant improvements of 25%, 15% and 2% over
expert-pronunciation lexicons, respectively. We also show that further improvements can be
made by combining our pronunciation learning framework with acoustic model training.
Thesis Supervisor: James Glass
Title: Principal Research Scientist
Acknowledgments
I am grateful to my advisor Jim Glass for his guidance and support.
His patience and
encouragement has helped me significantly in my past two years at MIT. While I was still
an undergraduate at the American University of Beirut, Jim accepted I intern at SLS as
a visiting student, despite the fact that I had no significant speech recognition, machine
learning or even programming experience to speak of. What convinced Jim to do so based
only on a half hour telephone conversation, a recommendation from my algorithms professor
and some patched up chess algorithm code I may never know. I will always be thankful
for the opportunity, studying and working at MIT has been a humbling and life changing
experience.
Much thanks to Ian McGraw for his patience in teaching me all the skills needed to
work at SLS, from how to restart the browser on CSAIL Debian and proper coding style to
belief propagation and discriminative acoustic modeling. I am grateful for all the help I have
received from all the people at SLS, the staff, as well as, the students. I would like to thank
Scott Cyphers, Stephanie Seneff, Lee Hetherington, Chao Wang, Hung-An Chang, Paul Hsu,
Alex Gruenstein, as well as, everyone else in the group, too many to name here, for all their
help and support.
I would also like to thank Michael Collins for letting me unofficially sit in on his NLP
course while I was still a visiting student. What I learned during that semester has heavily
influenced my research.
Last but not least, I would like to thank my friends and family for their continued encouragement and support - "nervat bi3tizir inch ibnik il sa2it" -
This research was sponsored by Nokia as part of a joint MIT-Nokia research collaboration.
fi
Contents
1
Introduction
1.1
15
M otivation . . . . . . . . . . . . . . .
. . . . . . . . . .. .
.
.
.
16
1.1.1
The Lexicon: A Hand-Written ASR Component .
..
.
.
.
16
1.1.2
The OOV problem and Dynamic Vocabulary Learning
.
.
.
17
.
1.2
Contributions of this Thesis . . . . . . . . . . . . . . . . . . .
.
.
.
19
1.3
Summary of Each Chapter . . . . . . . . . . . . . . . . . . . .
.
.
.
19
2 Background
21
2.1
Pronunciation Modeling.
2.2
Letter-to-Sound Models.... . . . . . . .
.. .
. . . . . ........
. ...
. . . . . . .
21
. . . . . . . . . . . .
22
2.2.1
The Graphoneme model . . . . . . . . . . .
. . . . . . . . . . .
.
22
2.2.2
Other Work on Letter-to-Sound Prediction .
. . . . . . . . . . . . .
23
.. . . .
24
2.3
Using Acoustics to Learn Pronunciations ......
2.4
Global Linear Models . . . . . . . . . . . . . . . . .
2.5
Summary .. . . . . . . . . . . .
. . . . . . . ..
. . . . . . . . . . . .
.. .
. . . . . . .
3 Learning Pronunciations from Isolated-Word Spoken Examples
3.1
3.2
Graphone-guided Phonetic Recognition...
. . . .... ..
. . . . .... ...
3.1.1
Cascading Recognizers.......
3.1.2
Pronunciation Mixture Model . . . . . . ...........
. . . .
. . .
Experimental Setup and Results . . . . . . . . . . . . . . . . . . . .
. .
. . . . . . . . . . . . . . . . ..
3.2.1
Experimental Setup...
3.2.2
Results using Un-Weighted Pronunciations.....
. . . ..
25
26
3.2.3
Results using Stochastic Lexicon
. . . . . . . . . . . . . . . . . . . .
34
3.3
Training Pronunciations using Noisy Acoustic Examples
. . . . . . . . . . .
35
3.4
Analysis of Learned Baseforms . . . . . . . . . . . . . . . . . . . . . . . . . .
37
3.5
Phone Versus Phoneme Pronunciations . . . . . . . . . . . . . . . . . . . . .
37
3.6
Sum m ary
40
. . . . . . . . . . .
. . . . . .
. . . . . . . . . . . . . .
4 Discriminative Pronunciation Training for Isolated-Word Speech Recognition
5
41
4.1
Global Linear Models for Lexicon Learning . . . . . . . . . . . . . . . . . . .
42
4.2
Implementing the Algorithm as an FST . . . . . . . . . . . . . . . . . . . . .
44
4.3
Experimental Setup and Results . . . . . . . . . . . . . . . . . . . . . . . . .
46
4.4
Summary......... . . . . . . . . .
47
. . . . . . . . . . . . . . . . . . .
Pronunciation Mixture Model for Continuous Speech Recognition
49
5.1
EM for Estimating Phoneme Pronunciations . . . . . . . . . . . . . . . . . .
50
5.1.1
. . ..
52
. . . . . . . . . . . . . . .
52
EM for Estimating Phone Pronunciations . . . . . . . ...
.
5.2
Implementation Details...... . . . . . . . .
5.3
Experimental Setup......... . . . . . . . .
5.4
5.5
5.6
. . . . . . . . . . . . . . .
5.3.1
Experimental Procedure....... . .
5.3.2
Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
55
55
56
Analysis of Learned Baseforms......... . . . . .
. . . . . . . . . . ..
58
5.4.1
Non-Native Speaker Pronunciation.......
. . . . . . . . . . ..
60
5.4.2
Slow vs. Fast Speech . . . . . . . . . . . . . . . . . . . . . . . . . . .
63
5.4.3
Word Confusion in Weather Domain and Lecture Domain Lexicon . .
63
Acoustic Model and Lexicon Co-Training . . . . . . . . . . . . . . . . . . . .
66
5.5.1
Real-W orld Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
5.5.2
Baseline Approach
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
5.5.3
PM M Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
Sum m ary . . . . . . . . . . . . . . . . . .
6 Summary and Future Work
. . . . . . . . . . . . . . .
68
71
A Weather Domain Lexicon
73
10
List of Figures
1-1
Major Components of a Speech Recognition System: The Acoustic Model,
Lexicon and the Language Model [1]
1-2
. . . . . . . . . . . . . . . . . . . . . .
Diagram of Speech Recognition System behavior when recognizing an utterance containing an OOV words [10]
3-1
18
. . . . . . . . . . . . . . . . . . ...
Diagram of Cascading Recognizers Model: Each utterance uses recognition
. . . . . . . . .
results of preceding utterances to decode..... . . . . .
3-2
16
29
Diagram of Pronunciation Mixture Model: Pronunciation weights are initialized with graphoneme letter-to-sound model. All utterances vote in parallel
to assign weights for pronunciations (E-step), Votes are then summed and
normalized (M-step). New weights are used to initialize the next iteration.
3-3
.
30
Word Error Rate (WER) as a function of the number of example utterances
used to adapt the underlying lexicon. We show results for using the Cascaded
Recognizers Model, the PMM, the Graphone model and the Expert Dictionary 33
3-4
Word Error Rate (WER) as a function of the number of example utterances
used to adapt the underlying lexicon.
We show results for using the un-
weighted learned pronunciations while varying the
T
threshold, as well as,
using the PMM (stochastic lexicon) . . . . . . . . . . . . . . . . . . . . ...
5-1
Constrained Graphoneme Recognizer n-best hypothesis for "houston texas"
along with total, AM and LM score . . . . . . . . . . . . . . . . . . . . . . .
5-2
35
54
Plot of H(w~b) and H(blw) as a function of EM iterations for both the Weather
and Lecture dom ain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
65
12
List of Tables
3.1
By varying a threshold T over the weights learned in the PMM, we can incorporate multiple baseform pronunciations for individual words...
3.2
33
PMM results incorporating spoken examples collected via Amazon Mechanical
Turk.................
3.3
. . ...
. ..... ....
. . . . . . .
. . . .
37
Example baseform changes between expert dictionary and top PMM hypothesis. Phonemes involved in the difference have been capitalized...
. ...
38
3.4
Top 10 phoneme confusion pairs for Phonebook lexicon . . . . . . . . . . . .
38
3.5
Top 5 Deleted Phonemes in Phonebook lexicon.... . . .
. . . . . . .
38
3.6
Top 5 Inserted Phonemes in Phonebook lexicon
. . . . . . . . . . . . .
39
3.7
PMM results for learning phone pronunciations from PhoneBook corpus.
4.1
Results for Discriminative training in WEB. on PhoneBook corpus . . . . . .
5.1
Results in WER on both the Weather and Lecture corpora using the PMM,
.
39
46
the expert dictionary and the Graphoneme model (%) . . . . . . . . . . . . .
57
5.2
Lexicon statistics for the weather and lecture domains.
58
5.3
Top 10 confusion pairs for Weather Domain lexicon . . . . . . . . . . . ...
59
5.4
Top 5 Deleted Phonemes for Weather Corpus lexicon . . . . . . . . . . . . .
59
5.5
Top 5 Inserted Phonemes for Weather Corpus lexicon . . . . . . . . . . . . .
60
5.6
Example phoneme deletions between expert dictionary and top PMM hypoth-
. . . . . . . . . . . .
esis for W eather Corpus . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.7
Example of phoneme insertions between expert dictionary and top PMM hypothesis for Weather Corpus.
5.8
60
. . . . . . . . . . . . . . . . . . . . . . . . . .
61
. . . .
61
Top 5 phoneme confusion pairs for Indian accented non-native talker
5.9
Example of differences between native and Indian accented non-native speaker
pronunciations.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
62
5.10 Top 5 phoneme confusion pairs for Dutch accented non-native talker . . . . .
62
5.11 Example of differences between native and Dutch accented non-native speaker
pronunciations............. . . . . . . .
. . . . . . . . . . . . . . .
5.12 Top 10 phoneme confusion pairs between "fast" speech and "slow" speech
62
. .
64
5.13 Top 5 inserted phonemes in "fast" speech pronunciations . . . . . . . . . . .
64
5.14 Top 5 deleted phonemes in "fast" speech pronunciations
64
. . . . . . . . . . .
5.15 Examples of pronunciation differences between fast and slow speech pronunciations........ . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . .
64
5.16 WER in percentage for E-book domain by iteratively training the acoustic
models and the lexicon. A gradual reduction in WER can be observed.
. .
.
68
5.17 WER in percentage for E-book domain using pmmlem3. An increase in WER
is observed possibly due to overfitting. . . . . . . . . . . .
. . . . . . . .
68
Chapter 1
Introduction
In many ways, the lexicon remains the Achilles heel of modern automatic speech recognizers (ASRs). Unlike stochastic acoustic and language models that learn the values of their
parameters from training data, the baseform pronunciations of words in an ASR vocabulary
are typically specified manually (usually along with the basic phoneme inventory itself), and
do not change, unless they are edited by an expert. The lexicon is usually a long list of these
hand-crafted pronunciations in the form of dictionary entries that map a word to one or more
canonical pronunciations.
A more desirable solution would be one whereby the basic linguistic units of a language,
and the associated lexical pronunciations could be determined automatically from a large
amount of speech data. In this thesis, we discuss methods for the latter, learning the lexical
pronunciations of words given both their spelling and spoken utterances of isolated-word
and/or continuous speech. This data-driven approach to lexicon generation might discard
the notion of canonicalization altogether, and instead generate a stochastic lexicon with
pronunciations weighted according to learned statistics.
Like the acoustic and language models, ideally pronunciations would be learned from
data closely matching the test domain. In our work, we use the same training data used for
Acoustic Model training to learn our pronunciations. Since this data is readily available at
no extra cost for all recognizers, we see our work as encouraging lexicon training to become
a standard procedure when building an ASR system, while taking a step towards being able
to train a speech recognizer entirely from an orthographically transcribed corpus.
11
..........
111............
.............
1111
....
................
..........
._ __I ..
...
....
.....
I....
....
-----_
. ...
......
_ __
=
_
4.
Figure 1-1; Major Components of a Speech Recognition System: The Acoustic Model, Lexicon and the Language Model [1]
1.1
1.1.1
Motivation
The Lexicon: A Hand-Written ASR Component
ASR is the process of decoding a spoken utterance into a string of words [10]. The spoken
utterance is usually modeled as a weighted network of sub-lexical units, which are typically
phones. A phone graph which models all speech corresponding to an input acoustic signal is
constrained by the acoustic model, the lexicon and the language model. The acoustic model
statistically models context-dependent or context-independent phones, typically Gaussian
Mixture models are used. It is trained on acoustic-phonetic measurements of transcribed
speech data. The lexicon maps words to their phonetic pronunciations, an example entry
would be colonel : k er n ax 1, and is typically crafted manually by experts. The language
model statistically models the probability of a word sequence and typically uses n-gram distributions to scores sentences. Figure 1-1 illustrates this process.
The fundamental equation of ASR seeks the most likely word sequence W* = w*,
...
, w*
given an utterance u:
W =argmaxP(Wu) = argmaxP(ulW)P(W)
W
W
(1.1)
Where P(ulW) is the acoustic model and P(W) is the language model. This equation implicitly implies that to each word corresponds one phonetic representation. A more accurate
model would be:
P(Wlu) =
(
PW,B
=
Zu)
P(u)
B,u) - ZBCBP(W)P(BIW)P(uIB,
P(u)
(1.2)
Here B is a phonetic pronunciation and B is the set of all phonetic pronunciations.
P(Wlu)
-
ZBc P(W)P(B|W)P(u|B)
P(u)
(1.3)
where we assume conditional independence P(ulB, W) = P(ulB). Hence:
W
arg max P(Wlu) = arg max
w
w
P(u|B)P(B|W)P(W)
(1.4)
BEB
P(B IW) is the lexicon. It assigns a probability to each phonetic pronunciation. Typically all
baseforms are weighed equally by setting P(BlW) = 1 for all B corresponding to W in the
lexicon, even if the lexicon admits multiple pronunciations per word [34, 25]. In this work we
propose to learn the lexicon, thereby avoiding the use of experts, as well as having to weight
all baseforms equally.
Learning the lexicon from data opens up the possibility of using discriminative training
in order to directly optimize on error rate. Discriminative training has been extensively
studied for acoustic and language models and has been shown to perform significantly better
than maximum likelihood methods. In this thesis, we explore incorporating Discriminative
training to enhance our stochastic lexicons.
1.1.2
The OOV problem and Dynamic Vocabulary Learning
Almost all speech recognizers have a finite vocabulary pronunciation dictionary. If the recognizer encounters an out of vocabulary (OOV) term, a word that is not in the lexicon,
it cannot produce a transcription for it. This limitation is detrimental in many ASR. applications. Figure 1-2 demonstrates a potential outcome to an OOV word euthanasia. A
Acoustic
P Feature Extraction
Speech
Example Input:
euthanasia
euthanasia f
Language
Kecognizer
Search Space
Word Hypotheses
Example Outputs:
youth in Asia
using Asia
use in Asia
Lexicon
Figure 1-2: Diagram of Speech Recognition System behavior when recognizing an utterance
containing an OOV words [10]
misrecognized OOV word can easily cause the surrounding words or even the whole sentence
to be misrecognized. Therefore, in order for a ASR systems to be used in the real world, techniques need to be developed to handle the OOV problem. A typical approach is to increase
the vocabulary size whether manually or automatically. This cannot completely eliminate
OOVs in an open domain recognition task. Other techniques include using confidence scoring
to detect OOVs [14], as well as, filler models to hypothesize the pronunciation of an OOV
word [6]. In our work, we focus on automatically learning quality baseforms for OOV words
in order to expand our lexicon.
The rate of OOV occurrence is tied to the design of the ASR vocabulary. Since constantly increasing the vocabulary size is bound to introduce acoustic ambiguity, ASRs should
undergo a paradigm shift from vocabulary design to that of an adaptive system that can
dynamically detect and learn new words. In a typical scenario, a speech recognizer remains
static while it is deployed. Its acoustic models, language models and lexicon are trained or
specified beforehand and remain unchanged while the recognizer is used. In our work, we
explore techniques to dynamically expand a recognizer's lexicon, as well as, improve on the
pronunciations already present in the lexicon. This will be done by either automatically
collecting new data to be used to retrain the lexicon or using the speech data collected from
our speech recognizer's tasks.
1.2
Contributions of this Thesis
This thesis offers the following specific contributions:
" A method for learning better-than-expert word pronunciations given their spellings, as
well as, spoken utterances.
- This method can learn from both isolated-word and continuous speech.
- This method is robust and hence can learn from cheaply collect noisy speech.
- This method can supplement the Acoustic and Language models without the use
of any additional training data.
* A method for discriminatively training lexicon pronunciation weights for isolated word
recognition.
" A method for iteratively training the Acoustic Models, as well as, the Lexicon.
1.3
Summary of Each Chapter
The rest of this thesis is organized into 4 chapters. The content of each of the remaining
chapters is summarized next:
" Chapter 2: Background
Chapter 2 starts with a description of how pronunciation variability is handled in
current ASR systems. It then presents a review of the major advances in pronunciation
learning for ASR. It also explains some of the mathematical frameworks applied in this
thesis.
" Chapter 3: Learning Pronunciations from Isolated-Word Spoken Examples
This chapter presents two Bayesian models that learn pronunciations from isolated word
examples. It reports on experiments that produce better-than-expert pronunciations
for isolated word recognition, as well as, an inherent robustness to real world noisy
data.
"
Chapter 4: Discriminative Pronunciation Training for Isolated-Word Speech
Recognition
In this chapter, we improve on our results for isolated-word recognition by discriminatevly training our learned pronunciations to minimize WER.
" Chapter 5: Learning Pronunciations from Continuous Speech
Here, we explore the use of continuous speech data to learn stochastic lexicons. Building
on the work of the previous chapter, we extend our framework to two domains containing spontaneous speech: a weather information retrieval spoken dialogue system and
an academic lectures domain. We find that our learned lexicons out-perform expert,
hand-crafted lexicons in each domain. We also explore the relationship between the
Lexicon and the Acoustic Models by iteratively re-training them. We show improvemerits on a Book Titles corpus and discuss limitations that should be addressed for
these scenarios.
" Chapter 6: Summary and Future Direction
This chapter concludes the thesis, and suggests some ideas for future research.
Chapter 2
Background
2.1
Pronunciation Modeling
Pronunciation variation has been identified as a major cause of errors for a variety of ASR
tasks [26]. Pronunciations are typically modeled in a speech recognizer by a phonemic dictionary which may be accompanied by a set of rewrite rules to account for phonological
variation.
The ASR system used in this paper incorporates manually crafted phonological rules that
account for segmental mismatches between the underlying phonemic baseforms and surfacelevel phonetic units. These rules have been shown to outperform relying on context-dependent
acoustic models to implicitly model phonetic variation [18].
In this work, we model the ASR's search space using a weighted finite-state transducer
(FST) [20]. The FST search space has four primary hierarchical components: the language
model (G), the phoneme lexicon (L), the phonological rules (P) that expand the phoneme
pronunciations to their phone variations, and the mapping from phone sequences to contextdependent model labels (C). The full network can be represented as a composition of these
components:
N =CoP oLoG
(2.1)
In this work, we experiment with learning context-independent phoneme pronunciations
along with their weights. That is, we try to replace the manually crafted FST L while keeping
the pronunciations rules FST P unchanged. We also explore learning phone pronunciations
directly, thus avoiding the use of the phonological rewrite rules altogether.
2.2
Letter-to-Sound Models
Generating pronunciations for new words has been the subject of much work [9, 7, 22, 33,
28, 5, 23, 21] . The common approach is to use a form of letter-to-sound model to predict
the pronunciations of new words using only their spelling. Despite the fact that for some
languages, mapping a spelling to a pronunciation is relatively straightforward, English has
shown itself to be rather challenging. Many models have been used for this task: in our work,
we compare our results to the graphone model which has been shown to produce state of the
art scores for letter-to-sound tasks [7, 36]. We address the mathematical formulation of the
graphone model in this next section.
2.2.1
The Graphoneme model
We utilize the joint-multigram approach employed in [9, 7] to model the relationship between
graphemes and phonetic units. In this work, we use the term graphone to denote a model
that maps graphemes to phones, and graphoneme to refer to a model that maps graphemes
to phonemes.
We begin by constructing an n-gram model over graphoneme sequences. We let w denote
a grapheme sequence drawn from the set of all possible grapheme sequences W and b denote
a phoneme sequence drawn from the set of all possible phoneme sequences, B. A joint model
of the letter-to-sound task can be formalized as:
b*
arg max P(w, b)
bEB
Generally speaking, a graphoneme, g = (w, b)
(2.2)
gC C W x B, is a sub-word unit that
maps a grapheme subsequence, w, to a phoneme subsequence, b. By analogy, a graphone is
an alternative sub-word unit that maps a grapheme subsequence to a phone subsequence.
In this work, we restrict our attention to singular graphones or graphonemes, in which a
mapping is made between at most one grapheme and at most one phonetic unit. The empty
subsequence c is allowed, however a mapping from c to e is omitted. Taken together, a
sequence of graphonemes, g, inherently specifies a unique sequence of graphemes w and
phonemes b; however, there may be multiple ways to align the pair (w, b) into various
graphoneme sequences g E S(w, b).
The following table shows two possible graphoneme
segmentations of the word "couple".
w
=
c
o
b
=
k
ah
-
k
u1
ah
k1 U/ a h1U/eC
1
p
p
ax
p
ax
P/ p
C
e
1
11/ 11
gi
C/
92
c/k o/ c u/ahp/p c/ax 1/1
/
Given this ambiguity, employing graphonemes in our joint model requires us to marginalize over all possible segmentations. Fortunately, the standard Viterbi approximation has
been shown to incur only minor degradation in performance [71.
P(w,b) =
P(g) ~ max P(g)
gGS(wvb)
(2.3)
gES(wb)
In our work, we use the open source implementation provided by [7], which runs the
Expectation-Maximization (EM) algorithm on a training corpus of word-phoneme pironunciation pairs to automatically infer graphoneme alignments. We then train a standard 5-gram
language model over the automatically segmented corpus of graphonemes. This configuration
has been shown to produce good results for singular graphonemes [36].
2.2.2
Other Work on Letter-to-Sound Prediction
While original work in Grapheme-to-Phoneme conversion consisted of rule-based methods [22],
these quickly became overly complicated and tedious for practical use and were soon replaced
by data-driven methods. An example of work that utilizes knowledge-based formal linguistic
methods in a statistical framework is [33]. As a first step, a hand-written grammar parses
words into a set of hand-written linguistically motivated sub-word "spellneme" units. Then,
after parsing a large lexicon into these segmented words, a statistical n-gram model is trained
and used later for decoding.
Most other work on letter-to-sound generation can be classified as using a pronunciation
by analogy technique or local classification technique. Local classification processes a word
spelling sequentially from left-to-right. For each input character a decision is made by looking
at the letter's context using decision trees [30] or neural networks [17]. As for pronunciation
by analogy, the main theme is to scan the training lexicon for words or part-of-words that
are in some sense similar to the word to be transcribed [28, 5]. The output pronunciation is
then chosen to be analogous to the retrieved examples.
While the most recent state-of-the-art work has been on probabilistic approaches such
as the joint-sequence model discussed in the previous section.
Some have also explored
discriminative training in a joint-sequence setting [21], as well as, grapheme-to-phoneme
conversion using a Statistical Machine Translation system (SMT) [23].
2.3
Using Acoustics to Learn Pronunciations
Other relevant work extends on the idea of letter-to-sound generation by incorporating spoken
examples to refine the generated pronunciations [27, 11]. The work of [4] deduces the baseform
b* given a, word or grapheme sequence w and an utterance u of the spoken word w. In this
work, a decision tree is used to model P(w, b) which was later shown to produce poorer
results when compared to graphone models.
The work of [24] adapts the graphone model parameters using acoustic data and is applied
to a name recognition task. Given a training set (Ui, wi) of pairs of words w and spoken
examples w, they maximize the likelihood of their data:
Al
M
log po(ui, w; 0) =
i=1
log
i=1
po(ui, w, b)
bc3
by using the standard EM algorithm to adjust 0, the graphone n-grain parameters. They
initially set the parameters of 0 to those of the graphone model and iterate until convergence.
They also experiment with discriminative training and show that it produces better results
than MLE. In our work, we use a similar framework to MLE geared towards predicting a
pronunciation for a single word given multiple example utterances.
The work of [35] uses the forced alignment of a phonetic decoder to generate a list of possible pronunciations for words, and then assigns weights using a Minimum-Classification-Error
criterion. They then test on a business name query corpus. Curiously, work on pronunciation
learning using acoustic examples is rarely applied across the entire lexicon to regularize the
pronunciations with respect to the underlying acoustic models. By contrast, in our work we
learn pronunciations across all lexical entries rather than the few out-of-vocabulary words
for which we do not have an expert opinion. Thus, our test experiments directly compare a
learned stochastic lexicon with manually-crafted pronunciations.
2.4
Global Linear Models
Here we describe a general framework of linear models that can be applied to a diverse range
of tasks, e.g. parsing or ASR hypothesis reranking. We will be modifying this framework to
train our pronunciation lexicon.
The framework is outlined in [12]. We learn a mapping from inputs x E X to outputs
y c Y by assuming:
" A set of training examples (xi, Ii) for i = 1
.
N
" A function GEN which enumerates at set of candidates GEN(x) for input x
" A representation <b mapping (x, y) to a feature vector <D(x, y) E Rd
" A parameter vector a E Rd
We define a mapping from an input x to an output F(x)
F(x) = arg max
yEGEN(x)
<D(x, y) -a
(2.4)
To illustrate how Global Linear Models can be applied to discriminative language model
training for ASR, we reference the work of [32].
" Each (Xi, yi) is an utterance and its reference transcription
" GEN(x) is the outputed word-lattice from a baseline recognizer
e <(x, y) tracks unigram, bigram and trigram word counts in y, as well as, the score for
(x, y) under the baseline recognizer
To optimize the parameters a, we consider this variant of the perceptron algorithm:
INPUT: Training examples (xi, yi)
INITIALIZATION: Set a = 0
ALGORITHM:
for t = 1. T,i = 1 --. N do
zi =arg maxzEGEN(xj)
(xi,Z) ' a
if zi 4 y, then
a =a + ID(Xi, yi)
-
T(Xi, Zi)
end if
end for
OUTPUT: a
For a complete discussion regarding the convergence of this algorithm, as well as, generalization to unseen data, we refer the reader to [12].
2.5
Summary
In this chapter, we presented a survey of approaches that handle pronunciation prediction
from word spellings, as well as, word spelling supplemented with acoustic examples. We also
explained how pronunciation variability is handled in the ASR system used in this thesis.
We concluded with a description of Global Linear Models, a mathematical framework used
for Discriminative training.
Chapter 3
Learning Pronunciations from
Isolated-Word Spoken Examples
We first tackle the simplified task of learning word pronunciations from isolated-word speech
and testing on isolated-word speech. Our research differs from previous work in the stochastic
learning framework. In our case, we use an n-gram graphoneme-based model as the basis for
our initial estimate of a pronunciation. The graphoneme model is used as a form of prior
to condition our expectation of possible pronunciations of a new word, given its spelling.
We then use spoken examples to further refine the pronunciations, and explore two different
stochastic pronunciation models: the first cascades all the examples to find a single best
pronunciation, while the second creates a pronunciation mixture model (PMM) to consider
multiple pronunciations. We compare both approaches on the telephone-based PhoneBook
corpus of isolated words [2] and find that they are able to recover expert-level pronunciation
baseforms with relatively few example utterances.
3.1
Graphone-guided Phonetic Recognition
We begin by exploring a model which incorporates one example utterance with the graphoneme model to find a single high probability baseform. Given a word with grapheme
sequence w and an example utterance of the word u, we deduce the baseform for b* using a
similar framework to that described in [4].
b* =argmaxP(blw,u) =argmaxP(b,w)p(ulb,w)
beB3
bcB
(3.1)
We replace the decision tree originally described in [4] with a graphoneme N-gram model
using independence assumptions and the Viterbi approximation.
b
arg
max
beIB7eS(w b)
p(ub)P(g)
(3.2)
For each word, w, a recognizer, R., can be constructed using weighted finite state transducers (FSTs) to model the mapping of acoustic model labels to phoneme sequences, weighted
by graphoneme n-gram parameters.
The procedure described above only incorporates a single example utterance into the pronunciation generation framework. The following sections introduce two methods of utilizing
a set of Al example utterances, u, , of a given word, w.
3.1.1
Cascading Recognizers
As in Equation 3.1, we apply Bayes rule with the additional assumption of independence
between example utterances given their pronunciations to model the probability of a baseform
given the data:
M
argmaxP(blw,u') =argmaxP(b,w)fjp(ujlb,w)
bE3
bE3
i=1
This multiple utterance recognizer can be implemented as a cascade of single utterance
recognizers. Figure 3-1 illustrates this approach. While this framework ties up all utterances
to predict a single baseform, in this next section, we explore a more parallel approach that
can model multiple pronunciations simultaneously.
Grahone
ASA
U2
00
+
-E
Un+4-
:
b
Figure 3-1: Diagram of Cascading Recognizers Model: Each utterance uses recognition results
of preceding utterances to decode.
3.1.2
Pronunciation Mixture Model
A second formulation of pronunciation generation informed by multiple example utterances is
that of a pronunciationmixture model (PMM). We parametrize our model with
Ob,=
P(b, w)
under the assumption that a particular word w and baseform b have some joint probability,
however small, of mapping to one another. In a setup similar to the work described in [24],
expectation maximization is used to update these parameters based on the data (ul , w).
Whereas Li et al. optimize graphoneme language model parameters, our goal here is to
directly learn weights for word pronunciations, hence the PMM characterization. We begin
by characterizing the log-likelihood of the data.
M
L(O)
(log
i=1
Al
p(ui,w; 0) =(log
i=1
Owb .p(u 1 w, b)
bE B
The parameters, 0, are initialized to our graphoneme n-gram model scores and run multiple iterations of the EM algorithm. The following equations specify the expectation and
maximization steps respectively:
-- --
--
-
-
-
--
-
-----
-
Initial Parameters
nrapho
N-gramn
I..------ ---.- - ---..-- -
Figure 3-2: Diagram of Pronunciation Mixture Model: Pronunciation weights are initialized
with graphoneme letter-to-sound model. All utterances vote in parallel to assign weights for
pronunciations (E-step), Votes are then summed and normalized (M-step). New weights are
used to initialize the next iteration.
E-step:
P(bluiw;0)
w,b - P(uiIb, w)
E ,,,p -p(uib,w)
1A
M-step:
6
*b =
P(blui, w; 0)
i=1
We illustrate this parallel approach in Figure 3-2.
As an initial experiment we pick the baseform b with the highest weight as the pronunciation of w and use it unweighted i.e. with a probability of 1.
b
arg max P(w, b; 0*) = arg max 0*
bE13
bE13
(3.3)
In later experiments, we use multiple pronunciations weighted by their mixture probabilities in a stochastic lexicon.
3.2
3.2.1
Experimental Setup and Results
Experimental Setup
To experiment with the two pronunciation models, we use a landmark-based speech recognizer [15].
MFCC averages are computed over varying durations around hypothesized
acoustic-phonetic landmarks to generate 112-dimensional feature vectors, which are then
whitened via a PCA rotation. The first 50 principal components are kept as the feature
space over which diphone acoustic models are built. Each model is a diagonal Gaussian
mixture with tip to 75 mixture components trained on a separate corpus of telephone speech.
The search space in the recognizer is modeled using a flexible weighted FST toolkit [20].
We consider the task of isolated word recognition using the NYNEX PhoneBook corpus. PhoneBook is a phonetically-rich, isolated-word, telephone-speech database of American English spanning a variety of talkers and telephone transmission characteristics. The
core section of PhoneBook consists of a total of 93,667 isolated-word utterances, totalling
23 hours of speech. This breaks down to 7,979 distinct words, each said by an average of
11.7 talkers, with 1,358 talkers each saying up to 75 words. All data were collected in 8-bit
in-law digital form directly from a Ti telephone line. Talkers were adult native speakers of
American English chosen to be demographically representative of the U.S [2].
To insure adequate data for our baseline experiments, we chose 2,000 words at random
from the subset of the corpus that had example utterances from at least 13 distinct speakers.
We held out two of the 13 utterances, one from a male speaker the other from a female
speaker, to generate a 4,000 utterance test set.
While the individual recognition experiments described in the next section are limited to
the 2,000 selected words, a far larger lexicon was used to train the initial graphone language
model parameters. For this work we used an internal dictionary that contains over 150,000
manually generated entries. To simulate the out-of-vocabulary scenario for which graphones
are typically employed, we removed the 2,000 trial words from our lexicon, and further
pruned similarly spelled words using a simple edit distance criterion. We then trained a
5-gram graphone language model according to the procedures described in [36].
We conduct two baseline experiments to frame our remaining results. The first is a
graphoneme-only baseline in which we performed isolated word recognition over the 4,000
test utterances using a 2,000 word lexicon generated from the graphoneme model alone
according to Equation 2.3. Since no acoustic information was used, this provides us with
an upper-bound on word error rate (WER) of 16.7%. The second baseline was again the
2,000 word-recognition task; however, this time we explicitly used the manually generated
pronunciations originally found in our lexicon [3], giving us a target WER of 12.4%, achievable
directly by experts.
It should be noted that about 160 words in the expert-lexicon had multiple baseforms
associated with them. For example, the word "youths" was represented as both y uw dh z
and y uw th s. Initial experiments indicated that allowing multiple baseforms could give an
advantage to the expert-lexicon that could be leveraged in the other frameworks. We begin
however by choosing only a single pronunciation for inclusion in an automatically generated
lexicon. Even so, were able to show the feasibility of recovering and even surpassing the
performance of manually generated baseforms.
3.2.2
Results using Un-Weighted Pronunciations
Having established our baseline experiments, we evaluated both the cascading recognizer
approach and the PMM by varying the number of training utterances for each, and evaluating
the WER of the test set against the lexicons produced under each condition. The resulting
plot is shown in Figure 3-3. It is encouraging to note that both models perform admirably,
achieving expert-level pronunciations with just three example utterances per word.
The cascading recognizer approach of Section 3.1.1 improves slightly faster than the PMM
technique. With seven utterances, this model surpasses the expert baseform WER by nearly
1%.
An inherent assumption of this model is that there is a single, correct underlying pronunciation. This fact may explain the slight advantage that this approach has, since our
experimental design only allows a single baseform for each word in our automatically generated lexicon. A model which directly computes the single most likely baseform given the
data is thus particularly well-suited to the task.
Ideally, a pronunciation generation model would be able to cope with words that have
Lexicon Adaptation using Acoustic Examples
I7
'4* Graphones Only
016
C) 15
-X- Cascaded Recognizers
0, Pronunciation Mixture Model
(D14
0
-
13
Expert Baseforms
Cc
W12
11
0
2
4
6
8
10
Number of utterances used to learn baseforms (M)
Figure 3-3: Word Error Rate (WER) as a function of the number of example utterances used
to adapt the underlying lexicon. We show results for using the Cascaded Recognizers Model,
the PMM, the Graphone model and the Expert Dictionary
r
Avg. # b per w
WER (%)
0.2
1.25
11.2
0.1
2.10
11.0
0.05
2.21
11.5
Table 3.1: By varying a threshold r over the weights learned in the PMM, we can incorporate
multiple baseform pronunciations for individual words.
multiple pronunciations, such as "either". It probably does not make sense, for example,
to be multiplying the acoustic scores of one utterance pronounced iy dh er with a second
pronounced ay dh er.
Lastly, another potential pitfall of the cascaded recognizers approach is that unless special
care is taken to customize the pruning procedure, acoustic variation will inherently cause the
pronunciation search space to become successively smaller as the compositions prune lowprobability paths. This is especially problematic when considering noisy utterances. Indeed,
even with the clean speech comprising the PhoneBook corpus, by the 11th utterance, N-best
lists produced by the cascaded recognizers contained an average of just 10.7 entries.
To illustrate the performance of the PMM, we show in Figure 3-3 the WER obtained
by generating a lexicon according to Equation 3.3 (i.e. taking the highest scoring pronunciation and using it unweighted in the lexicon) after two iterations of EM. This stopping
criterion was determined by constructing a development set of 1,500 previously discarded
PhoneBook utterances and running recognition using lexicons generated after each EM iteration. Alternatively, EM could have been run to convergence and then smoothed, again with
the aid of a development set.
While the PMM requires slightly more data to achieve the lowest reported WER of the
cascade approach (11.5%), it is eventually able to do so once all 11 training utterance are
incorporated into the mix. It is clear from the figure that with only a single training example
EM begins to over-fit the acoustic idiosyncrasies of that particular example. Though not
shown in the figure, this effect is magnified for small amounts of training data when EM is
run for a third and fourth iteration.
One big advantage of the PMM approach is that it directly models multiple pronunciations
for a single word, an avenue we begin to explore with a second set of experiments. We use a
simple weight threshold 0 ,,b > r, to choose baseforms for inclusion. As in the single baseform
case, we initially discard the weights once the baseforms have been chosen, but we ultimately
use them during decoding later on in experiments with stochastic lexicons.
Table 3.1 shows WER obtained by recognizers with lexicons trained with all 11 utterances
generated under varying values of r. Choosing
T
= 0.1 yields the best reported WER of
11.0%, a 1.4% absolute improvement over the expert-baseline. It's interesting to note that
this threshold implies an average of 2.1 pronunciations per word, almost double that of the
expert lexicon which has 1.08. We also plot WER using varying thresholds as a function of
the number of training utterances used in Figure 3-4. As can be seen, WER improves when
using r = 0.1 even with less than 11 utterances.
3.2.3
Results using Stochastic Lexicon
So far we have been using a threshold to figure out how many baseforms to include per
word. During decoding we throw out each baseform's associated score and weight all of
them equally. In this part we experiment with including as many baseforms as possible while
keeping their associated weight.
We run our experiments on the same test set as before by including the top 100 baseforms.
The intuition is that since most of the weight will be concentrated on the top 5 baseforms
Lexicon Adaptation using Acoustic Examples
---
top baseform
.
-- tau-0.2
- -
tau=0.1
-o-- tau-0.05
14-
PMM
--
11
0
1
2
3
4
S
8
7
8
Number of uttrances used to learn baseforms (M)
9
10
11
Figure 3-4: Word Error Rate (WER) as a function of the number of example utterances
used to adapt the underlying lexicon. We show results for using the un-weighted learned
pronunciations while varying the T threshold, as well as, using the PMM (stochastic lexicon)
including an arbitrary large amount should not hurt our scores. Again we notice improvements: the WER for the phoneme PMM drops to 10.6 while the WER for the phone PMM
drops to 9.9 which results in a 25% relative improvement over the expert-crafted lexicon.
Figure 3-4 plots WER using the stochastic lexicon while training on less than 11 utterances.
As can be seen, the stochastic lexicon out-performs using an unweighted lexicon and requires
only 6 training examples to achieve its best score.
3.3
Training Pronunciations using Noisy Acoustic Examples
While the PhoneBook data used for training simulated ideal training conditions by matching
the test data and the AM used, we now show the utility of un-matched speech collected
cheaply from a cloud-service in generating high-quality pronunciations in a fully automatic
fashion. Models that incorporate acoustic information into lexicon adaptation become particularly useful in domains where acoustic data is cheaper to obtain than expert input. In [24],
example utterances of spoken names are obtained in an unsupervised fashion for a voicedialing application by filtering for interactions where the user confirmed that a call should
be placed. Unfortunately, not all domains are amenable to such a convenient context-filter
to find self-labeled utterances.
To collect arbitrary acoustic data, we turned to the Amazon Mechanical Turk (AMT)
cloud-service. AMT has been described as a work-force in the cloud since it enables requesters
to post web-based tasks to any workers willing to accept micro-payments of as little as
$0.005 upon completion. The service has become popular in the natural language processing
community for collecting and annotating corpora, and has recently been gaining use in the
speech community. In [29], we were able to collect over 100 hours of read speech, in under
four days.
In this work, we used a similar procedure to augment our PhoneBook corpus with another
10 example utterances for each of its 2,000 words at a cost of $0.01 per utterance. Whereas
in [29] we took care to filter the collected speech to obtain high-quality sub-corpora, we
took no such precautions when collecting these example utterances. Thus, in addition to
other sources of mismatch between the data and our acoustic model, this noisy data poses
a challenge to even a recognizer built on expert pronunciations. Due to acoustic mismatch
between the utterances and our acoutic models, running the expert baseline recognizer over
these 20,000 utterances yields a very high WER of 50.1%. Of course, since we could make
no guarantees that the worker even read the word, the true error rate is unknown.
It might seem, then, that using this data to generate valid pronunciations is a dubious
exercise. Indeed, this data set confounds the cascading recognizer configuration since a single
noisy utterance can throw off the entire cascade. Fortunately, the PMM approach has the
nice property that a few noisy scores do not significantly affect the totals.
Repeating a subset of the experiments of the previous section, we again show four iterations of the PMM approach, using the PhoneBook utterances alone, AMT-PhoneBook combined utterances, and the AMT-collected corpus alone.
Despite the noisy nature of the
cloud-collected corpus, Table 3.2 shows that there is little degradation in WER when using
# Utts. Iter.1
Phonebook
AMT
Phonebook+AMT
11
10
21
12.3
12.3
12.3
2
11.5
12.0
11.6
3
11.7
13.0
11.6
4
12.0
15.3
12.0
Table 3.2: PMM results incorporating spoken examples collected via Amazon Mechanical
Turk.
all 21 utterances for every word. Perhaps more pleasing is the fact that generating pronunciations based on just the AMT-data still manages to out-perform even the expert generated
pronunciations, achieving a WER of 12.0% when using the top un-weighted learned baseform,
compared with 12.4% for the experts.
3.4
Analysis of Learned Baseforms
In order to quantify some of the differences between the expert and learned baseforms, we ran
NIST align software to tabulate differences between the reference expert baseform, and the
top choice hypothesis of the PMM model. Of the 2000 baseform pairs, 83% were identical,
while the remainder mostly contained a single substitution. Most of the substitutions involved
vowels, typically a schwa. Only 2% of the data contained an additional insertion or deletion.
Most of these involved retroflexed vowel sequences.
Table 3.3 shows examples of common confusions including vowel and consonant substitutions, vowel/semi-vowel sequence perturbations, syllable deletions, and outright pronunciation corrections. Although the latter were few, it was encouraging to see that they did
occur.
Table 3.4 shows the top 10 most confused phoneme pairs, while Table 3.5 and Table 3.6
show the most deleted and inserted phonemes.
3.5
Phone Versus Phoneme Pronunciations
As described in Section 2.1, our recognizer uses a lexicon of phoneme transcriptions to decode.
This is made possible by the use of a manually crafted pronunciation rules FST [19] that
expands sequences of phonemes to all their possible phone realizations. Given the promising
Word
parishoners [sic]
traumatic
winnifred
crosby
melrose
arenas
billowy
whitener
airsickness
Isabel
Dictionary Baseform
p AE r ih sh ax n er z
tr r AO rn ae tf &xkd
w ih n ax f r AX dd
k r ao Z b iy
m eh I r ow Z
ER iy n ax z
b ih l OW iy
w ay TF AX n er
eh r SH ih kd n EH s
AX S AA b eh l
Top PMM Baseform
p AX r ih sh ax n er z
tr r AX m ae tf ax kd
w ih n ax f r EH dd
k r aa S b iy
m eh I r ow S
AX R iy n ax z
b ih I AX W iy
w ay TD n er
eh r S ih kd n AX s
IH Z AX b eh I
Table 3.3: Example baseform changes between expert dictionary and top PMM hypothesis.
Phonemes involved in the difference have been capitalized.
Confusion Pairs
AX
#
of Occurences
=>IH
37
AO==>AA
ER
>B
R =EB
AX--> EH
31
26
23
10
Z =
S
10
AX
>AA
IH==>AX
AX==>AH
AH=>AX
10
9
9
6
Table 3.4: Top 10 phoneme confusion pairs for Phonebook lexicon
Deleted Phonemes
AX
#
of Occurences
12
TD
5
Y
4
S
3
AO
2
Table 3.5: Top 5 Deleted Phonemes in Phonebook lexicon
#
Inserted Phonemes
AO
AX
N
TD
L
of Occurences
18
11
4
2
2
Table 3.6: Top 5 Inserted Phonemes in Phonebook lexicon
I
0.2
0.15
0.1
0.05
0.025
0.01
Iter1
13.2
12.8
2
3
4
11.9
11.5
11.5
11.7
11.1
10.7
10.3
10.8
11.1
11.0
11.0
11.0
Table 3.7: PMM results for learning phone pronunciations from PhoneBook corpus.
results shown in previous sections, we set out to learn phone transcriptions of words which
would allow us to bypass the pronunciations rules file and move closer to a fully automatically learned speech recognizer. The FST implementation of the recognizer would then be
simplified to R = C o L o G where L is now a lexicon FST that maps phone sequences to
words. Note that since we are learning pronunciation in an isolated word setting we avoid
having to worry about cross-word pronunciation variation.
To automatically learn phone transcriptions of words we first have to train a graphone
model instead of the graphoneme model we were using previously. This is done by using the
pronunciation rules files to expand all the 200k phoneme transcriptions in our expert crafted
lexicon to all their phone realizations and train our graphone model on this phone lexicon.
We proceed by using our trained graphone model with a modified PMM implementation to
learn phone transcriptions. The results are displayed in Table 3.7. The table displays WER
scores by varying thresholds for baseform inclusion and number of EM iterations.
We notice that by using a threshold of r > 0.025 we achieve significant gains over the
phoneme PMM. It is interesting to note that our best phoneme PMM lexicon contained
on average 2.1 baseforms per word. When we used the pronunciations rules file to expand
these baseforms we ended up with an average of 5.3 baseforms per word. Our new phone
PMM lexicon contains an average of 5.1 baseforms per word which could indicate that our
pronuciations rules file tends to overgenerate phone realizations which in turn expands the
search space and leads to lower scores.
3.6
Summary
In this chapter, we have presented two Bayesian models that learn pronunciations from
isolated word examples: The Cascaded Recognizer and the Pronunciation Mixture Model
(PMM). We reported on experiments that produce better-than-expert pronunciations for
isolated word recognition. We also showed an inherent robustness to real world noisy data
when using the PMM. We experimented with using our learned pronunciations in both an
unweighted dictionary, as well as, a stochastic lexicon.
To understand why our learned
pronuciations outperform expert baseforms, we also described some of the differences between
our learned pronunciations and the expert pronunciations.
Chapter 4
Discriminative Pronunciation Training
for Isolated-Word Speech Recognition
As a refinement step, we attempted to adjust the pronunciation weights proposed by the
phoneme PMM using discriminative training to directly minimize errors. These new weights
are then to be used in a stochastic lexicon during decoding. The hope is that discriminative
training can address some of the concerns that maximum likelihood training cannot:
e Word Confusability: On average the PMM introduces more pronunciations then the
expert lexicon without taking into account that these extra pronunciations might be
close to or the same as pronunciations for other words. Hence, the PMM could potentially be increasing word-confusability and therefore increasing WER during testing.
e Viterbi Decoding: Our ASR system uses the Viterbi approximation while decoding
i.e. it does not marginalize or sum over the probabilities of all pronunciations mapping
to a word hypotheses. The Viterbi approximation forces the decoder to select the highest scoring pronunciation for any given word hypotheses. Since the PMM marginalizes
over all pronunciations to maximize data likelihood, there is mismatch between our
decoding and our training method.
4.1
Global Linear Models for Lexicon Learning
In this section, we adapt the approach presented in section 2.4 to discriminatively train the
weights of our pronunciation lexicon. Without loss of generality, we can modify Equation 3.1
as follows.
w*, b*
argmaxP(ub)P(b,w)
w.b
(4.1)
We now recognize word-pronunciation pairs instead of plain words.
For discriminative training, we consider the model below:
W b
arg max < a, CD(u, b, w) >
w~b
(4.2)
While <D(u, b, w) can represent some complex features such as word, letter, phoneme or letterphoneme pair ngrams, we only use the ones listed below:
* <bo(u, b, w) = log P(uIw) +log P(b, w) the acoustic model score and pronunciations score
under our our PMM-lexicon recognizer. We define the rest of the features as:
<b(u,b,w)=
1
ifb=pand w=1(
0
otherwise
This reduces equation 4.2 to the following:
w* b*
arg max ao(log P(vb) + log P(b, w)) + ab,w
w,b
(4.4)
The rest of our model is described as follows:
" Our training examples are (ui, wi). We use the same training set used in our MLE
approach, except that now we consider an utterance and its transcription individually,
i.e we do riot group all utterances of the same word together.
" GEN(u) is an n-best list or lattice of recognized word-pronunciation pairs (b, w), we
use our MLE PMM recognizer to generate GEN(u).
We optimize the parameters a as follows:
INPUT: Training examples (ui, wi)
INITIALIZATION: Set a = 0
ALGORITHM:
for t = 1 -.T,i =1. ...
N do
(b*, w*) = arg nax(b,w)EGEN(ui) F(ui, b, w) ' a
(boraie, wi) = arg mnax(b,w)EGEN(un) 4(ui, b, w)
a
if w* / wi then
a = a + I(uj, borawe, wi) - ID(uj, b*, w*)
end if
end for
OUTPUT: a
Proof of convergence:
Let GEN(uj) = GEN(uj) - {(b, w)
Iw
= wi}. In other words GEN(uj) is the set of
incorrect candidates for an utterance ui. We will say that a training sequence (aj, wi) for
i = 1... M is separable with margin a > 0 if there exists some vector U with |Ull = 1 such
that
Vi, V(p,l) c GEN(uj), Vb s.t. (b, wi) E GEN(uj)
U - <b(ui, b, wi) - U . <b(ui, p, l) > a (4.5)
The proof is adapted from proofs for the tagging case in [12]. Let ak be the weights before
the k'th mistake is made. It follows that a' = 0. Suppose the k'th mistake is made at
Take (b*, w*) to be the output proposed at this example, (b*, w*) =
the i'th example.
arg max(b,w)EGEN(uj)
(ui, b, w) ' ak and borace to be the highest scoring correct pronunciation
under this model, (baracie, wi) = arg max(b,W)EGEN(u-) 1(ui, b, w)
ak.
It follows from the
w=wuiwi) - <b(ui, b*, w*). We take inner products
algorithm updates that ak+1 - ak + <b(ui, borace,
of both sides with the vector U:
U -ak+1
=
u - a + U .,<(ui, borae, Wi) - U . <b(ui, b*, w*) > U - ak + o
(4.6)
where the inequality follows because of the property of U. Because a' = 0, and therefore
U . a, = 0, it follows by induction on k that for all k, U - ak+1 > ku. Because U- ak+1 <
IU|||Iak+1||, it follows that Iak+1I > ka.
We also derive an upper bound for |lak+ 12:
Sak+1 12 _
k
2
+ Ib((ui, borade, wi) - <b (ui, b*,
+2ak - (<b(ui, borade, wi) Ilak2+R
<b(ui, b*,
w*)I|2
w*))
2
Where R is a constant such that
Vi, V(p, l) e GEN(uj), V(b, w) s.t. w = wi and (b,w) e GEN(uj) ,
||<b(ui, b, w)-< (uip,l)||
(4.7)
Note that 2ak . (<b(ui, borade, wi) - <b(ui, b*, w*))
0 because (b*, w*) is the highest scoring
candidate for ui under the parameters ak. It follows by induction that Iak+ 12
kR 2.
Combining the bounds Ilak+1II > ku and IIak+1 II kR 2 give the results for all k that
k2U 2 < ||ak+11 2 < kR
4.2
2
--> k < -
(4.8)
Implementing the Algorithm as an FST
Our current implementation of the perceptron relies heavily on the FST toolkit [20].
We first build an FST T that expands a word to all possible word-pronunciations present
in the PMM lexicon. Here a word-pronunciation is basically one entry that is a word concatenated with its pronunciation, i.e. apple -- apple-ae-p-ax-l. By inverting the FST, inv(T),
we now have an FST that can map word-pronunciations back to words.
GEN(u) is represented as a lattice of word pronunciation pairs D" generated by a recognizer that uses the PMM lexicon but is modified so as to hypothesize both a word and
its pronunciation. We model the search space of this recognizer in a similar manner to that
described in section 2.1:
N = C o PoL'
o GT
(4.9)
R
Where C is the context-dependent labels and P are the pronunciation rules. We modify
the baseforms FST L' so that it can map a string of phones to a word-pronunciation where
the pronunciation side matches the input string of phones. Each mapping is weighted by its
PMM weight P(blw). We now have to modify the LM FST so that the LM score is still used
while maintaining the hypothesized word-pronunciation. This is done by composing G with
inv(T) and then projecting its input to its output. We call this FST GT.
We can also represent our feature vector weights as an FST. We build an FST L that maps
every word-pronunciation w-p to itself weighted by
a(L).
We can now represent Equation 4.2
as:
w*,b* = bestpath(aoLo D)
(4.10)
Where bestpath(.) finds the highest scoring path in a lattice.
Our perceptron algorithm is now implemented as follows:
INPUT: Training examples (ui, wi)
INITIALIZATION: Set all arcs weights in L to zero and ao
1
ALGORITHM:
for t= 1--T,i= 1-..Ndo
(b*,w*) = bestpath(aoL o Du,)
if w* f wi then
(boracle, wi) = bestpath(aoL o Du, o F,,)
Update ao and L appropriately.
end if
end for
Where F., is an FST that only accepts word-pronunciations such that the word side
matches wi.
iter.
0
1
2
3
4
5
6
7
8
9
10
Train
4.7
4.0
3.9
3.0
2.7
2.7
2.6
2.7
2.5
2.6
2.5
Test
10.6
10.2
10.1
10.7
10.4
10.5
10.4
10.9
10.7
10.3
10.4
Test-Avg
n/a
10.4
10.4
10.3
10.3
10.2
10.2
10.1
10.1
10.2
10.1
Table 4.1: Results for Discriminative training in WER on PhoneBook corpus.
4.3
Experimental Setup and Results
We ran the perceptron for multiple iterations on the training set. We report word error rates
on the training set and test set. We also test on a refinement of the perceptron algorithm
called the "average parameters" method. We define a'!" to be the value for the s'th parameter
after the i'th training example has been processed in pass t over the training data. Using the
"averaged parameters" a, = 1/nT E_=1.T
...n a
has been shown to perform significantly
better than the final parameters a.". The results are shown in Table 4.1. Note here that
iteration zero corresponds to using the same lexicon weights learned using the phoneme
PMM discussed in section 3.2.3. As expected, WER on the training set goes down with
every iteration since we are directly optimizing the feature weights on the training set. WER
on the test set goes down until iteration 3 where we begin overfitting. Since the averaged
parameters method corresponds to a smoothed out version of the learned parameters, we see
a steady decline in WER. The best achievable score is 10.1%.
Using the above described discriminative model results in an improvement in WER. While
these results are very promising, we leave further investigations into the approach for future
work.
4.4
Summary
In this chapter, we introduced a novel mathematical framework for discriminatively training
lexicon weights, along with convergence guarantees. We showed that more WER reductions
could be acheived using this method. We leave further experiments using more complex
features for future work.
48
Chapter 5
Pronunciation Mixture Model for
Continuous Speech Recognition
In this chapter, we extend our approach to the more complex task of learning pronunciations from continuous speech and testing on continuous speech. This is a more attractive
application than isolated-word speech since continuous data is more readily available.
To extend our isolated-word PMM requires additional considerations.
We once again
model the sequence of phonetic units as a hidden variable, however, the general ASR problem
is now a search for the most likely string of words W* =w,
W =argmaxP(Wlu)=argiax
w
w
...
, w* given an utterance u:
P(WBlu)
(5.1)
BGB
where now, B e B is a sequence of phonemes or phones that might span multiple words
and include silence. Thus, for the continuous case, we consider silence to be a phonetic unit
and denote it with a '-'. For example, a possible phoneme sequence B for an utterance with
transcription "the boy ate the apple" could be "dli ax b oy - - ey td - dh ah ae p ax 1".
Equation 5.1 can be decomposed as follows:
W =argmax(
P(W)P(B|W)P(u|W,B)
(5.2)
BGB
Where P(W) is the language model, P(B|W) can be computed using a stochastic lexicon
and P(ulW, B) can be approximated with the acoustic model P(ulB). Note that the speech
recognizer used in this thesis uses standard Viterbi approximations during decoding. This
reduces Equation 5.2 to the following:
arg max P(W)P(B|W)P(u|B)
W
(5.3)
Wn)
EM for Estimating Phoneme Pronunciations
5.1
We now extend the Pronunciation Mixture Model (PMM) framework developed for isolated
word recognition in Chapter 3 to learn the appropriate weights that can model P(B|W) in
continuous speech.
Our training data is comprised of A utterances and their transcriptions {Ui, Wi} where
W = wi,
,w.
We parameterize the log-likelihood as follows:
log P(ui, Wi6)
log
=
i=1
i=1
P (ui, W, B, V|)10
FEB OeGI(Wi.B)
where V, is an additional hidden variable defined to segment the phonetic sequence B
into k baseforms while deleting the silences. Thus, @ is drawn from possible segmentations
J(W, B) and can be indexed to retrieve a particular word-baseform pair. For example:
"
# 1(dh
ax b oy - - ey td - dh ah ae p ax 1) = dh ax
e i/ 2 (dh ax b oy
- -
ey td - dh ah ae p ax 1) =b oy
e V)3 (dh ax b oy
- -
ey td - dh ah ae p ax 1) = ey td
" V)4 (dh ax b oy
- -
ey td - dh ah ae p ax 1) = dh ah
o V 5 (dh
ax b oy - - ey td - dh ah ae p ax 1) = ae p ax 1
We can now further decompose the term as follows:
P(ui, Wi,, B,
) =
P(ujlB)P(w,.
w
bi,
,b)
where bi = @tj(B).
Our acoustic models are trained such that when B is the phoneme
alphabet, a pronunciation bi is context independent and the equation above can be rewritten
as:
ki
P(ui,
l Wi,
iB)
kelhood,
B, V@)
= PO(uu
(5.4)
(B)
j=1
where 6w,bjo = P(w) , bj). Our log-likelihood then becomes:
>3
Ylog P(uiWji6) = >logE
i=1
i=1
BEB
ki
P(uilB)
# ETW(W ,B3)
JOw,,,(B)
j=1
The parameters, 0, are initialized to our graphoneme n-gram model scores and multiple
iterations of the EM algorithm are run.
E-step:
Mo[w,p3=
>3
P (B, @| Iui, W, 0) M[p, w, Wi,, B, V)]
M-step:
Mo[w,p]
O7M _
.P Et',P'EV XB MOIL,
p]
where M[p, w, W , B, V)] is the number of times word w appears in W aligned with the pronunciation p.
M[p, w, Wi, B, @b]= |{j :,@j(B) = p and
W w|11
(5.5)
The weights learned are directly used in a stochastic lexicon for decoding continuous
speech. The term P(BIW) in Equation 5.2 can be computed as:
k
P(B|W) =
(5.6)
,EP(Wi,B) j=1 'PEB1 uEjP
5.1.1
EM for Estimating Phone Pronunciations
The phonetic rules our recognizer applies between the phoneme-based lexicon and the contextdependent acoustic models create context dependencies across words at the phone level. A
misguided attempt to apply the PMM directly to learning phone pronunciations would ignore
the independence assumption made in Equation 5.4, which is no longer valid. In this section,
we explore a model that assigns a probability to a word-pronunciation pair given the last
phoneme of the previous word's pronunciation:
P(wi, bi,. ...
, wn, bn)
=
P(wi, bilw1, bi, ... ,wi_ 1, bi-_1)
n
j=1
~j7JP(wi,b~Ibs_1)
i=1
~~H P(wi, biILAST(bi_1))
i=1
Where LAST(b) is the last phone in the phone sequence b. Here we make several independence assumptions, the first is similar to the Markov assumption made in n-gram language
modeling, the second references the fact that only the ending of the previous word's pronunciation can affect the current word's pronunciation. Our new features 6wb|p = P(w, b~p),
where b E t3 and p is a single phone, can now be used in equation 5.4 as follows:
|wi
P(ui, W, B, )) = P(u I|B)
17
0
ik(B),wm L
AST(pg - (b))
(5.7)
j=1
5.2
Implementation Details
Here we discuss some of the issues faced when implementing the continuous speech PPM for
phoneme pronunciations. The framework is split into two steps: we first build a constrainedgraphoneme recognizer and run it on each training utterance to generate an n-best list of
graphonemne hypotheses for the entire utterance. After creating an n-best list of graphoneme
hypothesis for each utterance, we iterate through the entire list computing expected counts
for every word and its pronunciation (E-step) and then normalizing the expected counts into
a probability distributions (M-step).
We first build a graphoneme recognizer for continuous speech. The search space is modelled by the following FST:
GR=CoPoGP*
(5.8)
Where C maps phones to context-dependent model labels and P are the pronunciation
rules. Here GP* is our graphoneme n-gram that has been expanded to allow the recognition
of multiple words. This is done by taking the graphoneme n-gram used for the isolated-word
PMM, concatenating it with an FST that emits a # symbol and then performing closure on
this new FST. The # symbol is used as a word boundary. One problem with this approach
was that the GP* FST was too big to build. To overcome this problem, we pruned out all the
paths in the graphoneme n-gram that do not appear in the top 700 pronunciations for any
word in our training set. This significantly reduced the size of GP* so that the recognizer
search space could be efficiently built.
We then distributively process each utterance and its word-transcription in the training
set. We first expand the word transcription into all graphoneme sequences whose letter side
match the letters of the transcription and encode those paths as an FST L. This FST is then
composed with the GR FST in Equation 5.8 to constrain the search space of the graphoneme
recognizer. We call this recognizer the constrained-graphoneme recognizer. Recognition is
performed on the utterance in question and an n-best list of 5000 hypotheses is stored on
disk. Figure 5-1 shows an example n-best list for the utterance: "houston texas". The three
scores at the beginning of each line represent the total score, the acoustic model score and the
graphoneme n-gram score respectively. The total score is the log-domain joint probability
P(ui, Wi, B, V) used in Equation 5.4, the AM score is P(ui IB) and the graphoneme n-gram
score is fl, P(wj, bj).
After generating an n-best list for each utterance in our training data, we can proceed
with the E-step of the first EM iteration. We keep a double hash table for every word and
pronunciation that maps to the expected counts of seeing that word aligned with that specific
pronunciation. For every n-best list, we first normalize the total scores to sum to one by
59.0655 83.9573 -24.8918 h/hh o/ /y u/uw s/s t/t o/ax u/n # t/t e/eh x/kd /s a/ax s/s
55.4080 83.1458 -27.7378 h/hh o/ /y u/uw s/s t/t o/ax n/n # t/t e/ey x/kd /s a/ax s/s
54.4675 83.9573 -29.4899 h/hh o/ /y u/uw s/s t/t- o/ax n/n # t/t e/eh x/kd /s a/ax s/s
50.8100 83.1458 -32.3359 h/hh o/ /y u/uw s/s t/t-
o/ax
n/n # t/t e/ey x/kd /s a/ax s/s
49.5057 78.9219 -29.4162 h/hh o/ /y u/uw s/s t/t o/ax n/n # t/t
e/eh x/kd /s a/ax s/z
47.5087 83.4593 -35.9506 h/hh o/ /y u/uw s/s t/t o/ax n/n # t/t e/eh x/kd /s a/ax s/s
47.2153 83.9573 -36.7420 h/hh o/ /y u/uw s/s t/t o/ax n/n # t/tf e/eh x/kd /s a/ax s/s
47.0608 78.0224 -30.9616 h/hh o/ u/uw s/s t/t o/ax n/n # t/t e/eh x/kd /s a/ax s/s
46.9454 83.9573 -37.0119 h/hh o/ /y u/uw s/s t/tq o/ n/en # t/t e/eh x/kd /s a/ax s/s
46.8134 77.7686 -3019552 h/ /y o/ u/uw s/s t/t- o/ax n/n # t/t e/eh x/kd /s a/ax s/s
Figure 5-1; Constrained Graphoneme Recognizer n-best hypothesis for "houston texas" along
with total, AM and LM score.
computing P(B,
b|ui, W)
=
P(uiW.BV')
.
We then iterate through the n-best list and
increment the double hashtable for every aligned word-pronunciation with its normalized
total score. After computing all the expected counts across all the generated n-best lists,
we normalize the counts according to the M-step equation. We now have a joint-probability
distribution for word-pronunciations i.e. P(w, b) =
0
,.b*
To compute the remaining EM iterations, we again iterate through all the n-best lists to
compute:
P(B,I), u, Wi) = P(Blui)P(B,@, W) = P(Blui) JO
3
,b
(5.9)
Where P(Blui) is the AM score extracted from the n-best list and Owjbj are the parameters
computed during the previous EM iterations.
After computing this joint probability for every hypothesis, we again normalize these probabilities to sum to one and compute expected counts for all the aligned word-pronunciations
in the n-best lists in a similar manner to what was described before. Finally, we once again
compute the updated features 0 using the M-step equation.
5.3
Experimental Setup
Experiments using the SUMMIT landmark-based speech recognizer [15] were conducted in
two domains: a weather query corpus [13] and an academic lecture corpus [31].
5.3.1
Experimental Procedure
To evaluate the performance of our PMM model we used the following procedure for both
the weather and lecture domains. We begin by cleaning the acoustic model training set by
removing utterances with non-speech artifacts to generate a training set for PMM pronunciations. We then prepare two recognizers, the first based on manually created pronunciations
of all the words in the training set and the second a learned PMM recognizer that contains
all the pronunciations generated by our PMM model. We then compare the Word Error Rate
(WER) of both recognizers on a common test set. Thus, both recognizers use precisely the
same vocabulary, but the pronunciations are chosen or weighted either by human or machine.
Although the expert-lexicon leaves pronunciations unweighted, it does include a number of
entries with multiple pronunciations. To keep the number of PMM pronunciations included
in the search space to a reasonable size we use a 0.005 threshold to prune out low probability
pronunciations.
The weather query corpus is comprised of relatively short utterances, with an average
of 6 words per utterance. After pruning the original training and test sets of all utterances
containing non-speech artifacts, we ended up with a 87,600 utterance training set with an
1,805 word vocabulary and a 3,497 utterance test set. The acoustic models used with this
corpus were trained on a large data set of telephone speech of which this corpus is a subset.
The lecture corpus contains audio recordings and manual transcriptions for approximately
300 hours of MIT lectures from eight different courses and nearly 100 MITWorld seminars
given on a variety of topics [31]. The lecture corpus is a difficult data set for ASR systems
because it contains many disfluencies, poorly organized or ungrammatical sentences, and
lecture specific keywords [8]. Compared to the weather corpus the sentences are much longer,
with about 20 words per utterance on average.
As in the weather domain, we discard utterances that contain non-speech artifacts from
the training set used to train the acoustic model and end up with a 50K utterance training
set. We then cleaned the remaining utterances to create a 6,822 utterance test set. We report
results on training pronunciations and decoding with back-off maximum likelihood trained
Acoustic Models [8]. We leave the use of discriminatively trained models for future work.
The back-off maximum likelihood model uses a set of broad phonetic classes to divide the
classification problem originating from context-dependent modeling into a set of subproblems.
The reported results differ from those in [8] because we use a 25K word vocabulary of all the
words in our training set. The original paper uses a 35K word vocabulary with some words
absent from the training set.
Graphone/Graphoneme Training
A 150,000 word dictionary of manually generated phoneme pronunciations was used to train
the graphoneme n-gram parameters according to the procedures described in [36].
To train our graphone model, one might be tempted to simply expand the phoneme-based
lexicon according to the pronunciation rules learned in Section 2.1. Unfortunately, given the
manner in which our acoustic models were trained, the beginnings and endings of pronunciations are context-dependent at the phone level. Thus, we must expand all the sentences
in our weather and lecture training corpora first to their phoneme pronunciations using the
manually crafted dictionary and then to their phone variations using the pronunciation rules.
These phone pronunciations were properly generated since the pronunciation rules had access
to the context of a word in a sentence.
5.3.2
Experimental Results
The results for learning pronunciations can be seen in Figure 5.1. A baseline using a lexicon
generated directly from graphonemes is shown to be significantly worse than both experts and
the PMM. More interestingly, phoneme PMM pronunciations achieve more than a 1.2% WER
reduction on the weather test set and a 0.4% WER reductions on the lectures test set over the
hand-crafted lexicon. Both results were deemed statistically significant (p<0.001) using the
Matched Pair Sentence Segment Word Error test. These results were achieved by running the
EM algorithm until convergence which took around 8 iterations. It is important to note that
Graphoneme L2S
Expert
Phoneme PMM
Phone PMM
Context PMM
Weather
11.2
9.5
8.3
8.3
8.3
Lectures
47.6
36.7
36.3
36.1
37.7
Table 5.1: Results in WER on both the Weather and Lecture corpora using the PMM, the
expert dictionary and the Graphoneme model (%)
we are learning these pronunciations without human supervision and hence this technique
can be reliably used to predict pronunciations for out-of-vocabulary words from continuous
speech.
We also show improvement over a baseline of expert crafted pronunciations by
training on the same data used for training our acoustic models. This suggests that not only
are we learning better-than-expert pronunciations, we are also allowing more information to
be extracted from the training set that can complement the acoustic models. This smaller
size vocabulary of the weather domain could explain the higher gains achieved: since the
PMM includes multiple pronunciations per word, this might make them more confusable a
fact that is more apparent in the 25k vocabulary Lecture domain. Another disadvantage of
the PMM approach when applied to the Lecture domain is due to the longer sentences present
in that corpus. The sheer size of some of the sentences caused the constrained graphoneme
recognition step to fail. This happened to about 2k utterances of our 50k utterance training
set because no complete path could be found through the FST lattice.
We also test on learning phone pronunciations in a similar context-independent setup as
that of phonemes (Section 5.1). The results are referenced as "Phone PMM" in Table 5.1.
One advantage of learning phone pronunciations in a context-independent setup is that we are
no longer using the pronunciation rules that might be over-expanding the search space. This
fact is made apparent in Table 5.2 where we see an increase in the number of pronunciations
per word but a smaller search space.
A second reason to favor removal of phonological rules from the recognizer is simply that,
when the lexicon is trained appropriately, they appear to be an unnecessary complication.
It is also interesting to note that Phone PMM training is faster and requires only 4 EM
iterations to achieve convergence. The disadvantage of the direct expansion approach we
Weather Lexicon
Expert
Phoneme PMM
Phone PMM
Context PMM
Lecture Lexicon
Expert
Phoneme PMM
Phone PMM
Context PMM
Avg
#
Prons
1.2
3.15
4.0
63
Avg. # Prons
1.2
1.8
2.3
8.9
#
#
States
32K
51K
47K
253K
States
226K
501K
243K
565K
#
Arcs
152K
350K
231K
936K
# Arcs
1.6M
7.6M
1.2M
2.7M
Size
3.5 MB
7.5 MB
5.2 MB
22 MB
Size
36 MB
154 MB
28 MB
61MB
Table 5.2: Lexicon statistics for the weather and lecture domains.
have described so far is that phone pronunciations are context-dependent with respect to the
underlying acoustic models, a fact that is not represented in the learned lexicon.
We tried to model these cross-word dependencies by using the context-dependent model
described in section 5.1.1. Whereas the move from graphonemes to graphones reduces the
complexity of our recognizer, incorporating context dependent models grows the search space,
as seen in Table 5.2. From the results in Table 5.1, however, it is clear that they are an unnecessary complication, and even hurt performance in the lectures domain. The degradation
is likely due to a problem of context sparsity, which is caused by a greater mismatch between
training and test in the lecture corpus.
5.4
Analysis of Learned Baseforms
We first attempt to measure how much of the expert lexicon the PMM was able to recover.
For the 1.8k words in the Weather Corpus vocabulary, the expert lexicon contained 2252
pronunciations (since the expert lexicon allowed multiple pronunciations for some words)
of these, 2002 or 88.9% were present in the PMM lexicon with a threshold of 0.005 (all
pronunciations with weights less than 0.005 were were not included in the PMM lexicon).
The average rank of the pronunciations that were found in the PMM lexicon was 1.2 and the
average weight was 0.73 . Given that most of the expert pronunciations were included in the
PMM with a relatively high weight, we believe that little gain can be achieved by combining
the expert lexicon and the PMM lexicon.
Confusion Pairs
AX=>IY
AX=>IH
AA ->AE
R
>ER
ER
>R
AA =>AX
IY=>Y
AX=>EH
UW
>AX
AX==>AA
#
of Occurences
22
19
16
15
14
10
9
9
9
8
Table 5.3: Top 10 confusion pairs for Weather Domain lexicon
Deleted Phonemes
AX
AO
TD
R
AA
#
of Occurences
15
13
4
3
3
Table 5.4: Top 5 Deleted Phonemes for Weather Corpus lexicon
Similarly to Chapter 3, we again try to quantify some of the differences between the
expert and learned baseforms. We run NIST align software to tabulate differences between
the reference expert baseform, and the top choice hypothesis of the PMM model for the
Weather Corpus lexicon (A list of learned pronunciations is listed in Appendix A). Of the
1805 baseform pairs, 81.5% were identical, while the remainder mostly contained a single
substitution. We found the alignment results to be similar to the ones conducted on the
isolated-word PMM lexicon. Again, most of the substitutions involved vowels, typically a
schwa.
Table 5.3 shows the top 10 most confused phoneme pairs, while Table 5.4 and Table 5.5
show the most deleted and inserted phonemes. About 3.4% and 3.1% of the data contained
an additional insertion or deletion, respectively, more than three times the amount observed
for the isolated-word case. We found that the increase in deletions could be explained by
the fact that during continuous speech speakers are more likely to skip phonemes. Table 5.6
shows some examples where the PMM learned pronunciations with plausible deletions.
Inserted Phonemes
AX
Y
HH
IY
AO
#
of Occurences
13
7
7
6
5
Table 5.5: Top 5 Inserted Phonemes for Weather Corpus lexicon
Word
already
antarctica
asked
barbara
bratislava
clothes
Dictionary Baseform
Top PMM Baseform
ao L r eh df iy
aa r eh df iy
ae nt aa r KD t ax k ax ae nt aa r tf ax k ax
ae s KD t
ae s td
b aa r b AX r ah
b aa r b r ax
b r AA tf ax s I aa v ax b r tf ax z 1 aa v ax
k I ow DH z
k l ow z
Table 5.6: Example phoneme deletions between expert dictionary and top PMM hypothesis
for Weather Corpus.
When taking a closer look at the data to explain the increase in insertions, we found that
out of the 62 insertions in our PMM lexicon about 16 were either at the beginning or at the
end of the pronunciation. Upon closer examination, we found that most of these insertions
were happening to words with a low number of occurrence in the training data and were
erroneous. We believe that these mistakes are due to words absorbing the pronunciations
of other surrounding words. For example, the word "missed" only appeared 3 times in
the training set. Of the 3 occurrences, 2 were preceded by the word "I". In this case, the
PMM gave a high probability to the pronunciation "ax m ih s td" i.e. the pronunciation of
"I" concatenated with the pronunciation of "missed". Table 5.7 contains examples of some
reasonable insertions, as well as, many erroneous insertions. We also compare the number of
times these examples appear in the training set.
5.4.1
Non-Native Speaker Pronunciation
In this section, we use the PMM to try to analyze pronunciation differences between native
and non-native speakers. We train two PMM lexicons, one on 3k utterances of lectures given
by an Indian accented English speaker and the other on 30k utterances of lectures given by
Word
iran
wow
deep
guadaloupe
towards
missed
Dictionary Baseforin
ax r aa n
w aw
d iy pd
g w aa df ax l uw pd
t ao r dd z
m ih s td
Top PMM Baseform
AY ax r ae n
w AY aw
d iy p AX L
g w aa df ax I uw P EY
t ao r dd S IY
AX m ih s td
rotterdam
r aa tf er d ae m
AX r aa tf er d ae m
snowbase
sea
s n ow b ey s
s iy
EH s n ow b ey s
S LY s iy
#
of Occurrences
16
19
2
2
1
3
1
1
2
Table 5.7: Example of phoneme insertions between expert dictionary and top PMM hypothesis for Weather Corpus.
Confusion Pairs
III =>
#
of Occurences
AX
8
AE=>AA
S=>Z
AE==>EH
P==>PD
5
4
3
3
Table 5.8: Top 5 phoneme confusion pairs for Indian accented non-native talker
a Dutch accented English speaker. Note that this data was not part of our original Lecture
corpus training set.
We again run NIST alignment on the top pronunciations learned from both speakers
with the top pronunciations learned from the entire Lecture corpus. To make sure that the
pronunciations learned from the lectures were correctly estimated, we restricted our analysis
to words that had at least 5 examples in the training set. For Indian accented speech, we
found that out of the 400 words we compared, about 87% of the pronunciations were the
same while 12.3% contained substitutions. Only a small fraction of the differences were due to
insertions or deletions. Table 5.8 lists the top 5 confusion pairs, mainly vowel substitutions,
while Table 5.9 lists 4 word pronunciations that illustrates differences between the native and
non-native speaker baseforms. As for Dutch accented speech, we found that out of the 2.5k
words we compared, 88.8% of the pronunciations were similar and that most of the changes
were again due to substitutions. Table 5.10 lists the top 5 confusion pairs, while Table 5.11
lists some examples.
Word
dependent
therefore
Native Speaker Pronunciation
d AX p EH N D ax n td
dh eh r f ao r
Non-Native Speaker Pronunciations
d IY p AX DF ax n td
dhehrAX faor
vary
v AE r iy
v EH r iy
various
only
simple
within
v AE r iy ax s
OW n I iy
s IH m p ax l
w IH dh IH n
vEH
AO n
sAX
w AX
riyaxs
1 iy
mpaxl
dh AXn
Table 5.9: Example of differences between native and Indian accented non-native speaker
pronunciations.
Confusion Pairs
AX=>IY
AX=>IH
DD==>TD
Z=>S
AA
>OW
# of
Occurences
8
5
4
3
3
Table 5.10: Top 5 phoneme confusion pairs for Dutch accented non-native talker
Word
fluid
estimate
reverse
Native Speaker Pronunciation
f I uw AX DD
eh s t ax m AX td
r AX v er s
Non-Native Speaker Pronunciations
f I uw IH TD
eh st ax m EY td
r IY v er s
pops
p AA pd s
p OW pd s
Table 5.11: Example of differences between native and Dutch accented non-native speaker
pronunciations.
5.4.2
Slow vs. Fast Speech
We now try to quantify the differences of word pronunciations during fast and slow speech.
We conducted the following experiment: We ranked every training utterance in our Lecture
corpus training set by its average word duration i.e we divided the total duration of the
utterance by the number of words in its reference transcription to compute the average word
duration. We then split the training corpus into two parts:
" The "fast speech" training set contained the 25% of the data with the shortest word
duration.
" The "slow speech" training set contained the 25% of the data with the longest word
duration.
We discarded the remainder of the training data.
After running the PMM on both training sets, we then used the NIST alignment software
on the learned baseforms provided they had appeared at least 4 times in both training set.
We ended up with a lexicon of 1783 words.
Only about 32% of the baseforms were the same in both the "fast speech" and "slow
speech" lexicon. 46% had substitutions, 21% of the baseforms in the "fast speech" lexicon had
deletions and 14% had insertions. On average, fast pronunciations contained 5.2 phonemes
while slow pronunciations contained 5.4 phonemes. Table 5.12, Table 5.13 and Table 5.14 list
some of the top confusion pairs, insertions and deletions. We also tabulate some the example
differences in Table 5.15.
5.4.3
Word Confusion in Weather Domain and Lecture Domain
Lexicon
Since our PMM defines a conditional distribution over words and pronunciations P(blw), by
assuming that all words are equally likely i.e. P(w) = 1/IVI where V is our vocabulary, we
can compute the conditional entropy H(blw) and H(wb).
* H(blw) is a measure of disorder when trying to predict a pronunciation given a word.
Hence, it is related to how many pronunciations are allowed per word and how uniform
Confusion Pairs
S=>Z
IY=>AX
IH =>AX
R =>ER
AX>
EH
AX=>IH
Z=>S
EH
>AX
ER
> R
IY>
IH
# of
Occurences
46
38
35
35
35
33
26
26
25
20
Table 5.12: Top 10 phonenie confusion pairs between "fast" speech and "slow" speech
Inserted Phonemes
AX
TD
DD
EH
N
#
of Occurences
51
40
20
15
14
Table 5.13: Top 5 inserted phonemes in "fast" speech pronunciations
Deleted Phonemes
AX
IY
L
TD
S
#
of Occurences
130
33
33
27
27
Table 5.14: Top 5 deleted phonemes in "fast" speech pronunciations
Word
leadership
obviously
learned
thousand
"Slow" Pronunciation
1 iy df er sh IH PD
aa bd v iy ax S I iy
1 EH R n dd
th OW Y AX z ax n dd
"Fast" Pronunciation
1 iy df er sh AX
aa bd v iy S ax 1 iy
1 ER n dd
th AW z ax n dd
Table 5.15: Examples of pronunciation differences between fast and slow speech pronunciations.
-- . .........................
...
.......
..................
.....
Eile
Edit
- -- ------__ _ ___ __
=
Yiew
insert
Desktop
1oos
Window
..-
-
--
Help
H(blw) an(wt)for Weather and Lecture Domain
076
.5
-
-.
Weather H(bw)
-
Weather H(wb)
Lecture H(blw)
-
-
-
Lecture H(wlb)
-D-
0. -
- 0-3
'U
0
1
1.5
2
2.5
3
8.5
4
Ntumberof EM itrations
4,5
5
5S.
6
Figure 5-2: Plot of H(wb) and H(bjw) as a function of EM iterations for both the Weather
and Lecture domain.
their distribution is.
* H(wjb) is a measure of disorder when trying to predict a word given a pronunciation.
This is a measure of how confusable words are when using a stochastic lexicon. In
our lexicons we allow different words to map to the same pronunciation, hence H(w|b)
allows us to measure the uncertainty of being able to map back from a pronunciation
to the word that generated it.
Figure 5-2 plots H(wlb) and H(blw) as a function of EM iterations for both the Weather
lexicon and the Lecture lexicon. As can be seen, while the Weather domain lexicon has a
higher H(blw) that seems to be increasing with EM iterations, it has a much lower H(wlb)
than the Lecture domain. This confirms our initial concerns that the Lecture domain lexicon
allowed for more confusion between words due to its large vocabulary size.
5.5
Acoustic Model and Lexicon Co-Training
One of the research activities currently undertaken by the Spoken Language Systems (SLS)
research group is utilizing crowd-sourcing for data, collection. In this section, we describe
applying our PMM model to the E-book (electronic book) corpus. This corpus was collected
using Amazon Mechanical Turk (AMT) and contains a large variety of words, including
many that have never been used in SLS's prior speech applications. We first describe the
data collection process and then discuss some of the experiments performed on this corpus.
5.5.1
Real-World Data
The E-book corpus contains a list of 10K book titles, containing 30K individual words and
about 9.5K unique words. The speech data was collected using the WAMI toolkit infrastructure [16] on Amazon Mechanical Turk (AMT). 10 utterances from 10 different speakers were
collected for each book title resulting in a total of 100K utterances.
As a final step, the data was then split up into a training set, development set and test
set. For each of the 10K titles, 7 utterances from different speakers were kept for training, 1
utterance was used in the development set and the last 2 were held out for testing.
5.5.2
Baseline Approach
In this part, we describe the usual procedure to build a speech recognizer for the E-book
corpus. As a first attempt, we used our generic Acoustic Models (genericAM) trained on
telephone speech, the expert dictionary (expertLexicon) to provide pronunciations for the
9.5K vocabulary, as well as, a Language Model n-gram trained on the training data. This
approach has two limitations: the generic Acoustic Models used do riot match the E-book
data because they have been trained on a different domain. The other limitation is due to the
pronunciation dictionary: about 16% of the 9.5K vocabulary is out-of-vocabulary (OOV),
hence the expert dictionary does not contain a pronunciation for those words. To better
evaluate the performance of this recognizer, we split our test set into a 17K-utterance invocabulary test set (IV-test) where none of the words are OOV and a 3K-utterance OOV test
set (OOV-test) where each utterance has at least one OOV word in its reference. Running
the now 8K-vocabulary generic-AM recognizer on the test set we end up with a fairly poor
WER of 24.7% on the IV-test and 98.1% on the OOV-test.
In order to improve our baseline performance, we attempt to retrain our AMs using the Ebook training set. Retraining AMs requires a training set of utterances and phone-transcribed
references. To generate the phone-transcriptions from the original word-transcription, we use
a forced-alignment recognizer using our expert dictionary and our generic models. Again since
about 16% of the words are OOV, we fail to generate forced-alignments for about 15K of
our training utterances. Using these new phone alignments, we can now retrain our Acoustic
Models (referred to as expertAM). A significant WER reduction can now be achieved: 11.1%
on the IV-test and 90.0% on the OOV-test. The WEB. on the OOV test set remains high
since each utterance has at least one word that is not in the ASR system's vocabulary.
5.5.3
PMM Approach
In this section, we combine our PMM pronunciation acquisition framework with AM training
and show improvements over the baseline approach. Using our generic AM, we first run the
PMM for two EM iterations on our training set to generate pronunications for the 9.5K words
in our vocabulary. We refer to this lexicon as pmmlemn2. We then use this new lexicon and
the generic AM to run phone forced-alignment on the training set and retrain new acoustic
models (referred to as pinmlem2AM). As a final step we again run the PMM for several
iterations using the new retrained AM.
We report our results in Table 5.16. Each row reports on WER for both the in-vocabulary
and OOV test set using different AM models and different lexicons. Here pmmlem2AM are
the acoustic models trained by running phone forced-alignment using the genericAM and the
learned PMM lexicon from EM iteration 2. pmm2em3 is the lexicon learned by running the
PMM until EM iteration 3 using the pmmlem2AM Acoustic Models. As can be seen, using
this PMM and Acoustic Model Retraining framework can achieve a significant reduction in
WER on both the OOV and the IV test sets.
genericAM + expertLexicon
expertAM + expertLexicon
genericAM + pmmleml
genericAM + pmmlem2
genericAM + pmmlem3
genericAM + pmmlem4
genericAM + pmmlem5
genericAM + pmmlem6
genericAM + pmmlem7
pmmlem2AM + pimnlem2
pmmlem2AM + pmm2eml
pmmlem2AM + pmm2em2
pmmlem2AM + pmm2em3
pmmlem2AM + pmin2em4
IV-test
24.7
11.1
25.6
24.1
23.5
23.5
23.3
23.1
23.0
10.6
10.6
10.4
10.3
10.4
OOV-test
98.1
90.0
26.6
24.6
24.6
24.6
24.9
24.9
25.0
14.6
13.6
13.8
14.0
14.2
Table 5.16: WER in percentage for E-book domain by iteratively training the acoustic models
and the lexicon. A gradual reduction in WER can be observed.
pmmlem2AM
pmmlem4AM +
pmmlem3AM +
pmmlem3AM +
pmmlem3AM +
+ pmm2em3
pmmlem4
pmm2eml
pmn2em2
pmm2em3
IV-test
10.3
10.8
11.0
10.7
10.7
OOV-test
14.0
14.6
13.6
13.9
14.0
Table 5.17: WER in percentage for E-book domain using pmmlem3. An increase in WER
is observed possibly due to overfitting
Overfitting Concerns
One important issue to address when considering PMM-AM co-training is overfitting. To
illustrate this point, we also tried using pmm1lem4 to retrain the AMs and observed degradation in performance. The results can be seen in Table 5.17. These initial results indicate
that further research into this approach is needed.
5.6
Summary
In this Chapter, we generalized our PMM approach to training and testing on continuous
speech. We tested and showed WER improvements on both a weather query corpus and
an academic Lecture corpus. After quantifying some of the differences between baseforms
learned from continuous speech and expert pronunciations, we used our PMM approach to
analyze baseforms learned from fast versus slow speech, as well as, non-native speakers. As
a final experiment, we introduced the notion of iteratively co-training the acoutic model and
the lexicon using the PMM. After testing on a corpus of book titles collected from AMT, we
showed that further improvements can be achieved through this method.
70
Chapter 6
Summary and Future Work
This work has introduced and compared several approaches to generating pronunciations by
combining graphone techniques with acoustic examples. Furthermore, we have shown that
even in the presence of significant noise, a pronunciation mixture model can reliably generate
improved baseform pronunciations over those generated by experts given both isolated-word
and continuous speech. Regardless, we believe that wrapping the lexicon into a statistical
framework is a constructive step, which presents exciting new avenues of exploration. We
have also shown that the PMM can be trained on the same data as the AM and LM, and
hence requires no additional training data to build. These properties make pronunciation
training a cheap and effective additional step to building an ASR system. We hope that this
thesis, encourages the training of a matched lexicon when training AMs.
Since our findings seem to extend to multiple corpora, the possibility of learning betterthan-expert baseforms in arbitrary domains opens up many possibilities for future work. For
example, when faced with an out-of-vocabulary word with a known spelling, any system
could programmatically post a task to AMT and collect example utterances to generate a
high quality entry in the lexicon.
There are two clear directions in which we hope to further this work. The first is to
explore acoustic model and lexicon co-training in an iterative fashion, effectively taking a
maximum-likelihood step along a set of coordinates in the probability space represented by the
recognizer. The second is to move beyond maximnum-likelihood, and explore discriminative
approaches to pronunciation learning on continuous speech. While we have only talked about
discriminatively training lexicon weights for isolated-word, we hope to generalize this work
for continuous speech by developing a frarnework for simultaneously discriminatively training
LM weights, lexicon weights, as well as, word boundary and phone n-gram weights.
Long term, the ultimate goal of this research might be to infer a phoneme set and learn
pronunciations from orthographically transcribed data only. If it were feasible to simultaneously train the lexicon and the acoustic model, large vocabulary speech recognizers could be
built for many different languages with little to no expert input, if given enough orthographically transcribed data.
Appendix A
Weather Domain Lexicon
The table below lists the 1.8K words present in the Weather Query corpus vocabulary, along
with their top PMM pronunciations, associated mixture weights and expert baseforms. We
underline the words whose top PMM pronunciation does not match its expert counterpart.
Word
PMM Baseforms
Weight
Expert Baseform
Word
PMM Baseforms
Weight
Rxpert
<oh>
<um>
aalten
ow
ahip m
aa 1t iy n
ao 1 tq en
1
1
0.58
0.27
ow
ahip m
aa 1 tq en
<uh>
a
ahip
ax
ey
ax b ae b aa
1
0.64
0.34
0.20
ah.fp
(ax
ey)
aa b
ae b
ae b
ax b
ae b
ey b
y ax
0.19
0.18
0.16
0.15
0.90
0.86
0.12
aberdeen
abilene
about
abu
accent
accumulated
ao
ae
ae
ax
aa
ae
ax
tf
Itf ax n
b er d iy n
b ax I iy n
b aw td
b uw
kd s ax n td
k y uw m y ax I ey
ax
dd
0.10
0.99
0.97
0.99
0.98
0.99
0.98
ababa
ae
ae
ax
aa
ae
ax
tf
b er d iy n
b ax I iy n
b aw td
b uw
kd s eh n td
k y uw m y ax
ax
dd
abidjan
able
ey
I
ax b aa
ae b ax
ax b ax
aa b ax
iy jh aa n
ax 1
b ax 1
Baseform
(aa
ae) b ( aa
b ( aa I ax)
aa b iy (jh
ey b axl
ax)
I zh ) aa n
Word
accumulations
activity
addis
address
advisories
afghanistan
after
again
ahead
air
airline
airplane
akron
alamos
albania
alberta
alert
alexandria
algiers
all
along
already
also
altoona
altus
[
PMM Baseforms
ax k y uw rs y ax 1ey
sh ax n s
ax k y uw A y ax 1 ey
sh ax n z
ae kd t ih v ax tf iy
aa df iy s
aa df ax s
ae df ax s
Weight
0.89
ae d iy z
0.13
ae df iy z
ae dr r eh s
ae dd v ay z ax r iy z
0.09
0.99
0.93
ae f g ae n ax s t ae n
ae f t er
ax g eh n
ax hh eh dd
eh dd
eh r
eh r 1ay a
eh r p 1 ey n
ae kd r ax p
ae 1ax m ow s
aa 1ax m ow s
ae 1 b ey n iy y ax
ae 1b ey n iy ax
aa 1b ey n y ax
ae 1 b er tf 4x
ax I er td
ae 1ax gd a ae n dr r iy
ax
ae 1 ax kd s ae n dr r iy
ax
ae I jh y ib r z
ae 1jh y er z
aa 1jh y er z
ao I
ax 1ao ng
aa r eh df iy
ao 1s ow
ao 1s ax
ae 1 t uw n 4x
ae 1 tf ax s
0.85
0.97
0.95
0.74
0.23
0.99
0.99
0.92
0.98
0.65
0.17
0.47
0.39
0.09
0.99
0.99
0.81
Expert Baseform
ax k y uw m y ax
sh ax n z
1ey
0.10
0.99
0.37
0.20
0.16
Weight I Expert Baseform
ax b ah v
0.99
acapulco
aa k ax p ax I k ow
0.31
accumulate
aa k ax p ow k ow
ae k ax p ax 1k ow
ow k ax p ow k ow
ax k y uw m y ax 1 ey
td
ax k y uw m y ax I ey
sh ax n
0.29
0.11
0.11
0.99
active
ae kd t iy v
ae kd t ax v
actually
ae kd ch uw I iy
0.52
0.39
0.47
accumulation
(ae ax) dr r eh s
ae dd v ay z ax r iy z
ae f g ae n ax s t ae n
eh r
eh r I ay n
eh r p I ey n
ae kd r ax n
ae 1ax m ow ( z
airlines
airport
g ow
n z w er th
r iy z
r iy z
r iy z
r I ay n z
r p ao r td
alabama
ae I ax b ae m ax
0.89
ae 1 ax b ae m aa
ax 1 ae s k ax
ao l b ax n iy
ae I b ax k er k iy
ax 1 er td s
r ax 1er td s
ae 1jh ih r iy ax
ae 1z jh ih r iy ax
ao I jh ax r iy ax
ae g aa ng k w ax n
0.09
0.92
0.94
0.93
0.73
0.17
0.78
0.10
0.10
0.90
advisory
africa
afternoon
ago
ainsworth
aires
ae 1b ey n iy ax
ae 1b er tf ax
ax 1er td
ae 1ax ( gd z kd s ) ae
n dr r iy ax
ae 1jh ih r z
alaska
albany
albuquerque
alerts
ao I
ax 1 ao ng
ao l r eh df iy
ao 1s ow
algeria
ae 1 t uw n ax
ao 1tf ax s
kd ch ax I iy
kd ch uw ax 1 iy
d ih sh ax n ax I
df ax I ey dd
df ax l ay dd
dd I ey dd
dd I ay dd
dd v ay z ax r iy
f r ax k ax
ax
ey
eh
ae
ay
eh
eh
additional
adelaide
s)
ae
ae
ax
ae
ae
ae
ae
ae
ae
algonquin
ae f t er n uw n
( aa I ae ) k ax p ( ow
uh 1) k ow
ax k y uw m y ax 1 ey
td
0.94
0.36
0.13
0.99
0.35
0.15
0.15
0.15
0.87
0.98
0.97
0.97
0.99
0.25
0.14
0.11
0.99
0.99
ae f t er
ax g eh n
ax hh eh dd
J PMM Baseforms
ax b ah v
ae kd t ih v ax tf iy
(aa I ae) df ax s
0.10
0.37
0.30
0.19
0.92
0.98
0.98
0.73
0.15
0.91
0.35
Word
above
ax k y uw n y ax 1 ey
sh ax n
ae kd t ax v
ae kd ch ( uw ax 1 uw
I I ax 1 ) iy
ax d ih sh ax n ax 1
ae df ax 1 ey dd
ae dd v ay z ax r iy
ae f r ax k ax
ae f t er n uw n
ax g ow
ey n z w er th
( eh I ae ) r ( iy
Ieh ) z
eh r 1 ay n z
eh r p ao r td
ae 1 ax b ae m ax
ax 1 ae s k ax
ao 1 b ax n iy
ae 1 b ax k er k iy
ax 1 er td s
ae 1jh ih r iy ax
ae
I ax
) I g ( aa
) ng k w ax n
amarillo
american
amman
amsterdam
anaheim
anchorage
anderson
angola
ann
anniston
another
antarctica
aa
ae
ae
ax
1 td
1 t uh z
m ax r ih I ow
m eh r ax k ax n
0.33
0.18
0.82
0.99
ax m aa n
ae m aa n
ae m s t- er d ae m
ae n ax hR ay m
ae ng k r ax jh
ae n d er s ax n
ih n d er s ax n
ae ng g ow ax 1 aa
0.80
0.11
0.83
0.99
0.88
0.51
0.48
0.52
ae ng g ow I ao
0.46
0.99
0.90
0.99
0.60
ae n
ae n ax s t ax n
ax n ah dh er
ae nt aa r
tf ax
k ax
ae nt aa r kd t ax k ax
0.38
ae
ax
ax
ax
m ax r ih 1 ow
m eh r ( ax I ih) k
n
m aa n
alright
alto
altos
ae m s t- er d ae mj
ae n ax hh ay m
ae ng k ( er I r ) ax
ae n d er s ax n
allentown
alot
jh
ae ng g ow 1 ax
ao
ax
aa
aa
ae
0.74
0.11
0.87
0.49
0.39
0.82
0.13
0.99
1 r ay td
I r ay td
1 tf ow
I tf ow z
1tf ow z
s
ae m
america
m
ax m eh r ax k ax
amherst
n ax s t ax n
n ah dh er
nt aa r ( tf I kd t
k ax
0.09
0.99
0.85
0.11
am
ae n
ae
ax
ae
ax
ax 1 g aa ng k w ax n
ae I ax n t aw n
ax 1 aa td
I aa td
amount
an
analog
ae
ae
ax
ax
m hh er s td
n er s td
m aw n td
n
ae n
0.65
0.29
0.92
0.85
0.12
ae n ax 1 aa gd
0.84
ah
ae 1 ax n t aw n
ax 1 aa td
ao
Ir
ay td
(ae I aa) 1 t ow
( ae aa) 1 t ow z
ae m
ax m eh r ( ax I ih ) k
ax
ae m ( hh er I er ) s td
ax m aw n td
( ax | ae ) n
ae n ax 1aa gd
Word
antonio
anything
anywhere
appreciate
april
arabia
arctic
area
argentina
arkansas
around
arthur
as
asia
asked
aspen
astoria
at
atlanta
au
august
austin
austria
average
aware
back
baghdad
bahrain
bakersfield
baltimore,
bangkok
bangor
barbados
barcelona
baseball
PMM Baseforms
ae n t ow n iy ow
ax n t ow n iy ow
eh n iy th ih ng
eh n iy w eh r
ax p r iy sh iy ey td
p r iy sh iy ey td
ax p r ax sh iy ey td
ey pd r ax I
ax r ey b iy ax
aa r kd t ax kd.
aa r tf ax kd
aa r tf ih kd
Weight
eh r iy ax
aa r jh ax n t iy n ax
aa r k ax n s ao
aa r k ax n s ax
aa r k ax n s aa
ax r aw n dd
aa r th er
ax z
ae z
ey zh ax
ae s td
0.99
ae s kd td
ae s p ax n
ax s t ao r iy ax
ax s t- ao r iy ax
ae td
ae td 1ae nt ax
ax td 1 ae nt ax
ow
ao g ax s td
ao g ah s td
aa g ax s td
0.19
0.97
ao s t ax n
aa s t ax n
0.83
ao s tr r iy ax
aa s tr r iy ax
ae v r ax jh
w eh r
ax w eh r
0.79
b ae kd
b ae gd d ae dd
b aa r ey n
b aa hh ax r ey n
b ey k er z f iy 1dd
b ao I tf ax m ao r
b ae ng k aa kd
b ae ng g ao r
b ae ng g er
b aa r b ey d ow s
b aa r b iy df ow z
b aa r s ax 1ow n ax
b ey s b ao 1
b ey z ax kd b ao 1
m b ey s b ao 1
0.99
0.92
0.78
0.19
0.92
0.99
0.45
Expert Baseform
ae n t ow n iy ow
and
eh n iy th ih ng
eh n iy w eh r
ax p r iy sh iy ey td
0.34
0.20
0.98
0.99
0.76
0.09
0.96
0.99
0.75
0.22
0.92
0.73
0.55
annapolis
annual
eh r iy ax
aa r jh ax n t iy n ax
aa r k ax n s ao
answer
antigua
ax r aw n dd
aa r th er
( ax I ae ) z
ey zh ax
ae s kd td
ae s p ax n
ae s t ao r iy ax
0.44
0.95
0.79
ae td
( ax I ae ) td I ae nt ax
0.20
0.79
0.62
ow I ao
ao g ( ah I ax ) s td
0.17
0.15
ao s t ax n
0.10
0.20
0.90
0.68
0.94
0.75
0.12
0.83
0.10
0.93
0.62
0.21
0.12
any
anytime
appleton
approximate
arab
arbor
are
areas
arizona
arlington
arrowhead
aruba
asheville
ask
asking
assistance
asuncion
ao s tr r iy ax
ae v r ax ( jh | zh
ax w eh r
0.31
0.74
0.16
0.88
0.98
angeles
ankara
ey pd r ax 1
ax r ey b iy ax
aa r ( tf I kd t ) ax kd
0.10
0.10
0.97
0.74
0.13
Word
b ae kd
b ae gd d ae dd
b aa r ey n
athens
atlantic
auckland
augusta
b
b
b
b
ey
ao
ae
ae
k er z f iy 1dd
I tf ax m ao r
ng k aa kd
ng g ( ao r I er)
b aa r b ey d ow s
b aa r s ax 1ow n ax
b ey s b ao 1
australia
available
aviv
b
bad
bahamas
baja
bali
PMM Baseforms
ae n ax 1 ao gd
ae n dd
ax n dd
ae n
ae n jh ax 1 ax s
ae ng k ax r ax
aa ng k ax r ow
aa n k aa r ax
ae n k ae r ax
ae ng kd y ax r ax
ax n ae p ax I ax s
ae n y uw ax 1
Weight
0.15
0.61
ae n y uh 1
ae n y uw w ax 1
ae n y ax 1
ae n s er
ae n t iy gd w ax
ae nt iy g ax
eh n iy
eh n iy t ay m
ae p ax 1tf ax n
ae pd 1tf ax n
ax p r aa kd s ax m ax
td
ae r ax bd
aa r b er
aa r
eh r iy ax z
ae r ax z ow n ax
aa r 1 ih ng t ax n
ae r ow hh eh dd
ax r uw b ax
ae sh v ih 1
ae s kd
ae s k ih ng
ax s ih s t ax n s
ax s ah n s iy ow n
ax s ah n ch ow n
ae s ax n zh ax n
ax s ah n sh ax n
ae th ax n z
ae td I ae nt ax kd
0.13
ax td 1 ae nt ax kd
ao kd I ax n dd
aa kd I ax n dd
ax g ah s t ax
aa g ah s t ax
ao s tr r ey 1y ax
ax v ey 1 ax b ax 1
ax v ey 1b ax 1
ax v iy v
b iy
b an dd
b ax hh aa m ax s
b ax hh aa m ax z
b aa hh aa,
b aa 1 iy
0.47
0.44
0.41
0.73
0.14
0.78
0.65
0.34
0.95
0.99
0.99
0.72
0.26
0.99
0.98
Expert Baseform
( ax | ae ) n dd
0.18
0.10
0.97
0.45
0.18
0.12
0.11
0.09
0.97
0.60
ae n jh ax 1 ax,,
ae ng k ax r ax
ax n ae p ax 1ax s
ae n ( y uh 1 y uw ax
1)
0.09
0.09
0.71
0.77
0.15
0.97
0.86
0.75
0.13
0.99
0.99
0.99
0.97
0.86
0.98
0.99
0.90
0.93
0.92
0.96
0.97
0.96
0.27
0.23
0.23
0.22
0.93
0.49
ae n s er
ae n t iy gd w ax
eh n iy
eh n iy t ay m
ae p ax 1tf ax n
ax p r aa kd s ax m
ey I ax ) td
ae r ax bd
aa r b er
aa r
eh r iy ax z
ae r ax z ow n ax
aa r 1 ih ng t ax n
ae r ow hh eh dd
ax r uw b ax
ae sh v ih 1
ae s kd
ae s k ih ng
ax s ih s t ax n s
ax s ah n s iy ow n
ae th ax n z
( ax | ae ) td 1 ae nt ax
kd
ao kd I ax n dd
( ao I ax ) g ah s t ax
ao s tr r ey 1y ax
ax v ey 1ax b ax 1
ax v iy v
b iy
b ae dd
b ax hh aa m ax z
b aa hh aa
b aa 1 iy
Word
baton
be
beaumont
bedford
'
PMM Baseforms
b ae tq en
Weight
0.92
b
b
b
b
0.99
0.81
0.11
0.99
iy
ow m aa n td
aw m ah td
eh dd f er dd
'
Expert Baseform
( b ae tq en | b ax t aa
n)
b iy
b ow m aa n td
b eh dd f er dd
'"
Word
bangalore
bangladesh
bar
barbara
'
PMM Baseforms
b ae ng g ax 1 ao r
Weight I Expert Baseform
0.75
aa ng g ax I ao r
b
b
b
b
ay ih ng g ax 1 ao r
ae ng g I ax d eh sh
aa r
aa r b r ax
0.23
b aa r b ax r ax
b ae r ax m eh tr r ax
kd
b aa z ax 1
b ey z ax 1
b ey
b iy ch
b ax k ax m
b ih n
b ax g ih n
b ey zh ih ng
b ey jh ih ng
b eh I ax r uw s
0.38
0.97
0.23
0.93
0.79
0.99
0.56
b iy f ao r
b ax f ao r
0.79
0.20
b ( ax I iy ) f ao r
b
b
b
b
b
b
b
b
b
b
ax
ey
ey
eh
eh
eh
eh
eh
eh
eh
g ih n ih ng
r uw td
uw td
r uw td
I f ae a td
1 f ae a
I f r ax s td
1 gd r ey dd
I g r ey dd
Iv y uw
0.99
0.44
0.27
0.17
0.70
0.14
0.13
0.49
0.29
0.97
b ax g ih n ih ng
b eh r uw td
b
b
b
b
b
b
eh I m aa
eh n dd
er 1ih n
er I ae n
er I ax n
er n
n td
0.98
0.93
0.63
0.19
0.09
0.96
b eh 1 m aa n td
b eh n dd
b er 1ih n
belgium
belize
b er n
bellingham
b iy eh I ax r uw s
b eh 1jh ax m
b ax 1 iy z
b eh 1iy z
bd 1 iy z
b eh I ih ng hh ae m
b
b
b
b
b
ax s ay dd z
iy s ay dd z
ax th eh a d ax
ax t ax z d ax
ax s th eh a d ax
0.67
0.30
0.49
0.26
0.22
b ( ax | iy ) s ay dd z
below
berkeley
bermuda
bernardino
b eh 1ih ng hh ax m
b ax 1 ow
b er kd 1 iy
b er m y uw df ax
b er n ax d iy n ow
0.18
0.87
betwe en
billing
birmir ngham
b ax t w iy n
b ih I ih ng z
b er m ih ng 1h ae m
0.95
0.99
0.92
best
better
big
b eh s td
b eh tf er
b ih gd
0.99
0.98
0.44
blah
bloomington
boca
b 1 aa
b 1uw m ih
b ow k ax
0.98
0.95
0.99
b ax t w iy n
b ih 1ih ng z
b er m ih ng ( hh ae
hh ax I ax) m
b 1aa
b 1uw m ih ng t ax n
b ow k ax
0.17
binghamton
b ey gd
b ax gd
b ih ng hh ax m t ax n
boise
b oy s iy
b oy z iy
b aa m b ey
b ao r d ow
b aa z n iy ax
b aa z n iy y ax
b aa z n y ax
b ow th
ow th
b ow 1 ih ng
0.69
0.24
0.94
0.88
0.40
0.36
0.13
0.82
0.16
0.97
b oy ( z | s ) iy
0.44
b ow I ih ng
bolivia
b ih ng ax m t ax n
b ih z m aa r kd
b I ih z er dd
b 1ah f
b l ax f
b l ao f
b ow g ax t aa
b ow gd ax t aa
b ow g ax tf aa
b ax I ih v iy ax
b ow z m ax n
0.99
0.33
b ow z m ax n
b r ax z ih I ( y ax | iy
ax )
bonn
b ow 1 ih v iy ax
b aa n
0.24
0.99
b ao r n iy ow
b ao s t ax n
b aa s t ax n
b ow 1df er
b oy
b oy s ax td s
b oy sh ax n s
b r ae dd f er dd
b r tf ax z 1 aa v ax
0.99
b r aa tf ax z 1 aa v
0.29
before
beginning
beirut
belfast
belgrade
bellevue
barometric
basel
b eh 1g r ( aa Iey) dd
bay
beach
become
been
begin
beijing
b eh 1 v y uw
belarus
b eh 1 f ae s td
0.58
0.11
0.96
0.97
0.97
0.96
0.99
0.57
b ae ng g 1ax d eh sh
b aa r
b aa r b (er Iax r r)
( ah I ax)
b ae r ax m eh tr r ax
kd
b ( aa [ ae) z ax 1
b ey
b iy ch
b ( ax I iy) k ah m
b ih n
b ax g ih n
b ey (zh I jh) ih ng
0.39
0.75
b ( eh 1 | ey 1 ) ax r uw
S
belmont
bend
berlin
bern
beside S'
bethessda
bombay
bordeaux
bosnia
both
bowling
bozeman
brasilia
brazil
breezy
bring
bristol
british
b r ax z ih
b
b
b
b
b
b
b
b
b
r
r
r
r
r
r
r
r
r
ng t
Iy
ax n
ax
ax z ax Iiy ax
ax z ih 1iy ax
ax s iy 1iy ax
ax z ih 1
iy z iy
ih ng
ax s t- ax 1
ih s t ax 1
ih tf ax sh
b r ax tf ax sh
0.25
0.20
0.19
0.98
0.98
0.99
0.50
0.32
0.82
0.13
b ax th eh z d ax
b aa m b ey
b ao r d ow
b aa z n iy ax
I
bismarck
blizzard
bluff
bogota
b ow th
borneo
boston
b
b
b
b
r ax z ih 1
r iy z iy
r ih ng
r ih s t ax 1
b r ih tf ax sh
boulder
boy
bradford
bratislava
0.79
0.10
b eh Ijh ax m
b eh 1 iy z
0.09
0.77
0.95
0.87
0.84
0.10
0.50
0.99
0.99
0.43
0.31
0.11
0.50
0.21
0.12
0.62
0.71
0.27
0.98
0.74
0.11
0.09
0.96
0.68
b eh 1ih ng ( hh ae I hh
ax ax) m
b ax 1ow
b er kd 1 iy
b er m y uw df ax
b er n ( aa r I er I ax
d iy n ow
b eh s td
b eh tf er
b ih gd
b ih ng (hh ax I ax) m
t ax n
b ih z m aa r kd
b 1 ih z er dd
b 1 ah f
b ow g ax t aa
b ( ow
ax
1
ax 1 ) ih v iy
b aa n
b ao r n iy ow
b ( aa I ao ) s t ax n
b ow 1 df er
b oy
b r ae dd f er dd
b r ( aa I ae ) tf ax ( s
I z) 1 aa v ax
Word
brooklyn
brunswick
bucharest
buenos
bulgaria
burlington
but
buzzards
bye
cairo
caledonia
cali
call
calling
cambridge
can
canberra
cannes
canyon
capital
caracas
caribbean
carlo
carolina
casablanca
causes
cayman
celsius
centigrade
central
chance
change
charleston
'
PMM Baseforms
b r uh kd I ax n
b r ah n z w ih kd
Weight
b r ah n z w ax kd
b uw k ax r eh s td
b w eh n ax s
b w ey n ow s
b w ey n ax s
b ax 1 g eh r iy ax
b er 1 ih ng t ax n
b ah td
b ax td
b ah z er dd z
b ao z er dd z
b ay
k ay r ow
k ae 1 ax d ow n iy ax
k ae 1ax d ow n y ax
k aa 1 iy
k ae 1iy
k ao
k ao 1 ih ng
k ey m bd r ih jh
k ey m b r ih jh
k ae n
k ax n
k ae n b eh r ax
k aa n
0.28
0.92
0.59
0.19
0.11
0.99
0.96
0.81
0.10
0.51
0.48
0.98
0.90
0.71
0.28
0.78
0.21
0.99
0.97
0.65
0.27
0.64
0.34
0.89
0.59
k ae
k aa
k ae
k ey
k ae
k ax
k ae
k ax
nz
nz
n y ax n
n y ax n
p ax tf ax 1
r aa k ax s
r ae k ax s
r ih b iy ax n
0.19
0.13
0.85
0.10
0.99
0.75
0.11
0.55
k ae r ax b iy ax n
k ae r ax b iy ae n
k ax r ih b iy ae n
k aa r 1ow
k ae r ax 1 ay n ax
k ae s ax b 1 ae ng k ax
k ae s ax b I aa n k ax
k ae s ax b 1aa n k ae
k aa s ax bd 1 aa ng k
ax
k ae s ax bd I ae ng k
ax
k aa z ax z
k ay z ax z
k ey m ax n
k ey m eh n
s eh 1s iy ax s
s eh nt ax g r ey dd
s eh n tr r ax 1
ch ae n s
ch ey n jh
ch aa r I s t- ax n
ch aa r 1s t ax n
0.19
0.13
0.10
0.99
0.99
0.27
0.19
0.19
0.12
0.99
0.71
[
'
Expert Baseform
b r uh kd I ax n
b r ah n z w ( ax I ih)
kd
'
Word
breckenridge
bridgeport
PMM Baseforms
b r eh k ax n r ib jh
b r ih jh p ao r td
Weight
b r ax z b ax n
b r ih z b ey n
b r ih z b ax n
b r ih tq en
b r uh kd 1 ay n
b uh kd 1 ay n
bd r uh kd 1 ay n
b r ax k 1 ay n
b r aw n z v ih 1
b r ah s ax 1z
b uw df ax p eh s td
b ah f ax 1ow
b er b ae ng kd
b er m aa
b er m ah
b ao r m aa
b y uw td
b ay
s iy
k ae 1 k ah tf ax
k ae 1 k aa tf ax
k ae 1 g ax r iy
k ae 1 gd r iy
k as 1 ax f ao r n y ax
k ao I dd
0.44
0.37
0.09
0.99
0.52
0.16
0.11
0.09
0.87
0.90
0.93
0.96
0.99
0.53
0.33
0.13
0.99
0.86
0.98
0.53
0.38
0.73
0.16
0.99
0.90
k
k
k
k
k
k
k
k
aa
ae
ae
ae
ae
ae
ae
ey
0.09
0.99
0.99
0.99
0.66
0.30
0.92
0.99
carson
casper
cayenne
k
k
k
k
k
k
k
k
k
aa r
eh r
y eh r
ae r ax b uw
aa r m eh 1
aa r m ax 1
aa r s ax n
ae s p er
ay ae n tf ax dd
cedar
brisbane,
b uw k ax r eh s td
b w ey n ( ax I ow ) s
britain
brookline,
b ax 1g eh r iy ax
b er 1 ih ng t ax n
b ah td
b ah z er dd z
b ay
k ay r ow
k ae 1 ax d ow n iy ax
brownsville
brussels
budapest
buffalo
burbank
burma
k aa 1 iy
k ao I
k ao 1 ih ng
k ey m b r ih jh
butte
by
c
calcutta
k ( ae I ax ) n
calgary
k as n b (eh r Ier ) ax
k ( ae n ae n z | aa n
california
called
k ae n y ax n
k ae p ax tf ax 1
k ax r aa k ax s
k ( as r ax | ax r ih ) b
iy ax n
cambodia
camden
canada
cancun
canton
cape
car
care
k aa r 1 ow
k ae r ax 1ay n ax
k ae s ax b 1 ae ng k ax
0.11
0.90
0.09
0.81
0.11
0.98
0.99
0.86
0.98
0.99
0.65
0.32
"
caribou
carmel
k ao z ax z
k ey m ax n
s eh 1s iy ax s
s eh nt ax g r ey dd
s eh n tr r ax 1
ch ae n s
ch ey n jh
ch aa r 1 ( s z ) t ax n
center
centigrades
champaign
chances
chapel
charlotte
charlottetown
'
1 dd
m b ow df iy ax
m d ax n
n ax df ax
n k uw n
n k ow n
n tq en
pd
0.97
0.97
'
Expert Baseform
h r eh k ax n r ih jh
b r ih jh p ao r td
b r ih z b ( ax I ey ) n
b r ih tq en
b r uh kd I ay n
b
b
b
b
b
b
r aw n z v ih 1
r ah s ax 1 z
uw df ax p eh s td
ah f ax l ow
er b ae ng kd
er m ax
b y uw td
b ay
s iy
k ae 1k ah tf ax
k ae 1 g ax r iy
k ae 1 ax f ao r n y ax
k ao 1 dd
k
k
k
k
ae
ae
ae
ae
m b ow df iy ax
m d ax n
n ax df ax
n k uw n
k ae n tq en
k ey pd
0.99
0.83
0.16
0.96
0.53
0.46
0.99
0.99
0.99
k aa r
k eh r
s iy df er
0.86
s iy df er
s iy df er uw
s eh nt er
s eh n t iy gd r ey dd z
s eh nt ax g r ey dd z
s eh n t iy g r ey dd z
s eh nt iy g r ey dd z
sh ae m p ey n
ch ae n s ax z
ch ae p ax 1
sh aa r 1ax td
sh er 1 ax t aw n
0.10
0.99
0.39
0.34
0.11
0.09
0.96
0.99
0.98
0.99
0.54
k ae r ax b uw
k aa r m ( ax 1| eh 1)
k aa r s ax n
k ae s p er
k ay ( ae I eh ) n
s eh nt er
s eh nt ax g r ey dd z
sh ae m p ey n
ch ae n s ax z
ch ae p ax 1
sh aa r 1ax td
sh aa r 1 ax t aw n
Word
charlottesville
chatham
PMM Baseforms
sh aa r 1ax td s v ih 1
ch ae tf ax m
k ae tf ax m
ch eh kd
sh ay ae n
s ay eh n
Weight
0.93
0.86
0.09
0.97
0.42
0.14
Expert Baseform
sh aa r I ax td s v ih 1
ch ae tf ax m
0.99
0.97
0.72
0.17
sh ax k aa g ow
ch ih I
ch ay n iy z
christi
church
cities
claire
clear
cleveland
sh ax k aa g ow
ch ih 1
ch ay n iy z
ch ay n iy z y ax tf ax
v
k r ih s t iy
ch er ch
s ih tf iy z
k 1 eh r
k 1ih r
k 1 iy v I ax n dd
0.99
0.99
0.99
0.99
0.94
0.99
k r ih s t iy
ch er ch
s ih tf iy z
k 1 eh r
k 1 ih r
k 1 iy v 1ax n dd
closest
cloud
cloudy
coastal
cocoa
cold
coldest
college
cologne
colombo
colorado
k 1 ow s ax s td
k l aw dd
k l aw df iy
k ow s t ax 1
k ow k ow
k ow I dd
k ow I df ax s td
k aa 1ax jh
k ax 1ow n
k ax 1 ah m b ow
k aa 1ax r aa df ow
0.99
0.84
0.89
0.99
0.99
0.98
k 1 ow s ax s td
k l aw dd
k I aw df iy
k ow s t ax 1
k ow k ow
k ow I dd
k ow 1 df ax s td
k aa 1 ax jh
k ax 1ow n
k ax I ah m b ow
k aa 1 ax r ( ae I aa) df
ow
0.32
0.92
0.98
0.95
0.86
0.83
0.16
0.98
0.92
cost
could
countries
county
k aa 1ax r ae df ow
k ax 1 ah m b ax s
k ah m ih ng
k ax m p y uw tf er
k ax n d ih sh ax n
k aa ng g ow
k ao ng g ow
k ax n eh tf ax k ax td
k ow p ax n hh ey g ax
n
k ao s td
k uh dd
k ah n tr r iy z
k aw nt iy
0.99
0.98
0.99
0.94
k ax n eh tf ax k ax td
k ow p ax n hh ey g ax
n
k ao s td
k ( uh I ax ) dd
k ah n tr r iy z
k aw nt iy
course
cozumel
k ao r s
k aa z ax m eb 1
0.94
0.43
k ao r s
k ( aa I ow) z ax m eh
k ow z ax n eh 1
k aa z ax n ax 1
k r iy kd
k r ow ey sh ax
k r ax w ey sh ax
k r aa s
k r ao s
k ao s,
k r uw z
k uw p er t iy n ow
k er ax n td 1 iy
s ay pd r ax s
d iy
d ax k aa r
0.42
0.13
0.99
0.73
0.21
0.56
0.27
0.16
0.98
0.95
check
cheyenne
chicago
chill
chinese
columbus
coming
computer
condition
congo
connecticut
copenhagen
creek
croatia
crosse
cruz
cupertino
currently
cyprus
d
dakar
0.97
0.99
0.90
0.99
0.66
0.97
0.99
0.99
0.73
ch eh kd
sh ay eh n
Word
chile
sh
ch
ch
ch
ch
ch
china
christchurch
ch ih 1 ey
ch iy Iey
ch ay n ax
k r ay s td ch er ch
chattanooga
checking
chi
I ah m b ax s
m ih ng
m p y uw tf er
n d ih sh ax n
ng g ow
k
k
k
k
k
k
k
k
k
k
k
1 ih r w ao tf er
I ay m ax td
1ow z
1 aw dd z
ow s td
ow td
aa dd
ow I df er
ax 1 eh kd sh ax n
aa 1ax n z
ao I ax n z
0.16
0.99
0.99
0.83
0.95
0.99
0.92
0.99
0.99
0.89
0.09
k
k
k
k
k
k
k
k
k
ax
ax
ax
ah
ah
aa
ax
aa
ax
0.72
k ax 1 ah m b iy ax
columbia
k
k
k
k
k
k
k
k
k
come
k ah m
0.87
complete
concord
k ax m
k ax m p 1 iy td
k aa ng k er dd
0.12
0.99
0.79
conditions
k aa ng k ao r dd
k ax n d ih sh ax n z
0.16
0.99
k ax n eh kd td
k ax I
k uw ax 1
k uw 1
k ao r p ax s
k ow s t ax
k aw n td
k ah n tr r iy
k ah p ax 1
k ah v r ax jh
k r iy ey tf ax dd
k r iy td
k r oy
k r uw s ax z
0.99
0.56
0.17
0.09
0.94
0.96
0.97
0.99
0.99
0.99
0.89
0.97
0.98
0.38
city
clara
clearwater
0.58
k r ih s m ax s
s ih n s ax n ae tf iy
s ih tf iy
k l ae r ax
k I ih r w ( aa ao uh
) tf er
climate
clothes
clouds
coast
coat
cod
colder
collection
collins
color
( k r iy kd | k r ih kd)
k r ow ey sh ax
k r uw z
k uw p er t iy n ow
k er ax n td 1 iy
s ay pd r ax s
d iy
d aa k aa r
Weight I Expert Baseform
0.42
ch ae tf ax n uw g ax
0.99
ch eh k ih ng
0.95
k ay
0.46
0.36
0.56
ch ( iy 1ey I ih I ey I ih
I iy )
0.24
0.13
ch ay n ax
0.96
k r ay s ch er ch
0.94
0.95
0.84
0.10
0.99
0.93
connect
cool
k r ao s
as r 1 ax t aw n
ae tf ax n uw g ax
eh k ih ng
iy
ay
ih I iy
k r ih s m ax s
s ax n s ax n ae tf iy
s ih n s ax n ae tf iy
s ih tf iy
k I ae r ax
k 1 ih r w aa tf er
christmas
cincinnati
colombia
k ax
k ah
k ax
k ax
k aa
[PMM Baseforms
corpus
costa
count
country
couple
coverage
created
crete
croix
cruces
1 ah m
1 ah m
1 ah m
I er
1 er z
l er
1 er
1 r td
1ah m
b iy ax
bd iy ax
bd y ax
b iy ax
1 ay m ax td
1ow ( dh z z)
l aw dd z
ow s td
ow td
aa dd
ow 1df er
ax 1 eh kd sh ax n
aa 1 ax n z
0.14
0.09
0.31
0.20
0.16
0.12
0.10
0.93
k ah 1er
k ax 1 ah m b iy ax
k ah m
k ax m p 1 iy td
k aa ng k ( er I ao r
dd
k ax n d ih sh ax n z
k ax n eh kd td
k uw I
k
k
k
k
k
k
k
k
k
k
ao r p ax s
ow s t ax
aw n td
ah n tr r iy
ah p ax 1
ah v ( er I r ) ax jh
r iy ey tf ax dd
r iy td
r oy
r uw s iy z
Word
database
PMM Baseforms
d ae k ao r
d ae I ax s
d ae n b eh r iy
d ae m b er iy
d ae n b er iy
d aa r w eh n
d aa r w ax n
d ey tf ax b ey s
Weight
0.15
0.99
0.49
0.31
0.15
0.89
0.10
0.99
davenport
day
days
dayton
de
d ae v ax n p ao r td
d ey
d ey z
d ey tq en
d iy
0.99
0.96
0.96
0.99
0.47
0.44
0.88
0.10
0.40
0.40
0.11
0.88
0.10
0.83
0.98
0.99
0.70
0.24
0.50
0.38
0.09
0.96
0.99
0.99
0.89
0.09
0.96
dallas
danbury
darwin
[ Expert Baseform
d ae I ax s
d ae n b eh r iy
Word
cuba
current
cyclones
d aa r w ax n
d ( ey
Iae
tf ax b ey
czech
daily
[
PMM Baseforms
k r uw s ih z
k r uw s ax s
k y uw b ax
k er ax n td
s ay kd Iaa n z
s ay k 1 aa n z
ch eh kd
d ey 1iy
Weight
0.19
0.17
0.97
0.96
0.81
0.97
0.99
ch eh kd
d ey 1 iy
d ax k ow tf ax
d ax m a~es k ax s
d ae n v ih 1
d ey tf ax
d ae tf ax
d ey td
0.94
0.99
0.99
d
d
d
d
ax k ow tf ax
ax m ae s k ax s
ae n v ih 1
( ey Iae ) tf ax
0.25
0.99
0.99
0.99
0.99
0.97
0.98
0.65
0.31
d
d
d
d
d
d
d
ey td
ey v ax s
ey 1 ay td
ey t ay m
ey t ow n ax
ih r b ao r n
(ax iy) s eh m b er
0.84
d (ax
0.14
0.97
0.98
0.94
0.89
0.43
d
d
d
d
d
Expert Baseform
k y uw b ax
k er ax n td
s ay k 1 ow n z
0.18
s
d ey
death
d eh th
t eh th
deep
degrees
delaware
delmar
denver
describe
d iy p ax 1
aa d iy pd
d iy pd
d iy g r iy z
d ax g r iy z
d eh I ax w eh r
d eh 1m aa r
d eh n v er
d ax s k r ay bd dd
d ax s k- r ay bd dd
detailed
d iy t eh 1 dd
d iy t ey I dd
d iy t eh 1
dew
dhabi
diego
different
d
d
d
d
d
d
direction
distance
djibouti
dodge
doing
dominican
dorado
down
driving
dublin
dude
duluth
durham
uw
aa b iy
iy ey g ow
ih f r ax n td
ih f er ax
ax r eh kd sh ax n
jh ax b uw tf iy
d ih jh ax b uw tf iy
0.99
0.50
0.42
d aa jh
d uw ih ng
d ax m ih n ax k ax n
0.93
0.82
0.89
d ax m ih n ax k ax
d ax r aa df ow
d ax r ey d ow
d ax r ey df ow
d aw n
dr r ay v ih ng
d ah bd 1ax n
d uw dd
d uh dd
d ax 1 uw th
d ao r ax m
0.10
0.56
0.19
d ih s t ax n s
d ae v ax n p ao r td
d ey
d ey z
d ey tq en
d ( eh I ey iy
d eh th
d iy pd
d ( ax | iy) g r iy z
d eh 1ax w eh r
d eh 1m aa r
d eh n v er
d ( ax | iy) s k- r ay bd
d ( ax iy ) t ey 1dd
d
d
d
d
uw
aa b iy
iy ey g ow
ih f ( er I r ) ax n td
date
davis
daylight
daytime
daytona
dearborn
december
define
del
delhi
denmark
des
detail
detroit
d ey v ax s
d ey 1 ay td
d ey t ay m
d ey t ow n ax
d ih r b ao r n
d ax s eh m b er
d iy s eh m b er
d ax f ay n
d ax s f ay n
d eh 1
d eh 1 iy
d eh n m aa r kd
d ax
d iy t eh 1
d iy t ey 1
0.74
0.23
d iy tr r oy td
0.77
d ax tr r oy td
0.20
0.69
0.30
0.93
d ( ay r I ax r ) eh kd
sh ax n
d ih s t ax n s
jh ax b uw tf iy
did
difference
digital
directions
d ih f r ax n s
d ih jh ax tf ax 1
d ax r eh kd sh ax n z
0.99
0.85
0.99
d
d
d
k
district
do
d ih s tr r ax kd td
d
d uw
0.99
d ah z
d ax m ih ng g ow
d ah n
0.97
0.96
0.91
0.99
0.99
aajh
uw ih ng
( ax I ow ) m ih n ax
ax n
d ax r aa df ow
d aw n
dr r ay v ih ng
d ah bd Iax n
d uw dd
d ax 1 uw th
d ( er Iao r ) ax m
dewey
does
dorningo
done
dover
drive
dry
d ow v er
dr r ay v
dr r ay
t r ay
dubuque
dulles
durango
d
d
d
d
ax b y uw kd
ah I ax s
ow I ax s
ax r ae ng g ow
iy) f ay n
eh 1
eh 1 iy
eh n m aa r kd
(ax eh )
( ax iy) t ey 1
0.27
d iy t ae 1
d uw iy
d uw r iy
d ih dd
0.17
0.99
0.99
0.97
0.86
0.13
0.91
0.64
dakota
damascus
danville
data
0.71
d (ax
iy) tr r oy td
d uw iy
d ih dd
d ih f ( er r ) ax n s
d ih jh ax t ax 1
d ( ay r I ax r ) eh kd
sh ax n z
d ih s tr r ax kd td
d uw
0.26
0.85
0.09
0.99
0.85
0.14
0.39
d ah z
d ow m ih ng g ow
d ah n
d ow v er
dr r ay v
dr r ay
d ax b y uw kd
d ah 1 ax s
d ( ax r Iao r ) ae ng g
ow
dusseldorf
d er ax m
jh ax r ax m
d uw s ax I d ao r f
d ax 1 s ax 1 d ao r f
0.13
0.10
0.88
0.11
ax r ey n g ow
y ao r ae ng g ow
d uw s ax 1 d ao r f
during
ao r ih ng
er ih ng
0.34
0.10
0.51
0.27
d ( er Iao r ) ih ng
Word
earlier
early
earthquake
east
eat
edinburgh
edward
egypt
eighteenth
eighty
el
eleventh
elmira
emergency
end
PMM Baseformis
er I iy er
eh r 1iy er
er 1iy
er th k w ey kd
iy s td
iy td
eh dd en b er ow
eh df ax n b er ow
eh dd en b er gd
eh dd w er dd
ey dd w er dd
iy jh ax pd td
ey t iy n th
ey tf iy
eh I
ax 1eh v ax n th
iy 1eh v ax n th
eh 1 m ay r ax
ax iy m er jh ax n s iy
eh n dd
Weight I Expert Baseform
0.72
er I iy er
0.21
er 1 iy
0.99
0.89
er th k w ey kd
iy s td
0.96
iy td
0.99
eh df ax n b er ( ax
0.52
ow I gd )
0.26
0.13
0.76
eh dd w er dd
0.11
iy jh ax pd td
0.97
ey t iy n th
0.99
ey tf iy
0.99
0.94
eh 1
0.72
ax 1eh v ax n th
0.21
eh I m ay r ax
0.88
ax m er jh ax n s iy
0.90
eh n dd
0.84
Word
e
earliest
earth
earthquakes
eastern
ecuador
edmonton
effect
eight
eighth
either
eleven
elkins
else
emirates
PMM Baseforms
jh er ih ng
iy
er 1iy ax s td
er th
er th k w ey kd s
iy s t er n
eh kd w ax d ao r
Weight I Expert Baseform
0.09
0.95
er 1 iy ax s td
0.99
0.96
er th
0.97
er th k w ey kd s
0.99
iy s t er n
eh kd w ax d ao r
0.61
eh kd w ax df ao r
eh dd m ax n t ax n
ax f eh kd td
ey td
ey td th
ey th
ay dh er
iy dh er
ax 1eh v ax n
1 eh v ax n
eh I k ax n z
eh 1 s
eh m ax r ax td s
0.28
ih ng g 1 ax n dd
iy n ih dd
iy n ax dd
ih n t ay r
eh n t ay r ax
0.98
0.70
0.96
0.91
0.99
0.72
0.21
0.47
0.34
0.88
0.11
0.99
0.99
0.99
eh dd m ax n t ax n
( ax | iy ) f eh kd td
ey td
ey ( td th Ith)
( iy
I ay )
dh er
ax I eh v ax n
eh 1 k ax n z
eh Is
eh m ax r ( ey I ax) td
s
english
enough
equator
essen
estonia
eugene
evansville
evening
ever
every
exact
except
exit
expanded
expected
expects
extended
f
factor
fair
fairfax
fallen
falls
far
ih n dd
ih ng g I ax sh
ax n ah f
iy k w ey tf er
ih z eh n
eh s t ow n ly ax
eh s t ow n iy y ax
y uw jh iy n
eh v ax n z v ih 1
iy v ax n ih jg
iy v n ih ng
eh v er
eh v r iy
aa f ax r iy
eh gd z ae kd td
0.37
ey gd z ae kd td
ax gd z ae kd td
0.34
0.27
eh
eh
eh
eh
0.90
0.33
kd
gd
kd
kd
s eh pd
z ih td
z ih td
s ax s
0.15
0.98
0.91
0.99
0.95
0.44
0.99
0.99
0.78
0.20
0.92
0.80
y uw jh iy n
eh v ax n z v ih 1
iy v ( ax n I n ) ih ng
enid
entire
ii nt ay er
eh ( gd
td
kd ) z ae kd
eh kd s eh pd td
eh ( gd z | kd s ) ax td
f eh r f ae kd s
0.80
I ax n
0.99
0.99
0.98
0.69
0.32
eh n t ay r
0.29
0.12
0.11
0.65
ih r iy
0.27
0.99
0.42
0.19
0.18
0.74
0.99
0.66
everett
everything
exactly
ax
eh
eh
eh
0.33
0.99
0.99
0.99
v eh n td s
v r ax td
v r iy th ih ng
gd z ae kd td 1 iy
eh s t ax m ey tf ax dd
iy th iy ow p iy ax
y ( er I ao r ) ax pd
0.11
iy v ax n
iy v eh n td s
iy v ax n
( ax | eh iy)vehntd I
eh
eh
eh
kd
eh
v ( ax r I r ) ax td
v r iy th ih ng
( gd z kd s ) ae
td I kd) 1iy
kd s k- y uw z
excuse
kd s k- y uw z
eh kd s k- y uw z
0.63
0.36
eh kd s p- eh kd t ax dd
expand
0.56
0.43
eh kd s p- ae n dd
eh kd s p- eh kd td s
eh kd s t- eh n d ax dd
expect
expecting
eh kd s p- ae n dd
ax kd s p- ae n dd
eh kd s p- eh kd td
eh kd s p- eh kd t ih ng
eh kd s p- 1ey n
ax kd s p- 1 ey n
eh kd s p- 1ey n dd
eh kd s tr r iy n
f ae bd y ax l ax s
0.94
eh kd s p- eh kd td
eh kd s p- eh kd t ih ng
eh kd s p- I ey n
f ae
f eh
f eh
f ao
0.80
0.17
explain
eh f
0.13
0.97
eh n t ay r iy
ih r iy
y ih r iy
eh s t ax m ey tf ax dd
iy th iy ow p iy ax
iy th iy ow p iy y ax
iy t iy ow p iy ax
y ao r ax pd
y er ax pd
ih ng g 1ax n dd
iy n ax dd
0.25
even
events
eh kd s p- ae n d ax dd
0.19
0.79
estimated
ethiopia
europe
0.27
0.17
0.71
0.27
0.98
0.75
erie
eh v er
eh v r iy
0.19
0.16
0.99
f ao I z
f aa r
ih ng g l ax sh
(ax iy) n ah f
(ax iy) k w ey tf er
eh s ax n
eh s t ow n iy ax
0.38
eh kd s ih td
iy eh kd s p- ae n d ax
dd
eh kd s p- eh kd t ax dd
ax kd s p- eh kd t ax dd
eh kd s p- eh kd s
eh kd s t eh n d ax dd
eh kd s t- eh n d ax dd
eh f
eh ax f
f ae kd t er
f eh r
f ao
england
f ae kd t er
f eh r
extreme
f eh
f ao
f ao
f aa
fahrenheit
r f ae kd s
1ax n
1z
r
fabulous
fairbanks
fall
r ax n hh ay td
r ax n hh ay td
r b ae ng kd s
1
0.89
0.57
0.16
0.12
0.99
0.95
0.99
0.96
eh kd s tr r iy m
f ae bd ( y ax | y uw)
l ax s
f ae r ax n hh ay td
f eh r b ae ng kd s
f ao I
Word
fast
fayetteville
february
PMM Baseforms
f ao r
f ae s td
f ey y ax td v ih 1
f ey ax td v ih 1
f eh bd y uw eh r iy
Weight
0.28
0.92
0.77
Expert Baseform
f ae s td
f ey ax td v ih
Word
failing
falmouth
PMM Baseforms
f ao 1 ih ng
f aw 1m ax th
fargo
f aa 1 m ax th
f ae I m ax th
f aa r g ow
1
0.11
0.44
f eh bd ( r uw | y uw)
Weight
0.99
0.66
0.16
0.13
Expert Baseform
f ao I ih ng
f ( aw 1 aa I ) m ax th
0.97
f aa r g ow
0.63
0.36
0.96
0.99
0.99
0.99
0.99
0.90
f ae kd s
( eh r I er ) iy
feet
fergus
few
fifth
fiji
find
finished
first
flagstaff
flight
flint
flooding
florence
flurries
flying
foggy
football
forecast
forecasts
forget
form
forty
fourteenth
framingham
f eh bd r uw eh r iy
f iy td
f er g ax s
f er g ax z
f y uw
f ih th
f ih f th
f iy jh iy
f ay n dd
f ih n ax sh td
f ih n iy sh td
f er s td
f I ae gd s t- ae f
f 1 ay td
f 1 ih n td
f 1eh n td
f 1 ah df ih ng
f I ao r ax n s
f 1 ao r ax n td s
f 1 ao r iy z
f 1 ay ih ng
0.32
0.99
f ao g iy
f aa g iy
f uh td b ao 1
l ax f uh td b ao l
f ao r k ae s td
f ao r k ae s td s
f er g eh td
f ao r m
f ao r tf iy
f ao r t iy n th
f r ey m ih ng hh ae m
fax
f ae kd s
f f ae kd s
fe
feeling
fell
fernando
fifteenth
fifty
f ey
f ay n dd
f ih n ax sh td
finally
f ay n 1 iy
f ay 1 iy
f ay n dd 1 iy
0.95
f er s td
f 1 ae gd s t- ae f
0.99
0.69
f I ay td
f 1 ih n td
fine
finland
five
flash
f 1 ah df ih ng
f 1 ao r ax n s
flights
0.99
0.99
f I er iy z
f l ay ih ng
flood
floods
florida
0.66
f ( aa I ao ) g iy
0.56
0.42
0.97
0.51
0.46
0.99
0.99
0.80
0.12
0.96
f iy td
f er g ax s
f y uw
f ih ( f th I th)
f iy jh iy
0.27
0.90
0.84
0.13
f iy 1 ih ng
f eh 1
f er n ae n d ow
f ih f t iy n th
f ih f t iy
f ay n
f
f
f
f
f
f
f
f
f
ih n 1ax n dd
ay v
1 ae sh
I ae sh s
1 ae sh f
1 ay td s
I ah dd
1 ah dd z
I ao r ax df ax
0.51
0.30
0.13
0.99
0.85
0.99
0.68
0.17
0.14
0.98
0.99
0.97
0.53
f
f
f
f
f
f
f
ey
iy 1 ih ng
eh 1
er n ae n d ow
ih f t iy n th
ih f t iy
ay n ax 1 iy
f ay n
f ih n 1ax n dd
f ay v
f1 ae sh
f l ay td s
f I ah dd
f 1 ah dd z
f 1 ( ao r ax I ao r ) df
ax
0.38
0.97
0.65
0.28
forks
f I ao r df ax
f I ay
f ao gd
f aa gd
f aa 1 ow ih ng
f ao r
f er
f ao r k ae s t ax dd
f ao r ax n
s f ao r ax n
f ao r kd s
0.99
f ao r kd s
( ax I ae ) n s ih s k
fort
f ao r td
0.98
f ao r td
eh dr r ax kd s b er
four
f ao r
0.97
f ao r
iy z
eh n ch
fourth
france
f ao r th
f r ae n s
f r ae n td s
f r ae ng kd f er td.
f r iy
f r iy z ih ng
f r eh z n ow
f r ax m
f r ah m
f ah kd
f aa kd
f ao ng kd sh ax n
f ah ng kd sh ax n
f er dh er
jh iy
jh iy iy
g ey n z v ih 1
0.98
0.55
0.42
f ao r th
f r ae n s
0.90
0.99
0.99
0.99
0.66
0.31
f
f
f
f
0.46
0.37
0.53
0.41
0.98
0.83
0.15
0.89
f ah kd
0.32
0.79
fly
f uh td b aol
fog
0.10
0.99
0.94
f ao r k ae s td
f ao r k ae s td s
following
0.83
f ( ao r I er ) g eh td
0.99
0.99
0.99
f ao r m
f ao r tf iy
f ao r t iy n th
forecasted
0.97
f r ey m ih ng ( hh ae |
for
foreign
0.99
0.53
0.42
0.99
0.85
0.14
f I ay
f ( aa Iao ) gd
f aa 1 ow ih ng
f ( ao r I er Iax r)
f ao r k ae s t ax dd
f ao r ax n
ax) m
francisco
f r ax n s ih s k ow
0.98
fredericksburg
f r eh dr r ax kd s b er
gd
f r iy z
f r eh n ch
f r ae n ch
f r ay df ey
f r ao s td
f r aa s td
f ax 1
f ao 1
f ah n iy
f ah n iy ey tf ax dd
f y uw ch er
g ae dd z d ax n
g ae 1 ax kd s iy
g ae 1ax kd s iy iy
g ey m
g ae td w ih kd
g ae td w iy kd
0.98
freeze
french
friday
frost
full
funny
future
gadsden
galaxy
game
gatwick
0.86
fr
ow
fr
gd
fr
fr
0.12
0.99
f r ay df ey
0.99
0.67
0.26
0.64
0.34
f r ao s td
frankfurt
free
f uh 1
freezing
fresno
0.66
f ah n iy
from
0.32
fuck
0.99
0.99
f y uw ch er
g ae dd z d ax n
0.70
0.29
g ae 1 ax kd s iy
0.99
0.61
g ey m
0.37
g ae td w ( ax I ih) kd
function
further
g
gainesville
r ae ng kd f er td
r iy
r iy z ih ng
r eh z n ow
f r ( ah l ax ) m
f ah ng kd sh ax n
f er dh er
jh iy
g ey n z v ih
1
Word
geneva
georgetown
I PMM Baseforns
jh ax n iy v ax
jh ih n iy v ax
jh ao r jh t aw n
jh ao r zh t aw n
Weight I Expert Baseform
jh ax n iy v ax
0.60
0.30
jh ao r jh t aw n
0.82
0.16
great
green
greensboro
jh er m ax n
jh er m ax n iy
g eh tf ih ng
g ih tf ih ng
g ih v
g I ey sh er
g 1 eh n w uh dd
g 1 ax n w uh dd
g ow ih ng
g uh
g aa td
g aa
g r ey td
g r iy n
g r iy n z b er ow
0.99
0.96
0.48
0.38
0.98
0.86
0.81
0.17
0.73
0.21
0.65
0.22
0.98
0.99
0.63
grenoble
g r iy n z b ao r ow
g r iy n z b aa r ow
g r ax n ow b ax I
0.17
0.10
0.99
ground
guadaloupe
guangzhou
g r aw n dd
g w aa df ax I uw p ey
g w aa ng z ow
0.99
0.99
0.66
guatemala
g w aa ng zh uw
g w aa tf ax m aa 1 ax
0.25
0.75
g w aa tf ax m aa I aa
0.14
w aa tf ax m aa 1ax
0.10
german
germany
getting
give
glacier
glenwood
going
got
ax
guiana
g y iy ae n
gulfport
guyana
g iy ae n ax
g aa iy ae n ax
iy ae n ax
g y aa n ax
g ah 1f p ao r td
g y ax n ax
0.21
0.15
0.09
0.09
0.80
0.49
g ay ae n ax
ey ch
0.49
0.91
0.84
0.99
0.95
0.99
0.74
0.99
0.61
0.18
0.12
0.92
0.99
0.96
h
hagerstown
haiti
hamburg
hampton
hanoi
happen
happening
harbor
harrisburg
has
hh ey g er
at
0.30
aw n
hh
hh
hh
hh
hh
hh
hh
hh
hh
hh
hh
ey
ae
ae
ae
ae
ae
ae
ae
aa
ae
ae
tf iy
m b er gd
m pd t ax n
n oy
p ax n
p ax n ih ng
pd n ih ng
p ax n iy ng
r b er
r ax s b er gd
z
hh
hh
hh
hh
hh
hh
ax v ae p ax
ax v aa n ax
ey v ax n
ax w ay iy
ax w ay iy iy
ih r
jh er
jh er
g eh
Word
PMM Baseforms
g ey n z v ih 1 ax 1
g ae 1v ax s t ax n
g aa r b ax jh
jh eh n ax r ax 1
Weight
0.09
0.95
0.99
0.51
go
good
jh eh n r ax 1
jh ao r jh
jh ao r jh ax
jh er m ax n t aw n
g eh td
jh ax b r ao I tf er
g ih v ih ng
jh ih v ih ng
g 1 ae z g ow
g Iae s k- ow
g 1ae s k- aw
g I aa s k- ow
g 1aa s g ow
g ow
g uh dd
0.34
0.99
0.98
0.99
0.97
0.99
0.85
0.14
0.27
0.26
0.11
0.10
0.09
0.98
0.92
grand
greece
greenland
g r ae n dd
g r iy s
g r iy n 1 ax n dd
0.98
0.95
0.48
g r ae n dd
g r iy s
g r iy n I ax n dd
greenville
g r iy n 1 ae n dd
g r iy n 1 ay n dd
g r iy n v ih 1
0.26
0.09
0.91
g r iy n v ih
1
g r aa tq en
g w aa df ax l ax hh aa
r ax
g w aa df ax aa hh aa
r ax
w aa df ax 1 ax hh aa r
ax
g w aa df ax l ax hh ae
r ax
g w aa m
g w aa r df iy ax
g eh s
g ah I f
g ay
g ay z
0.93
0.36
g r aa tq en
g w aa df ax
r ax
1 ax hh aa
0.81
0.92
0.99
0.98
0.99
0.74
g ay tf ax z
hh ae dd
hh ey I
hh ae l ax f ae kd s
hh ae m pd sh er
hh ae ng
hh ae n ow v er
hh ae p ax n dd
hh ax r aa r ey
hh ax r aa r ax
hh ax r aa r iy
hh ae r ax m ih n
hh aa r td f er dd
hh ae tf ax r ax s
0.25
0.95
0.83
0.93
0.99
0.80
0.99
0.99
0.72
0.13
0.13
0.99
0.98
0.86
hh
hh
hh
hh
hh
hh
0.12
0.99
0.91
0.94
0.88
0.11
galveston
garbage
general
m ax n
m ax n iy
( tf ih ng I tq en)
g ih v
g 1ey sh er
g 1eh n w uh dd
george
georgia
germantown
get
gibraltar
giving
g ow ih ng
glasgow
g aa td
g r ey td
g r iy n
g r iy n z b ( er
ow
ao r)
g r ax n ( ow aa ) b
ax 1
g r aw n dd
g w aa df ax 1 uw pd
g w aa ng ( z | zh Ijh)
(aw I ow)
g w aa tf ax m aa l ax
g ( ay I iy ) ( ae
n ax
I aa)
g ah If p ao r td
g ( ay I iy) ( aa
n ax
ey ch
hh ey
hh ey
hh ae
hh ae
groton
guadalajara
ae)
guam
guardia
guess
gulf
guy
guys
hh ae p ax n
hh ae p ax n ih ng
had
hail
halifax
hampshire
hang
hanover
happened
harare
hh aa r b er
hh ae r ax s b er gd
hh ae z
harriman
hartford
hatteras
g er z t aw n
tf iy
m b er gd
m pd t ax n
hh ( ae
I ax
) n oy
Expert Baseform
g ae Iv ax s t ax n
g aa r b ax jh
jh eh n ( ax r I er r )
ax 1
jh ao r jh
jh ao r jh ax
jh er m ax n t aw n
g eh td
jh ax b r ao 1 tf er
g ih v ih ng
g 1 se z g ow
g ow
g uh dd
0.31
0.10
0.10
g
g
g
g
g
g
w aa m
w ? aa r df iy ax
eh s
ah I f
ay
ay z
hh
hh
hh
hh
hh
ae dd
ey 1
ae I ax f ae kd s
ae m pd sh er
ae ng
hh ae n ow v er
hh ae p ax n dd
hh ax r aa r ( ey I ax)
hh ae r ax m ax n
hh aa r td f er dd
hh ae tf ( er I ax r ) ax
s
havana
haven
hawaii
hear
0.75
0.11
0.98
0.84
0.10
0.96
hh ax v ae n ax
hh ey v ax n
hh ax w ay iy
hh ih r
have
having
head
heard
ae tf er s
ae v
ae v ih ng
eh dd
er dd
ey dd
hh
hh
hh
hh
,
ae v
ae v ih ng
eh dd
er dd
Word
heat
heavy
heights
help
helsinki
here
hi
high
highest
hill
hilo
hingham,
hit
hobart
hole
hollywood
hong
hope
hotter
hours
how
humidity
hundred
huntington
hurricane
hutchinson
i
iceland
if
image
in
inclement
independence
PMM Baseforms
hh iy td
hh eh v iy
hh ae v iy
hh ay td s
hh eh 1 pd
hh eh 1 s ih ng k iy
hh ih r
hh y er
y ih r
hh ay
hh ay
hh ay ax s td
hh ih 1
hh iy 1ow
hh ih ng ax m
hh ih td
hh ow ax b aa r td
hh ow b aa r td
hh ow b er td
hh ax b aa r td
Weight
hh
hh
hh
hh
hh
hh
hh
ow 1
ow ax 1
aa 1iy w uh dd
ao 1 iy w uh dd
aa ng
ao ng
ow pd
0.66
0.31
hh ow l
0.88
hh ao 1 iy w uh dd
hh aw pd
hh aa tf er
aw er z
hh aw
hh y uw m ih df ax tf
iy
hh ah n d er dd
0.24
0.99
0.66
0.99
0.99
hh ah n dr r ax dd
hh ah nt ih ng t ax n
hh er ax k ey n
hh ah ch ax n s ax n
ay
ay s 1 ax n dd
ih f
ax f
ih m ax jh
ih n
ih n k 1 ax m ax n td
0.23
0.91
0.93
0.99
0.99
0.92
ih n d ax p eh n d ax
td s
ih n d ax p eh n d ax n
0.74
0.95
0.63
0.33
0.99
0.99
0.86
0.37
0.32
0.11
0.95
0.98
0.97
0.89
0.93
0.95
0.99
0.38
0.25
Expert Baseform
hh iy td
hh eh v iy
Word
heathrow
hh
hh
hh
hh
ay td s
eh 1 pd
eh 1 s ih ng k iy
ih r
heidelberg
hello
hh
hh
hh
hh
hh
hh
hh
hh
ay
ay
ay ax s td
ih 1
iy 1ow
ih ng ax m
ih td
ow b ( aa r I er) td
helpful
hemisphere
hey
hidden
higher
highs
hills
hilton
history
0.24
0.11
0.29
0.22
0.97
0.37
0.34
0.16
0.09
0.99
0.34
holland
honduras
honolulu
hot
hottest
houston
hh
hh
hh
hh
hh
0.20
hh aa tf er
aw er z
hh aw
hh y uw m ih df ax tf
iy
hh ah n (d er dr r ax
) dd
humid
hh ah nt ih ng t ax n
hh er ax k ey n
hh ah ch ax n s ax n
ay
ay s 1ax n dd
( ih I ax ) f
hummer
hungary
huntsville
hurricanes
hyannis
ice
0.75
hh ow pd
0.25
0.99
0.91
0.99
Weight I Expert Baseform
0.52
hh iy th r ow
0.18
0.16
hh ay df ax 1 b er gd
0.99
0.77
hh ( eh I ax ) 1ow
0.21
hh eh 1 pd f ax 1
0.99
hh eh m ax s f ih r
0.99
0.97
hh ey
hh ih dd en
0.99
0.42
hh ay r
0.33
hh ( aa I ao ) ng
0.72
hh iy th r ow
hh iy td r ow
hh iy tf r ow
hh ay df ax l b er gd
hh ax 1ow
hh eh 1ow
hh eh 1pd f ax 1
hh eh m ax s f ih r
hh ey
hh ih dd en
hh ay er
hh ay td er
hh ay eh r
hh ay z
iy 1 s
hh ih I z
hh ih I s
iy 1 z
hh ih I tq en
hh ih s t ax r iy
hh ih s tr r iy
hh ih s t er iy
hh ow
hh ow 1dd
td hh ow 1dd
hh aa 1ax n dd
hh aa n d ao r ax s
0.10
0.76
0.22
0.66
[PMM Baseforms
ho
hold
ih m ax jh
(ih ax) n
ih | ax) n k I ( ax
eh ) m ( ax I eh ) n td
ih n d ax p eh n d ax n
aa n d er ax s
aa n ax 1 uw 1uw
aa td
aa tf ax s td
y uw s t ax n
hh ay z
hh ih I z
hh ih 1 tq en
hh ih s ( t ax r
iy
tr r
0.20
0.99
0.87
0.12
0.92
0.79
hh ow
hh ow 1 dd
hh aa 1 ax n dd
hh aa n d ( er ao r)
ax s
0.87
hh
hh
hh
hh
hh y uw m ax dd
0.55
hh y uw m ax dd
0.43
0.98
0.84
0.87
0.98
0.97
0.48
0.41
hh ah m er
hh ah ng g ax r iy
hh ah n td s v ih 1
hh er ax k ey n z
hh ay ae n ax s
ay s
idaho
illinois
hh y uw m ih dd
hh ah m er
hh ah ng g ax r iy
hh ah n td s v ih 1
hh er ax k ey n z
hh ay ae n ax s
ay s
ay iy s
ay ay
ay df ax hh ow
ih 1 ax n oy
impressive
ax m p r eh s ax v
0.99
0.97
0.99
0.96
0.09
0.95
0.81
aa n ax 1 uw 1 uw
aa td
aa tf ax s td
y uw s t ax n
ay df ax hh ow
ih 1ax n oy
( ih I ax ) m p r eh s ax
v
s
0.20
inches
ih n ch ax z
0.99
ih n ch ax z
s
india
ih n d iy ax
0.93
ih n d iy ax
including
ih n k 1 uw df ih ng
0.96
indianapolis
ih n d iy ax n ae p ax 1
ax s
ih n f er m ey sh ax n
0.96
ih n d iy ax n ae p ax 1
ax s
ih n f ( ao r Ier ) m ey
sh ax n
index
ih n d eh kd s
0.99
( ih I ax ) n k 1 uw df
ih ng
ih n d eh kd s
indiana
ih n d iy ae n ax
0.99
ih n d iy ae n ax
ax n f er m ey sh ax n
0.09
indonesia
ih n d ax n iy zh ax
0.46
ih n d ( ow I ax ) n iy
zh ax
inquiring
ax n k w ay ih ng
0.48
ih n d ow n iy zh ax
0.33
0.24
0.18
0.84
innsbruck
instead
international
ih n z b r ax kd
ax n s t- eh dd
ih nt er n ae sh ax n ax
0.91
interested
iy n k w ay r ih ng
ih n k w ay er ih ng
ih n tr r ax s t ax dd
ih nt ax r ax s t ax dd
ih nt er n eh td
0.13
internet
iowa
ih nt er n ae sh n ax 1
ay ax w ax
0.18
0.62
information
0.87
0.52
(ih ng
ih ng
ih n)
k w ay r
ih ( nt ax In tr ) r ( eh
| ax ) s t ax dd
ih nt er n eh td
III
.
.
0.73
0.78
ih n z b r ( uh I ax ) kd
( ih I ax ) n s t- eh dd
ih nt er n ae ( sh ax
sh ) n ax 1
ay ax w ax
Word
iran
PMM Baseforms
ih nt er n eh th td
ih nt er n s td
ih nt er n ax td
ay ax r ae
y ax r aa
ireland
is
n
n
iy r aa n
y ax r ae n
ax r aa n
ay r ae n
ay r I ax n dd
ay r ax 1 ax n dd
ih z
s
island
isle
israel
it
ithaca
ivory
jacket
jacksonville
jamaica
january
jeopardy
jerusalem
johannesburg
johns,
jordan
juan
june
jupiter
k
kahului
kalispell
katmandu
kentucky
key
kilimanjaro
kind
kingdom
kinshasa
kittyhawk
knowledge
knoxville
ay I ax n dd
ay 1
ax ay ax 1
ih z r iy ax I
ih z r ey ax 1
ih z r ey 1
ih z r iy 1
ih td
ih th ax k ax
ay v r iy
jh ae k ax td
jh ae kd s ax n v ih 1
jh ax m ey k ax
jh ae n y uw eh r iy
jh ae n y so r iy
jh eh p er df iy
jh ax p er df ly
jh ax r uw s ax 1ax m
jh ax r uw s ax 1 aa m
jh ow hh ae n ax s b er
gd
jh aa n z
jh ao n z
jh ao r dd en
w aa n
jh uw n
jh uw p ax tf er
k ey
k aa hh ax 1 uw iy
k ae 1 ax s p- eh 1
k ae 1 ax s p eh I
k ae td m ae n d uw
k ax n t ah k iy
k iy
k ih I ax m ax n jh aa r
ow
k ay n dd
k ih ng d ax mn
k ih n sh aa s ax
k ih n sh aa sh sh ax
k ih n sh aa s aa td
k ih tf iy hh ao kd
k ih tf iy hh aa
n aa l ax jb
n aa kd s v ib 1
Weight I Expert Baseform
0.17
0.17
0.10
0.17
0.16
0.15
0.13
0.12
0.09
0.81
0.11
0.72
0.25
0.99
0.72
0.22
0.31
0.21
0.21
0.12
0.99
0.95
0.95
0.99
0.95
0.94
0.78
iraq
(ih r I ay r I ax r) (aa,
| ae ) n
irvine
islamabad
ay r 1 ax n dd
( ih
Iax
)z
ay 1ax n dd
ay 1
ih z r ( iy 1 Iiy ax 1I ey
ax l )
ih td
ih th ax k ax
ay v ax ? r iy
jh ae k ax td
jh ae kd s ax n v ih 1
jh ax m ey k ax
jh ae n y uw eh r iy
0.72
0.15
0.84
0.85
0.62
0.99
0.95
0.99
0.89
0.99
0.60
0.19
0.19
0.75
0.23
0.90
0.93
k ae td m ase n d uw
k ( eh I ax ) n t ah k iy
k iy
k ih 1ax m ax n jh aa r
ow
k ay n dd
k ih ng d ax m
k ih n sh aa s ax
iy r ae kd
0.15
ax r aa kd
0.14
ay r aa kd
er v ay n
ih z 1aa m ax b aa dd
ih z 1 aa m ax b ay dd
ax z 1aa m ax b aa dd
ay I ax n dd z
ay 1z
ih s t ae n b ax 1
ih s t- ae n b ax 1
ih s t ax n b ax 1
0.13
0.92
1
0.48
(ih r I ay r I ax r) (aa
I ae) kd
er v (ay I ax ) n
ax z 1 aa m ax b aa dd
0.31
0.18
0.97
0.99
0.35
ay I ax n dd z
ay 1z
ih s t aa n b uw I
0.20
0.17
0.14
ax td s
0.54
( ih
ih td s
0.30
ih z
0.14
0.98
0.98
jackson
jakarta
janeiro
jh ey
jh ae kd s ax n
jh ax k aa r tf ax
zh ax n eh r ow
jh ax n eh r ow
jh ax p ae n
jh er z iy
jh aa bd
y aa n
jh r aa n
jh ow n z b r ow
jh ow n z b er ow
hh ow z ey
0.91
0.70
0.28
I ax
) td s
jh ey
jh ae kd s ax n
jh ax k aa r tf ax
( zh I jh ) ax n eh r ow
0.94
0.99
ax p ae n
er z iy
0.80
aa bd
aa n
0.66
0.33
0.81
0.18
0.98
jh ow n z b er ow
0.47
jh ( uh I | ax
juneau
just
jh uh I ay
jh ax 1ay
jh uw n ow
jh ah s td
0.57
jh uw n ow
jh ah s td
kabul
jh ax s td
k aa b ax 1
0.34
0.30
k aa b uh 1
k ax b uw ax I
k ae b uw I
0.17
july
k ax b uw
kalamazoo
kansas
kenosha
kenya
kiev
killington
kinds
k ih tf iy hh ao kd
kingston
n aa 1 ax jh
n aa kd s v ih 1
0.42
its
0.32
0.79
0.09
ih tf ax 1 iy
jose
jh uw n
jh uw p ax tf er
k ey
k aa hh ax 1 uw iy
k ae I ax s p- eh I
Expert Baseform
0.93
jh ow hh ae n ax s b er
gd
jh aa n z
0.98
Weight
0.17
ih tf ax 1 iy
jonesboro
w aa n
Baseforms
ay ax w aa
ay ow w ax
ay r ae kd
italy
jh ax r uw s ax 1ax m
jh ao r dd en.
[ PMM
ih s t aa n b ax
jh eh p er df iy
0.99
0.99
0.99
islands
isles
istanbul
japan
jersey
job
john
0.10
0.67
0.28
0.85
0.12
0.85
Word
kitts
k
k
k
k
k
ax b uw 1
ae 1 ax m ax z uw
ae n z ax s
ax n ow sh ax
ax n ow sh s ax
0.46
0.99
hh ? ow z ey
1 ) ay
0.14
0.12
0.10
0.98
0.97
0.89
k ae 1 ax m ax z uw
k ae n z ax s
k ax n ow sh ax
0.10
k ( eh I iy ) n y ax
k iy eh v
k eh n y ax
k iy eh v
k iy ae f
k ih 1 ih ng t ax n
k ay n dd z
k ay n dd
0.96
k ih ng s t- ax n
0.89
k ih ng s t ax n
k ih ng s t ax n
0.09
0.99
k ih td s
k ih td s
0.65
0.14
0.98
0.67
k ih 1 ih ng t ax n
k ay n dd z
0.32
Word
kong
kosovo,
kuala
kuwait
labor
laconia
lagos
lake
lancaster
lanka
large
las
late
latest
lauderdale
lebanon
less
level
lewiston
lhasa
life
lihue
likelihood
lima
lisbon
listening
little
liverpool
located
locations
london
look
looks
lottery
louisiana
low
PMM Baseforms
k aa ng
k ao ng
k ow s ax v ow
k ao s ow v ow
k ow s ow v ow
k w aa 1 ax
k w aa 1 aa
k uw w ey td
eh 1
1 ey b ax r
1 ax k ow n iy ax
1 ax k ow n iy y ax
1aa g ow s
1 ey g ow s
1 ey kd
1 ae ng k ax s t er
Weight I Expert Baseform
0.70
k ( aa I ao ) ng
1 ae
1 aa
1 aa
1 aa
ng k ae s t er
ng k ax
ng k aa,
r jh
0.40
aa r jh
1 aa s
1 ey td
1 ay tf ax s td
1 ey tf ax s td
1ao df er d ey 1
1 aa df er d ey 1
1 eh b ax n aa n
1 eh b ax n ax n
1 eh s
1 eh v ax 1
I ax v ax l
1 uw ax s t ax n
1uw ax s t- ax n
1 aa s ax
I ay f
1 ax hh uw ey
1 ax hh uw iy
l ay kd 1 iy hh uh dd
0.32
0.94
1 iy m ax
1 iy in aa
1 ih z b ax n
0.59
1 ih s ax n ih ng
1 ax s t ax n ih ng
1ih tf ax 1
1 ih v er p ax 1
1 ih v er p ao 1
1 ow k ey tf ax dd
1 ow k ey sh ax n z
1 ah n d ax n
1 uh kd
I uh kd s
1uw kd s
I aa tf er iy
1 uw iy z iy ae n ax
1 ow
0.19
0.53
0.13
k ow s ax v ow
"'
Word
know
knox
kodiak
Weight _]Expert Baseform
n ow
0.99
0.94
n aa kd s
k ow df iy ae kd
0.39
laboratory
PMM Baseforms
n ow
n aa kd s
k ow df iy ae kd
k ow df iy y ae kd
td ow df iy ae kd
k ax r iy ax
k ao r iy ax
k ax r uw
k ao r uw
k ow ao r uw
k uh n m ih ng
k ah n m ih ng
k y ow tf ow
1ax
I aa
1ae bd r ax t er
0.27
lafayette
1ae
1ae
1ae
1aa
0.13
0.83
0.16
0.92
0.91
0.99
0.68
0.10
0.59
0.39
0.90
0.49
0.65
0.18
0.67
0.99
0.50
k w aa 1 ax
korea
k uw w ey td
eh 1
1ey b er
1ax k ow n iy ax
kourou
1aa g ow s
kyoto
kunming
la
1ey kd
1 ae ( ng [ n ) k ( ax
ae ) s t er
1 aa ng k ax
1 aa r jh
lahaina
1 ao df er d ey 1
lakes
0.21
0.84
1eh b ax n ( aa I ax) n
1eh s
1eh v ax 1
languages
lansing
largest
0.75
0.24
1 uw ax s t ax n
last
0.92
1 ( aa I ae ) s ax
I ay f
1 ax hh uw iy
0.15
0.99
0.83
0.10
0.98
0.62
0.09
0.99
later
latvia
lawrence
l ay kd 1iy hh uh dd
1 iy m ax
leesburg
let
1 ih z b ax n
1 ih s ( ax n n ) ih ng
levels
1 ih tf ax 1
1 ih v er p uw 1
lexington
libya
1 ow k ey tf ax dd
1 ow k ey sh ax n z
1 ah n d ax n
lightning
like
likely
0.98
0.59
0.35
1 uh kd
1 uh kd s
lincoln
0.99
0.92
1 aa tf er iy
1 uw iy z iy ae n ax
l ow
0.29
0.79
0.71
0.19
0.94
0.59
0.11
0.94
0.99
0.96
0.98
bd r ax tf er
bd r ax t er iy
bd r ax tf er iy
f iy eh td
1aa
1 aa s
1ey td
1ey tf ax s td
0.49
0.72
'
list
lithuania
0.35
0.22
0.85
0.12
k ( ax I ao ) r iy ax
0.52
k uw r uw
0.26
0.17
0.64
0.35
0.96
k uh n m ih ng
0.51
k ( iy y ) ow tf ow
I (ax I aa )
0.32
0.30
1ae bd r ax t ao r iy
0.23
0.14
0.42
f ey y eh td
1ae f iy eh td
Iaa f iy y eh td
1aa hh ay n ax
1ax hh ay n ay iy
1aa hh ey n ax
1ey kd s
l ay kd s
1 ae ng g w ax jh ax z
I ae n s ih ng
I aa r jh ax s td
1 aa r jh ax td
I ae s td
1ey tf er
1 ae td v iy ax
1 aa td v iy ax
1 ao r ax n s
1 aa r ax n s
1 iy z b er gd
1eh td
1 ax
1eh
1eh v ax 1 z
1eh v w ax 1z
1eh kd s ih ng t ax n
I ih b iy ax
0.14
0.10
1 ih bd y ax
1ay td n ih ng
l ay kd
1 ay kd 1 iy
1 ih ng k ax n
1 ih s td
1 ih th ax w ey n iy ax
1 ih th ax w ih n iy ax
1 ih td th w ey n iy ax
1 ih tf ax w ey n iy ax
0.36
0.99
0.99
0.09
0.69
0.16
0.14
0.77
1 ( aa I ae) f ( ay I iy
eh td
1ax hh ay
n ax
1ey kd s
0.22
0.93
0.86
0.77
1 ae ng g w ax jh ax z
1 ae n s ih ng
1 aa r jh ax s td
0.13
0.51
1 ae s td
1 ey tf er
1 ( aa I ae ) td v iy ax
0.48
0.64
I ao r ax n s
0.95
0.94
0.29
0.99
0.46
0.35
0.09
0.77
0.15
0.78
0.63
0.98
0.99
0.99
0.44
0.17
0.16
0.09
I iy z b er gd
1 eh td
1 eh v ax 1 z
1 eh kd s ih ng t ax n
1 ih ( b iy ax I bd y ax
I ay td n ih ng
l ay kd
I ay kd 1 iy
1 ih ng k ax n
I ih s td
I ih th uw ey n iy ax
Word
lower
lows
lucia
lumpur
luxembourg
lyon
macon
madison
madrid
major
malaysia
malta
managua
manhattan
]PMM
Baseforms
1 ow ao r
1 ow er
1 aw
1ao r
1 ow z
1 ax s y ax
1 uw sh ax
I ah m p ao r
I ah kd s ax m b er gd
1 ah kd s ax n b ao r gd.
1 ah kd s ax m b ao r
gd
1 iy aa n
1 y ao,
Weight
0.33
0.25
0.13
0.11
0.99
0.57
0.36
0.94
0.63
0.24
0.11
m
m
m
m
m
m
m
m
m
m
m
m
m
m
ey
ay
ae
ax
ey
ax
aa
aa
aa
aa
ax
ax
aa
ae
k ax ni
k ax n
df ax s ax n
dr r ih dd
jh er
1ey zh ax
1tf ax
1t aa
1t ax
1tf aa
n aa gd w ax
n aa gd w ah
n aa g w aa
n hh ae tq en
Expert Baseform
'
Word
''
I ow er
live
1ow z
1uw sh ax
local
location
logan
1ah m p ao r
l ah kd s ax m b er gd
long
looking
los
0.74
0.24
1 ay ax n
louis
louisville
0.81
0.18
0.99
0.76
0.99
0.94
0.50
0.23
0.16
0.09
0.64
0.11
0.09
0.87
m ey k ax n
m
m
m
m
ae df ax s ax n
ax dr r ih dd
ey jh er
ax 1ey zh ax
m ao
1tf
lowell
lowest
lubbock
luis
ax
m ax n aa g w ax
lusaka,
lynn
m
made
m ( ax I ae ) n hh ae tq
PMM Baseforms
1 ih td ax w ey n iy ax
1 ih v
I ax v
1ow k ax 1
1ow k ey sh ax n
1ow g ax n
1ow g ax n z
1ao ng
I uh k ih ng
1aa s
l ao s
Weight
0.09
0.82
0.14
0.96
0.89
0.86
0.13
0.96
0.99
0.57
0.40
1uw ax s
0.99
0.55
1 uw iy v ih 1
I uw ax v ax l
I uw ax v ih 1
1ow ax I
1ow ax s td
I ah b ax kd
Iuw ax s
1uw iy z
1uw iy s
1uw iy v z
I uw s aa k ax
1ih n
eh m
m ey dd
m ey td
0.28
0.10
0.97
0.99
0.96
0.36
0.33
0.10
0.10
0.99
0.99
0.97
0.88
0.09
Expert Baseform
1 ih v
1ow k ax 1
1ow k ey sh ax n
1ow g ax n
l ao ng
1 uh k ih ng
1 ( ow I ao I aa) s
1 uw (iy ax s)
1uw ( ax iy ) v ( ax
ih ) 1
l ow
1ow
1 ah
I uw
ax 1
ax s td
b ax kd
( ax s iy)
1uw s ( aa| ae) k ax
1ih n
eh m
m ey dd
en
manitoba
m ae n ax t ow b ax
0.96
m ae n ax t ow b ax
madras
m ax dr r aa s
0.76
map
marine
m ae pd
0.99
0.97
0.99
0.42
0.30
0.27
0.63
0.80
0.14
0.99
0.84
0.97
0.99
0.91
0.89
0.96
m ae pd
maine
m er iy n
make
mali
m
m
m
m
m
m
m
m
m
m
m
m
m
m
m
m
ey n
ey kd
aa1 l iy
ae n
ax n
ae n ch eh s t er
ax n ih I ax
ax n ax I ax
eh n iy
aa r ch
eh r ax t ay m
aa r 1 b r ow
aa r 1 b er ow
aa r z
aa r tq en
eh r ax 1 ax n dd
0.98
0.99
0.82
0.70
0.15
0.93
0.72
0.16
0.94
0.91
0.99
0.65
0.23
0.86
0.99
0.99
m ax r iy n
market
marquette
m
m
m
m
marseille
martinique
m aa r s ey
mass
maui
may
mckinley
mean
meant
mediterranean
m ae s
aa
aa
aa
aa
r
r
r
r
k ax td
k eh td
k ae td
kd ae td
m aa r tq en iy kd
m aa r tq en ey iy kd
memorial
m
m
m
m
m
m
iy
m
aw iy
ey
ax k ih n 1iy
iy n
eh n td
eh df ax tf ax r ey n
ax n
ax m ao r ly ax 1
menlo
menu
m ax m ao r ax 1
m eh n 1 ow
m eh n y uw
m ey n y uw
message
mexico
michigan
middle
midwest
m
m
m
m
m
m
ih n y uw
eh s axjh
eh kd s ax k ow
ih sh ax g ax n
ih df ax I
ih d w eh 6 td
m ih dd w eh s td
m aa r k ax td
m aa r k eh td
m aa r s ey
m aa r tq en iy kd
m
m
m
m
ae s
aw iy
ey
ax k ih n I iy
man
manchester
manila
many
march
maritime
marlborough
0.33
0.91
0.73
0.15
0.11
0.99
0.97
0.99
0.99
0.53
0.38
aa) s
m ey n
m ey kd
m aa 1 iy
m ae n
m ae n ch eh s t er
m ax n ih I ax
m eh n iy
m aa r ch
m ae r ax t ay m
m aa r I b ( r er ) ow
m eh n td
m eh df ax tf ax r ey n
mars
martin
maryland
ax n
m ax m ao r iy ax I
massachusetts
m ae s ax ch uw s ax td
s
0.97
maximum
maybe
m ae kd s ax m ax m
m ey b iy
m ae kd s ax m ax m
m ey b iy
me
meaning
m iy
iy n ih ng
eh dd f er dd
eh 1b er n
eh Ib ao r n
eh m pd f ax s
eh n sh ax n dd
eh n t ax n dd
0.99
0.91
0.99
0.98
0.89
0.62
0.36
0.94
0.88
0.09
m ax r ih df iy ax n
0.39
m ax r ih df iy ax n
m iy n
iy
0.66
m ( aa dr r ax | ax dr r
m eh n 1 ow
m eh n y uw
m
m
m
m
m
td
eh s ax jh
eh kd s ax k ow
ih sh ax g ax n
ih df ax 1
( ih I ax ) d w eb s
medford
melbourne
memphis
mentioned
meridian
m
m
m
m
m
m
m
m aa r z
m aa r tq en
m eh r ax I ax n dd
m ae s ax ch uw s ax td
s
m iy
m iy n ih ng
m eh dd f er dd
m eh 1 b er n
m eh m (f I pd f) ax s
m eh n sh ax n dd
Word
PMM Baseforms
Weight
might
m ay td
mild
mind
m ay 1 dd
m ay n dd
0.99
0.93
0.95
Expert Baseform
m ay td
m ay 1dd
m ay n dd
minneapolis
m ih n iy ae p ax I ax s
0.99
m ih n iy ae p ax 1ax s
minot
m ih n ow
0.39
m ( ih n ow I ay n aa td
m ay n aa td
m ay n ae td
m ay n ow
0.28
0.17
0.13
mississippi
m ih n ax td
m ih s ax s ih p iy
0.93
0.90
m ih n ax td
m ih s ax s ih p iy
missouri
m ax z er iy
0.73
m ax z ( er I ao r ) ( iy
m ax z ao r iy
m ow g ax d iy sh uw
m ow m ax n td
m aa m ax n td
m ah n d ey
m aa n ax k ax
m aa n s uw n z
m aa nt iy
m ah nt ax
m aa nt ax r ey
m aa nt ax v ax d ey eh
0.20
0.99
0.69
0.26
0.98
0.99
0.96
0.79
0.18
0.99
0.35
minute
mogadishu
moment
monday
monica
monsoons
monte
monterey
montevideo,
ow
m aa nt ax v ih df iy uw
m aa nt ax v ax d ey
Word
0.37
0.22
0.58
0.99
m ay ae m iy
mid
midland
midwestern
m ih dd
m ih dd I ax n dd
m ih d w eh s t ax n
0.99
0.88
0.86
milan
milwaukee
m ax 1 aa n
m ax 1w ao k iy
0.86
0.84
m
m
m
er
m
m
m ih 1 w ao k iy
0.09
mobile
m ih n
m ih n ax s ow tf ax
m ax n ax s ow tf ax
m ih ng td s kd
m ih n s kd
m iy n td s kd
ax m ih s td
m ih s td
m ax z uw I ax
m ax z ax 1ax
m ow b iy 1
0.99
0.84
0.13
0.56
0.25
0.14
0.52
0.47
0.75
0.20
0.94
moines
monaco
m oy n
m aa n ax k ax
0.95
0.60
m aa n ax k ow
0.35
miami
minnesota
minsk
m ah n d ey
m aa n ax k ax
missed
m aa n s uw n z
m aa nt iy
missoula
m aa nt ax r ey
m aa nt ax v ( ax d ey
ow I ih df iy ow)
0.28
0.12
m eh tr r ax p aa 1 ax
tf ax n
0.41
ih dd
ih dd I ax n dd
( ih I ax) d w eh s t
n
ax 1 (aa |ae ) n
( ih 1 | ax 1 ) w ao k
1y
minh
m ow g ax d iy sh uw
n ow m ax n td
Weight I Expert Baseform
m ax r ih df iy ae n
m ax r ih df iy eh n
m eh tr r ax p aa 1 ax
tq en
m eh tr r ax p aa 1 a x
tf ax n
m ay ae m iy
metropolitan
lax )
JPMM Baseforms
m ih n
m ih n ax s ow tf ax
m ih n s kd
m ih s td
m ax z uw
1ax
m ow b ( ax 1 ay 1 | iy
1)
m oy n
m aa n ax k ow
ow
m aa nt ax v ih df ey
0.11
ow
month
months
montpelier
m aa nt ax v ih d ey ow
0.10
m
m
m
m
m
m
0.54
0.44
0.34
0.30
0.20
0.62
aa
ah
ah
aa
aa
aa
n th
n th
n th s
n
n th s
n p iy 1 y er
mongolia
m ah n th
m ah n th s
m aa n p ( iy 1 y er
1iy
ax
0.43
ow 1 iy ax
ow 1y ax
ax n y ax n
ax ow n
ae n ax
0.36
0.10
0.48
0.47
0.99
0.82
montana
montego
m
m
m
m
m
m
monterrey
montgomery
m ah n t iy g ow
m aa nt ax r ey
m aa n td g ah m r iy
0.10
0.99
0.35
0.23
0.15
0.62
0.18
0.09
0.97
monsoon
Iax
m aa ng g ow
aa
aa
aa
aa
aa
ng
ng
ns
ns
nt
aa n t iy g ow
1 y ey )
m aa n td p iy 1 y er
m aa n y ax m ax n td
m aa n y uw m eh td
0.22
0.45
0.31
most
m aa n y uw m ax n td
m ao r hh eh dd
m ao r n ih ng
m ao r ax s t aw n
m aa r ax s t aw n
m ow s td
0.21
0.99
0.99
0.72
0.24
0.89
mountain
m aw n tq en
0.81
m aa n t ax n
m uw v
m aw ey
m ax v ax
m uw v iy z
m ax 1v iy z
m ah ch
m y uw z ax kd
m ay
m ay r z
m ay er z
m ay r ax sh
eh n
0.16
0.33
0.28
0.25
0.80
0.10
0.98
0.97
0.99
0.50
0.29
0.09
0.74
0.24
monument
moorhead
morning
morristown
move
movies
much
music
my
myers
n
ih n
m aa n y ax m ax n td
m ao r hh eh dd
m ao r n ih ng
m ao r ax s t aw n
monthly
m ow s td
monticello
m
m
m
m
m
m
m aw n tq en
montreal
m aa n tr r iy ao 1
0.51
m uw v
moon
more
morocco
m
m
m
m
m
m
m
m
m
m
m
m
eh
m
0.40
0.76
0.99
0.74
0.14
0.11
0.50
0.42
0.89
0.79
0.12
0.71
0.28
0.99
m uw v iy z
m
m
m
m
ah ch
y uw z ax kd
ay
ay r z
moscow
mount
mountains
movie
eh n
.rnozambique
ah td g ah m r iy
aa nt g ah m r iy
ah n th 1 iy
ah n th ax I iy
aa n t 1iy
aa nt ax s eh 1ow
ah n tr r iy ao 1
uw n
ao r
ax r aa k ow
aa r aa k ow
aa ax r aa k ow
aa s k ow
aa s k aw
aw n td
aw n tq en z
aw n tq en s
uw v iy
m eh uw v iy
ow z ae m b iy kd
m ( aa I ax) (n
g ow 1 iy ax
ng )
m aa n s uw n
m aa n t ae n ax
m aa n t iy g ow
m aa nt ax r ey
m aa n td g ah m (r
ax r ) iy
m ah n th 1 iy
m aa nt ax ( ch I s) eh
1 ow
m ( aa I ah ) n tr r iy
ao 1
m uw n
m ao r
m ( ax ao) r aa k ow
m aa s k (ow I aw)
m aw n td
m aw n tq en z
m uw v iy
m ow z ae m b iy kd
Word
nairobi
named
nanchang
nantucket
naples
nashville
natchez
near
neat
need
nepal
nevada
new
newfoundland
'
[ Expert Baseform
Word
munich
muskogee
myanmar
[ PMM Baseforms
Weight I Expert Baseform
m y uw n ax kd
0.90
0.88
m ax s k- ow g iy
m ( iy Iay) (aa ax)
0.66
n m aa r
0.16
0.16
0.98
m er tf ax 1
n aa g aa n ow
0.47
PMM Baseforms
n ay r ow b ly
n ey m dd
n ae n dd ch ae ng
Weight
n ae n t al k ax td
n ae n t al k eh td
n ey p ax 1z
n ae sh v ili I
n ae ch iy z
n ae ch ax z
n ae ch ax z s
n ih r
n y ih r
n iy td
n iy dd
n ax p ao 1
n ey p ao 1
n ax p aa 1
n eh p aa 1
n ax v ae df ax
n ax v aa df ax
n uw
n uw f ao n dd 1ae n dd
0.78
n uw f ax n dd I ax n
dd
n uw f ax n dd I ae n
dd
n uw f ax n dd I aa n
0.28
n ay tf er
0.17
0.24
n ay td er
0.13
0.09
n iy dh eh r
0.10
n
n
n
n
ay dh er
eh dh er 1ax n dd z
eh v er
uw ao r kd
0.09
0.99
n uw er kd
n uw kd
n uw z
n ay ae gd r ax
n ay ae g r ax
n ay s
n iy s
n ay td
n ay n
n ay n t iy n th
n iy n y ow
n ow
n ah n
n aa n
n ow pd
n ao r m ax 1
n ao r th
n ao r th hh iy s t er n
n ao r th iy s t er n
n ao r th w eh s td
n aa td
ow
n ow
n ow v ax
n aw
n ah m b er z
n ah m b er z s
0.14
1
0.90
0.99
0.92
0.12
0.99
0.97
0.66
'
n ay r ow b iy
n ey m dd
n ae n ch ae ng
n ae n t ah k ax td
n ey p ax 1z
n ae sh v ih 1
n ae ch ax z
0.21
0.11
0.44
'"
n ih r
0.27
myrtle
nagano
name
names
nanjing
naperville
nashua
nassau
0.99
0.99
0.36
0.23
0.19
0.09
n iy td
n iy dd
n ax p (aa 1 I ao 1)
0.67
n ax v ( ae [ aa ) df ax
nearest
0.32
0.99
0.30
n uw
n uw f ax n dd 1 ax n
nebraska
neither
national
'
m y uw n ax kd
m ax s k- ow g iy
m ay ae n m aa r
m iy ax n m aa r
m y aa n m aa r
m er tf ax 1
n aa g ax n ow
n aa g aa n ow
n ey m
n ey m z
n ae n jh ih ng
n ey p er v ih 1
n ae sh uw ax
n ae s ao
n ae s ao 1
n ae s s ao
n ae sh ax n ax 1
n ae sh n ax 1
n ih r ax s td
n iy ax s td
n ax b r ae s k ax
n iy dh er
'
0.36
0.99
1
0.98
0.89
0.89
0.39
0.16
0.13
n
n
n
n
n
n
ey
ey
ae
ey
ae
m
mz
n jh ih ng
p er v ih I
sh uw ax
ae s ao
0.80
0.14
0.77
0.11
n ae sh ( ax n n) ax 1
0.99
n ax b r ae s k ax
n ( ay I iy ) dh er
0.45
n ih r ax s td
dd
dd
newport
next
nicaragua
nigeria
nighttime
nineteen
ninety
ninth
nome
noon
norfolk
normally
northeast
northern
norway
nothing
november
number
0
obispo
occured
n
n
n
n
uw p ao r td
eh kd s td
ih k ax r aa gd w ax
ih k ax r aa gd w aa
n
n
n
n
n
n
ay
ay
ay
ay
ay
ay
jh ih r iy ax
td t ay m
n t iy n
n d iy
nt iy
n th
n ow in
n uw n
n ao r f ax d
n ao r f ao 1kd
n ao r f ax 1 kd
n ao r f ow k
n ao r ax I iy
n ao r th iy s td
n ao r t iy s td
n ao r dh er n
n ao r w eh ey
n ao r w ey
n ah th ih rg
n ow v eh m b)er
n ah m b er
ow
ow ax 1
ax b ih s p ow
ax b ih z p ow
ow b ih s p ow
ax k er dd
0.89
0.91
0.56
n uw p ao r td
n eh kd s td
n ih k ax r aa g w ax
0.35
0.94
0.99
0.99
0.81
0.18
0.91
0.93
0.99
0.46
0.20
n
n
n
n
ay jh ih r iy ax
ay td t ay in
ay n t iy n
ay ( nt I n d ) iy
n ay n th
0.74
0.23
0.96
0.52
0.21
0.99
0.92
0.93
0.88
0.10
0.26
0.24
0.13
0.99
newk
news
niagara
nice
n ow m
n uw n
n ao r f ax kd
0.17
0.11
0.99
netherlands
never
newark
n ao r m ax 1 iy
n ao r th iy s td
night
nine
nineteenth
nino,
no
none
n ao r dh er n
n ao r w ey
nope
normal
north
northeastern
n ah th ih ng
n ow v eh m b er
n ah m b er
ow
northwest
not
ax b ih z p ow
ax k er dd
nova
now
numbers
0.98
0.68
0.99
0.59
0.25
0.76
0.23
0.99
0.96
0.99
n eh dh er 1 ax n dd z
n eh v er
n (uw er I uw aa r ao
r) kd
n uw z
n ay ae g r ax
n ( iy | ay ) s
n
n
n
n
n
n
ay td
ay n
ay n t iy n th
iy n y ? ow
ow
ah n
0.99
0.99
0.99
0.90
0.09
n
n
n
n
ow pd
ao r m ax 1
ao r th
ao r th iy s t er n
0.92
0.58
n ao r th w eh s td
n aa td
0.92
0.99
0.81
0.11
0.16
0.14
0.99
0.95
0.46
0.25
n ow v ax
n aw
n ah m b er z
Word
PMM Baseforms
Weight
Expert Baseform
october
off
aa kd t ow b er
0.97
0.88
0.11
0.95
0.92
0.93
0.84
0.12
0.45
0.33
aa kd t ow b er
ogden
okay
old
omaha
once
only
ao f
aa f
aa gd d ax n
ow k ey
ow 1 dd
ow m ax hh aa
ow m ax hh ao
w ah n s
w aa n s
w ah n s eh
ow n 1 iy
ao n 1 iy
ow n iy
ow 1 iy
ow ax 1 iy
ao 1 iy
options
aa pd sh ax n z
orange
ao r n jh
oregon
ao r ax n jh
ao r ax g ax n
ao r ax g aa n
Word
ao f
aa gd d ax n
ow k ey
ow 1 dd
ow m ax hh ( aa I ao)
PMM Baseforms
Weight
Expert Baseform
n ow m b er z
0.15
0.93
0.99
ow kd 1 ax n dd
ax k er
ow kd
ocean
of
ow sh ax n
ax v
ao f er
aa f er
ow hh ay ow
ow kd I ax hh ow m ax
ax 1 ih m p iy ax
offer
w ah n s
oklahoma
olympia
ow n 1 iy
on
one
ow
1ih
m p iy ax
aa n
w ah n
w aa n
aa n t eh r iy ow
ao r
ao r df er
ow r df er
ao r iy ax n td
ao r iy eh n
ao r iy eh nt
ao r 1 ax n z
ontario
or
order
aa pd sh ax n z
ao r ax n jh
n dd
ax k er
ohio
0.20
0.31
0.18
0.14
0.12
0.11
0.09
0.89
0.56
0.43
0.88
0.10
1ax
oakland
occur
orient
ao r ax g ( aa Iax) n
orleans
0.86
0.76
0.73
ow sh ax n
( ah I ax ) v
ao f er
0.25
0.98
0.98
0.48
0.36
0.94
0.90
0.09
0.96
0.93
0.80
0.18
0.66
ow hh ay ow
ow kd 1 ax hh ow m ax
(ow l ax) l ih m p iy
ax
( ah I aa ) n
w ah n
aa n t eh r iy ow
ao r
ao r df er
ao r iy ( eh I ax ) n td
0.22
0.11
0.71
ao r
1 ( ax
| iy
I iy
ax )
nz
orlando
orly
oslo
ottawa
out
outside
overcast
oxford
p
pakistan
palo
paraguay
ao r 1ae n d ow
w ao r 1 iy
ao r 1 iy
aa z 1 ow
aa s 1 ow
aa tf ax w aa
aa tf ax w ax
aw td
aw td s ay dd
ow v er k ae s td
aa kd s f er dd
aa kd s f er td
p iy
p iy ey
p ae k ax s t ae n
p ae k ax s t ax n
p aa k ax s t aa n
p aa 1 ow
p ae 1 ow
0.98
0.73
0.26
0.47
0.46
0.57
0.34
0.98
0.91
0.99
0.62
0.35
0.67
0.26
0.62
0.11
0.10
0.63
0.33
ao r 1 ae n d ow
p ae r ax g w ay
0.46
p ae r ax g w ( ay
ao r I iy
osaka
aa ( z | s ) 1ow
other
our
outlook
over
overnight
ozone
aw td s ay dd
ow v er k ae s td
aa kd s f er dd
pacific
palm
p iy
p ae k ax s t ae n
aa r
0.43
0.31
0.23
0.92
( aw er I aa r)
0.98
ow v er
0.99
ow v er n ay td
ow z ow n
p aa m
p aa 1 m
p ae n ax m aa
p aa r dd en
p aa r df ax n
p aa r kd
p ax r tf ax s ax p ey tf
ih ng
p ax r t ih s ax p ey tf
panama
pardon
park
participating
p ( ae I aa) 1 ow
0.20
0.99
ax ey
aa r ey
aw td 1 uh kd
ow v er
ow v er n ay td
ow z ow n
ow z ow n ax td
p ax s ih f ax kd
aa tf ax w ax
aw td
ao r 1 iy n z
ow s aa k ax
ah dh er
ey
0.94
0.79
0.19
0.92
0.73
0.23
0.91
0.74
0.17
0.99
0.77
ow s aa k ax
ah dh er
aw td 1 uh kd
p ax s ih f ax kd
p aa 1 ? m
p ae n ax m aa
p aa r dd en
p aa r kd
p aa r t ih s ax p ey tf
ih ng
0.17
ih ng
paris
part
partly
paso
paul
paz
pegasus
pendleton
p
p
p
p
p
p
p
p
p
p
p
p
p
p
ae
ae
ae
aa
aa
ae
ao
aa
aa
eh
ey
eh
eh
eh
r ax g w eh ey
r ax g w ey
r ax s
r td
r td 1 iy
s ow
1
z
ay z
g ax s ax s
g ax s ax s
n d ax 1 tf ax n
n dd ax 1 tf ax n
n ax 1tf ax n
0.22
0.22
0.97
0.99
0.94
0.85
0.97
0.78
0.11
0.79
0.16
0.34
0.33
0.20
pasadena
past
p ae s ax d iy n ax
p ae r ax s
paulo
p aa r td
p aa r td 1 iy
p ae s ow
p aa 1 ow
p ao I ow
pearson
k er s ax n
0.99
peking
p iy k ih ng
p ey k ih ng
0.65
penh
p ae n
pensacola
p eh n s ax k ow I ax
p eh g ax s ax s
peoria
p y ao r iy ax
p iy ao r iy ax
p eh n d ax 1 tf ax n
percentage
peru
p er s eh nt ax jh
p er uw
p ax r uw
0.97
0.83
0.71
0.24
0.85
p ao I
p aa z
,
p ae s td
,
0.99
0.99
0.49
p ae s ax d iy n ax
p ae s td
p ( aw I ao) 1 ow
0.37
p ih r s ax n
p iy k ih ng
0.33
0.36
0.25
p eh n
p eh n s ax k ow 1 ax
p iy ao r iy ax
p er s eh nt ax jh
p er uw
Word
pennsylvania
people
percent
perth
petersburg
philippines
phoenix
picture
pine
pittsburgh.
pizza
places
plains
planning
play
plus
pocatello
poland
PMM Baseforms
p eh n s ax I v ey n y
ax
p iy p ax 1
p er s eh n td
p er th
p iy tf er z b er gd
f ih I ax p iy n z
f iy n ax kd s
p ih kd ch er
p ay n
p ih td s b er gd
p iy td s ax
p 1ey s ax a
p 1ey n z
p 1ay ae n z
p1 ae n i ng
p 1ey
p 1ay
p 1ah s
p 1ax s
p ow k ax t eh 1ow
p aa k ax t eh I ax
p ow I ax n dd
Weight I Expert Baseform
0.99
| p eh n s ax 1v ey n y
ax
0.92
p iy p ax 1
0.81
p er s eh n td
0.89
p er th
0.98
p iy tf er z b er gd
0.99
f ih I ax p iy n z
f iy n ax kd s
0.99
0.99
p ih kd ( ch sh ) er
0.99
p ay n
0.99
p ih td s b er gd.
0.99
p iy td s ax
0.95
p 1 ey s ax z
0.90
p 1 ey n z
p aa 1 ax n
p ao r td
p ao r td s rn ax th
p aa s ax b ih 1ax tf iy
p ax k ih pd s iy
p r aa gd
p r aa kd
p r ax d ih kd td
0.09
1
0.84
0.11
0.77
0.15
0.90
0.09
0.97
"
Word
philadelphia
PMM Baseforms
f ih 1 ax d eh If iy ax
Weight [_Expert Baseform
0.97
f ih 1 ax d eh 1f iy ax
0.99
0.91
price
n aa m
f ow n
p iy eh r
p ih s k ae tf ax w ey
p ax s k ae tf ax w ey
p ax s k ae tf ax w ay
p ih td s f iy I dd
p 1ey s
p 1ey n
p 1ae n ax td
p 1ae n ey
p 1 ae td
p 1ax td
p 1iy z
p 1ih m ax th
p 1 ih m ax th td
p oy n td
p ao I
p ow 1
p ow ax 1
p aa pd y ax 1 ey sh ax
n
p ao r td I ax n dd
p ao r ch ax g ax I
p aa s ax b ax 1
p aa s ax b ax 1 z
p ao I
p aa ax 1
p aw ax 1
p r ax s ih p ax t ey sh
ax n
p r ax d ih kd t ax dd
p r ax d ih kd sh ax n
p r eh z ax n td
p r eh s kd
p r eh sh kd
p r ax t ao r iy ax
p r ax t ao r y ax
p r ay s
princeton
probably
project
p r ih n s t ax n
p r aa b ax 1 iy
p r aajh eh kd td
0.90
0.99
p
p
p
p
p
0.16
projection
providence
provo
puerto
pyongyang
p ao r tf ow
p y aa ng y ae ng
0.37
0.47
qingdao
p y ah ng y ae ng
p y aa ng y ax ih ng
ch iy ng d aa ow
0.25
0.11
0.70
jh ih ng d aw
k ax b eh kd
k w ax b eh kd
k w ax b ae kd
k r iy
0.16
phnom
phone
pierre
piscataway
pittsfield
place
plain
planet
platte
p I ae n ih ng
p Iey
please
plymouth
p 1ah s
point
pole
p ow k ax t eh I ow
p ow I ax n dd
population
0.92
0.98
0.96
0.99
0.95
0.73
0.14
0.99
p aa I ax n
p ao r td
p ao r td s m ax th
p aa s ax b ih 1ax tf iy
p ax k ih pd s iy
p r aa gd
portland
portugal
possible
p r ax d ih kd td
precipitation
p r ax d ih kd t ih ng
p r ax d ib kd sh ax n z
p r eh z ax df ax n td
p r eh sh er
p r ih tf iy
p r ih n s
p r ax n td s
p r aa b ax b ih I ax tf
iy
p r ow gd r ae m
p r axjh eh kd t ax dd
p r ax v ay dd
0.99
0.99
0.96
0.99
0.99
0.73
0.25
0.94
p
p
p
p
p
p
predicted
prediction
present
presque
p
p
p
p
p
0.87
0.12
0.43
0.40
0.99
p r aa v ax n s t- aw n
k y uw z
k y uw
0.50
0.49
k y uw
quality
queens
question
k w aa 1ax tf iy
k w iy n z
k w eh s ch ax n
0.97
0.90
0.52
k w aa I ax tf iy
k w iy n z
k w eh ( s ch I sh ch ?
) ax n
quincy
k w eh sh ax n
k w eh sh ch ax n
k w ih n s ly
k w ih n z ly
k ih n iy
0.29
0.10
0.48
0.32
0.14
pollen
port
portsmouth
possibility
poughkeepsie
prague
predict
predicting
predictions
president
pressure
pretty
prince
probability
program
projected
provide
provincetown
pueblo
puson
q
r aa v ax n s t aw n
r aa v ax n s t ax v n
w eh bd Iow
w eh b 1ow
uw s aa n
0.99
0.99
0.99
r ax d ih kd t ih ng
r ax d ih kd sh ax n z
r eh z ax df ax n td
r eh sh er
r ih tf iy
r ih n s
pr
iy
pr
pr
pr
aa b ax b ih
Iax
tf
ow gd r ae m
ax jh eh kd t ax dd
ax v ay dd
p w eh b 1ow
p uw s aa n
powell
pretoria
quebec
k w ih n ( s I z ) iy
query
'
r aajh ax kd td
r ax jh eh kd sh ax n
r aa v ax df ax n s
r ow v ow
ao r tf ax
0.86
0.38
0.37
( f n I p n I n ) aa m
f ow n
p iy eh r
p ax s k ae tf ax w ey
0.10
0.97
0.99
0.98
0.76
p ih td s f iy Idd
p 1ey s
p 1 ey n
p 1 ae n ax td
0.09
0.86
0.12
0.99
0.71
0.09
0.93
0.39
0.37
0.13
0.85
0.99
0.99
0.67
0.32
0.49
0.39
0.11
0.95
0.99
0.99
0.99
0.60
0.31
p 1 ae td
p 1iy z
p 1 ih m ax th
p oy n td
p ow 1
p
n
p
p
p
aa pd y ax 1ey sh ax
ao r td I ax n dd
ao r ch ax g ax 1
aa s ax b ax 1
p aw ax 1
p r ax
ax n
p r ax
p r ax
p r eh
p r eh
s ih p ax t ey sh
d ih kd t ax dd
d ih kd sh ax n
z ax n td
s kd
0.87
p r ax t ao r iy ax
0.09
0.99
p r ay s
0.79
0.99
0.93
0.99
0.59
0.56
p r ih n s t ax n
p r aa b ax b 1iy
p r ( ax I aa ) jh eh kd
td
p r ax jh eh kd sh ax n
p r aa v ax df ax n s
p r ow v ow
p ( w eh ao) r tf ( ow
I ax )
p y ( aa I ah) ng y (aa
I ae ) ng
ch iy ng d aw
k ( ax w ax) b eh kd
0.32
0.09
0.34
k w ( ih r I eh r ) iy
Word
quite
r
rain
[ PMM Baseforms
Weight
k w ay td
aa r
r ey n
0.99
0.99
record
r ey n dd
r ey n ih ng
r ey n iy
r ey n jh
r ae p ax dd
r ae p ax td
r ax t ow n
r eh df ih ng
r iy df ih ng
r iy 1iy
r ax s iy v dd
r eh k ax gd n ih sh ax
n
r eh k ax gd n dd ih sh
ax n
r eh k er dd
regarding
r iy g aa r df ih ng
0.99
region
relative
r iy jh ax n
r eh 1 ax tf ih v
r eh 1 ax tf ax v
r ax p iy td
r iy p iy td
r iy p ao r td
r ax p ao r td
r iy p ao r td s
r ax p ao r td s
r ax k w eh s td
r iy k w eh s td
r iy s t- aa r td
r eh s tr r aa n td s
0.95
r eh s t ax r aa n td s
0.21
r ey kd y ax v ih kd
0.46
r ey kd jh ax v ih kd
r ey kd y ax v iy kd
0.19
0.13
rhodes
richmond
right
rise
r ow dd z
r ih ch m ax n dd
r ay td
r ay z
0.91
river
road
rochester
r ih v er
r ow dd
r aa ch eh s t er
0.91
0.99
rockford
rocky
rome,
r aa kd f er dd
r aa k iy
r ow m
0.93
0.99
rosa
rouge
r ow z ax
r uw zh
r uw jh
r ah td 1 ax n dd
s ae kd r ax m eh nt ow
s eh dd
s ey 1 ax m
s ao 1 z b eh r iy
s ae 1v ax d ao r
s ey m
s ae n
0.91
rained
raining
rainy
range
rapid
raton.
reading
really
received
recognition
repeat
report
reports
request
restart
restaurants
reykjavik
rutland
sacramento
said
salem
salisbury
salvador
same
san.
0.85
0.99
0.98
0.99
0.97
0.72
0.17
0.79
0.86
Expert Baseform
k w ay td
aa r
r ey n
r
r
r
r
r
ey
ey
ey
ey
ae
n dd
n ih ng
n iy
n jh
p ax dd
r ax t ow n.
r ( eh I iy ) df ih ng
0.12
0.83
0.99
0.87
r iy 1iy
r ax s iy v dd
r eh k ax gd n ih sh ax
n
Word
questions
quit
quito
rabat
raincoat
rainfall
rains
raleigh
0.12
0.99
0.66
0.25
0.50
0.48
0.55
0.44
0.50
r ( eh k er ax k ao r)
dd
r ( iy I ax) g aa r df ih
ng
r iy jh ax n
r eh I ax tf ( ih I ax ) v
ranges
r (iy
readings
ax ) p iy td
0.59
0.94
0.99
0.99
0.97
0.98
0.80
0.18
0.96
0.98
0.81
0.99
0.84
0.95
0.99
0.99
k w eh sh ax n z
k w ih td
k w eh td
k iy tf ow
k iy t ow
k w iy tf ow
r aa b ao td
r ey n k ow td
r ey n f ao 1
r ey n z
r ey n s
r aa 1 iy
0.35
r ao 1 iy
0.10
r ey n jh ax z
0.68
r ey n jh iy z s
0.27
r
r
r
r
r
0.98
0.80
r ae p ax dd z
r iy ch
0.10
0.49
r iy df ih ng z
ae p ax dd z
iy ch
iy ch s
iy df ih ng z
iy ih ng z
0.33
0.31
0.62
0.75
0.48
0.28
0.15
0.94
0.96
0.98
0.88
0.83
0.44
0.61
0.38
r ( iy I ax ) p ao r td s
recently
recognize
0.93
redmond
r iy s ax n td 1 iy
r eh k ax gd n ay z
ax r eh k ax gd n ay z
r eh k ax gd n ay z z
r eh k ax gd n ay s ax
r eh dd m ax n dd
regina
r ax jh ay n ax
0.53
r iy jh ay n ax
0.28
related
r ax jh ax n ax
r iy 1ey tf ax dd
0.10
0.75
r
r
r
r
ax 1ey dd
iy n ow
iy f r ey z
ax p ao r tf ax dd
0.23
reno
rephrase
reported
r ih v er
r ow dd
r aa ch eh s t er
0.29
0.10
republic
r iy p ao r tf ax dd
r iy p ao r df ax dd
r ax p ah bd l ax kd
r aa kd f er dd
r aa k iy
r ow m
r iy p ah bd I ax kd
r eh s td
r eh s tr r aa n td
0.42
rest
restaurant
r eh s tr r ax n td
r ax s t ax r ax n td
r eh s t er aa n td
r eh s t ax n
r ow dd
r iy k ax
r iy k ow
r iy ow
r ay z ih ng
ax r ay z ih ng
r iy aa dd
0.18
ax ) k w eh s td
r iy s t- aa r td
r eh s ( tr r It ax r )
ax j aa) n td s
r ey kd (iy ax I y ax)
v ih kd
r ow dd z
r ih ch m ax n dd
r ay td
r ay z
r ow z ax
r uw zh
r ah td 1 ax n dd
s ae kd r ax m eh nt ow
s eh dd
s ey I ax m
s ao 1z b ( er I eh r ) iy
s ae 1 v ax d ao r
s ey m
s ae n
reston
rhode
rica
rico
rio
rising
riyadh
k w ih td
k ( iy t I w iy tf ) ow
r ax
r ey
r ey
r ey
b aa td
n k ow td
n f ao 1
n z
0.11
r iy s iy v
r ax s iy v
r (iy
k w eh ( s ch I sh ch ?
) ax n z
0.23
receive
0.45
0.99
Weight
r ( iy Iax ) p ao r td
0.48
0.49
rapids
reach
Expert Baseform
PMM Baseforms
k w eh r iy
k w ih r iy
k w eh s ch ax n z
0.46
r ( aa 1 I ao 1 ) iy
r ey n jh ax z
r ax s iy v
r iy s ax n td 1 iy
r eh k ax gd n ay z
0.22
0.13
0.12
0.99
0.89
0.99
0.59
0.57
0.93
0.40
0.15
0.09
0.99
0.99
0.98
0.99
0.94
0.90
0.09
0.41
r eh dd m ax n dd
r ax ( jh
) n ax
g) ( iy I ay
r ( iy I ax ) 1 ey tf ax
dd
r iy n ow
i ( iy ax ) f r ey z
r ( iy I ax ) p ao r tf ax
dd
r ( iy
ax kd
ax ) p ah bd 1
r eh s td
r eh s ( tr r I t ax r
ax I aa ) n td
eh s t ax r
ow dd
iy k ax
iy k ow
iy ow
ay z ih ng
iy aa dd
Word
sanibel
PMM Baseforms
s ae n ax b eh I
Weight
0.83
II
s aa nt iy aa g ow
s ah nt iy aa g ow
s ae nt iy aa g ow
0.16
0.40
0.29
0.27
0.83
0.09
0.35
s ae n ax b eh
santiago
sao
s aw
S ow
sarajevo
s ae r ax y ey v ow
Expert Baseform
s ae n ax b ( ax I Ieh I
Word
Weight
0.24
r iy y ae dd
r iy ax dd
r ow ax n ow kd
r aa kd
r aa kd v ih 1
r ow m ey n y ax
r ah m ey n y ax
0.17
0.09
0.90
0.98
0.93
0.45
0.17
root
r ah m ey n iy ax
r ow m iy n y ax
r uw td
0.13
0.09
0.65
r ( uw I uh ) td
rotterdam
russia
r uh td
ax r aa tf er d ae m
r ah sh ax
0.34
0.93
0.95
r aa tf er d ae m
r ah sh ax
s
eh s
s ae g ax n aa
s ey g ax n aa
0.93
0.64
0.25
0.09
0.97
eh s
saginaw
0.49
0.15
0.14
0.97
0.61
0.38
0.85
0.10
0.51
0.42
0.99
0.64
0.18
s ax 1 iy n ax
s aa nt iy aa g ow
roanoke
rock
rockville
romania
s aw
|
PMM Baseforms
r iy ae dd
s ( aa r I ae r ) ax y ey
Expert Baseform
ow ax n ow kd
aa kd
aa kd v ih 1
ow m ey n iy ax
v ow
s eh r ax y ey v ow
s aa
sardinia
r ax y ax v ow
s eh r d iy n y ax
0.25
0.19
0.47
s aa r d iy n ( y ax
I iy
ax )
saskatoon
saturday
saudi
savannah
scandinavia
science
scores
scotland
scottsdale
scratch
sea
s aa r d iy n y ax
s aa r d iy n iy ax
s ae s k ax t uw n
0.26
0.24
0.99
s ae tf er df ey
s aw df iy
s ao df iy
s ax v ae n ax
s k- ih n d ax n ey v iy
ax
0.99
0.71
0.25
0.87
0.99
s ay ax n s
s
s
s
s
s
s
s
k- ao r z
k- ao r s
k- ao r ax s
k- aa td 1 ax n dd
k- aa td s d ey 1
k- r ae ch
r ch
s iy S iy
S y uw
s iy uw
S iy
s k- ae n d ax n ey v iy
ax
saint
s ae g ax n ao ow
s ey n td
1
0.43
0.38
0.11
0.98
0.83
0.89
0.10
0.45
0.22
0.20
0.11
0.99
0.96
0.99
0.61
0.30
s ay ax n s
salina
s ax
s eh pd t eh m b er
season
second
select
senegal
s iy z ax n
september
s eh pd t eh m b er
0.87
s ax pd t eh m b er
0.10
s er v ax s
s er v ay s
0.64
0.16
0.11
0.99
0.94
0.89
service
s
s
s
s
eh
ax
eh
eh
k ax n dd
1eh kd td
n ax g ao 1
n ax g aa 1
s s er v ax s
setting
seventeenth
seventy
s eh tf ih ng
s eh v ax n t jy n th
s eh v ax n d iy
seville
s ax v ih 1
shanghai
shining
shore
should
showers
shut
s ax v ih I1
sh ae ng hh ay
sh ay n ih ng
sh ao r
sh uh dd
sh ax dd
ch uh dd
sh aw er z
sh ah td
sh aw td
sh ah td ax
s ( ae Iaa ) s k ax t uw
n
s ae tf er df ey
s ( ao I aw ) df iy
s ax v ae n ax
s k- ao r z
s k- aa td 1 ax n dd
s k- aa td s d ey I
s k- r ae ch
santa
santo
s iy z ax n
s eh k ax n dd
s ax 1eh kd td
s eh n ax g ao 1
s er v ax s
s eh tf ih ng
s eh v ax n t iy n th
s eh v ax ( nt I n d ) iy
s ax v ih
0.82
sh
sh
sh
sh
0.87
0.22
0.21
0.15
samoa
sa.ndusky
S iy
0.86
0.12
0.91
0.99
0.66
0.20
0.11
salt
salzburg
I
1ay n ax
s ae
s ao
I ax n ax
1ay n ax
s
s
s
s
s
s
s
1td
Itd s b er gd
1z b er gd
m ow ax
b ow ax
n d ah s k iy
n d ah s k iy
ao
ao
ao
ax
ax
ae
ax
s ae nt ax
s ae nt ax
s ey nt ax
0.15
say
schedules
s ae nt ow
s ax p ao r ow
s ae r ax s ow tf ax
s ae s k ae ch ax w aa
n
s ax s k ae ch ax w aa
n
s ax s k- ae ch ax w aa
n
s ae tf ax 1 ay td
s ae tf er n
s ao s ax I iy tf ow
s aw td s ax 1 iy tf ow
s ey
s k- eh jh ax 1 z
score
s k- ow
0.38
0.33
0.27
0.93
0.99
0.92
0.99
0.48
0.30
0.12
0.99
0.93
sapporo
sarasota
saskatchewan
satellite
saturn
sausalito
s k- ao r
ae ng hh ay
ay n ih ng
ao r
uh dd
scotia
scotts
scranton
screwed
seal
sh aw er z
sh ah td
s
s
s
s
s
kkkkk-
w ao r
ow sh ax
aa td s
r ae n tq en
r uw dd
S iy 1
s y ax 1
s iy ax 1
seattle
s iy ae tf ax 1
see
S iy
0.99
0.99
0.46
s ae g ax n ao
s ey n td
s ao 1 td
s ao I td s b er gd
s ax m ow ax
s ae n d ah s k iy
s ae nt ax
s ae nt ow
s ( aa I ax ) p ao r ow
s ae r ax s ow tf ax
s ae s k ae ch ax w aa
n
0.38
0.09
0.99
0.99
0.82
0.09
0.90
0.99
s ae tf ax 1 ay td
s ae tf er n
s ao s ax 1 iy tf ow
s ey
s k- ehjh ( uh
z
s k- ao r
s
s
s
s
s
kkkkiy
ow sh ax
aa td s
r ae n tq en
r uw dd
I
s iy ae tf ax 1
S iy
1
ax 1)
Word
sicily
significant
PMM Baseforms
sh ax td
sh ah tf ax
s ih s ax 1iy
s ax gd n ih f ax kd eh
n td
s ax gd n ih f ax kd ae
n td
s ax gd n ih f ax k ax n
Weight I Expert Baseform
0.14
0.13
0.99
0.42
'"
'
s ih s ax 1 iy
s ax gd n ih f ax k ax n
td
Word
Seneca
seoul
s er b iy ax
Weight _[Expert Baseform
s eh n ax k ax
0.99
s ow 1
0.73
0.11
s er b iy ax
0.88
s er b iy ae m
0.10
set
s eh td
0.98
s eh td
seven
seventh
severe
sex
shine
shit
s eh v ax n
s eh v ax n th
0.99
0.99
0.94
0.93
0.99
0.51
0.36
0.99
0.94
0.96
0.94
0.99
0.51
0.44
0.99
0.86
0.67
0.14
0.99
0.62
0.37
0.46
0.22
0.18
0.11
0.92
s eh v ax n
s eh v ax n th
serbia
0.23
0.22
PMM Baseforms
'
s eh n ax k ax
S ow I
s ow ax 1
t ax
sioux
six
sixth
ski
sky
sleet
slow
smith
snow
snowed
snowing
snowstorm
snowy
sofia
some
s uw
s s uw
s n ow dd
s n ow ih ng
s n ow s t ao r m
s n ow iy
0.88
s n ow iy
s ow
s ow
s ow
s ow
s ah
0.33
0.30
0.19
0.14
0.70
0.29
0.96
0.84
0.62
0.20
0.10
0.99
0.95
0.94
0.95
0.49
0.27
0.20
0.56
0.18
s ow f iy ax
s ih kd s
s ih kd s th
s k- iy
s k- ay
s 1iy td
s 1iy iy td
S 1ow
s m ih th
s m ih td
s m eh th
s n ow
f iy ao
f iy ax
f iy ax b y ax
ax f y ax
m
s ah m th ih ng
s ah m w eh r
s aa r iy
s ao r iy
s aa r ay
sounds
southeast
southern
southwest
space
spanish
speak
specific
speed
spell
sports
spring
springs
squall
sri
s uw
0.09
0.96
0.96
0.89
0.94
0.86
0.13
0.99
0.55
0.31
0.12
0.99
0.99
0.99
0.97
s ax m
something
somewhere
sorry
0.84
s
s
s
s
s
s
s
s
s
aw n dd z
aw th iy s td
ah dh er n
aw th w eh s td
p- iy s
p- ey sh y ax
p- ey sh
p- ae n ax sh
p ae n ax sh
s p- ae n ih z
s p- ae n ax s
s p- iy kd
s p- ax s ih f ax kd
s p- iy dd
s p- eh 1
s p- ao r td s
s p- r ih ng
s p- r ih ng z
s k- w ao 1
s k- w aa
s r iy
sh r iy
s ih kd s
s ih kd s th
s k- iy
s k- ay
s 1 iy td
S 1 ow
s m ih th
S n ow
s n ow dd
s n ow ih ng
s n ow s t ao r m
0.47
0.72
0.21
s eh kd s
sh ay n
sh ih td
sh ax t s
short
show
shreveport
siberia
side
singapore
situation
sixteenth
sixty
skiing
sleep
sh r
sh ow
sh r iy v p ao r td
s ay b ih r iy ax
s ay dd
s ih ng gd ax p ao r
s ih ng ax p ao r
s ih ch uw ey sh ax n
s ih kd s t iy n th
s ih kd s t iy
s ih kd s t ax
s k- iy ih ng
s 1 iy pd
s 1 iy
s ah m
s ah m th ih ng
s ah m w eh r
s ( aa r I ao r ) iy
s aw n dd z
s aw th iy s td
s ah dh er n
s aw th w eh s td
s p- ey s
slovakia
s
s
s
s
smart
smog
s m aa r td
s m aa gd
s m ao gd
eh s n ow b ey s
snowfall
snowmass
snowstorms
s
s
s
s
s
so
s ow
s p- ae n ax sh
someone
s p- iy kd
sometime
s p- iy dd
s p- eh
1
s p- ao r td s
s p- r ih ng
s p- r ih ng z
s k- w ao 1
n
n
n
n
ow
ow
ow
ow
b ey s
f ao 1
m ae s
s t ao r m z
n ow ax S t- ao r m z
s ax
somalia
s p- ax s ih f ax kd
I ow v aa kd y ax
1 ow v aa kd y uw
1 ax v aa k iy ax
1 ow v aa k y ax
snowbase
0.11
0.09
0.99
0.99
0.99
0.99
0.93
0.99
0.99
0.52
s ax v ih r
soon
sound
south
southeastern
( s I sh ) r iy
southernmost
s ax m aa 1 y ax
s ax
s ah
s ah
s ah
ax s
m aa 1 iy ax
m w ah n
m w aa n
m t ay m
ah m t ay m
s uw n
s ax n
s aw n dd
s aw th
s aw th iy
s aw th iy
s aw th iy
s ah dh er
s t er n
z t er n
sh t er n
n m ow s td
0.77
0.22
0.53
0.46
0.97
0.99
0.78
0.11
0.86
0.09
0.65
0.28
0.55
s ax v ih r
s eh kd s
sh ay n
sh ih td
sh ao r td
sh ow
sh r iy v p ao r td
s ay b ih r iy ax
s ay dd
s ih ng ax p ao r
s ih ch uw ey sh ax n
s ih kd s t iy n th
s ih kd s t iy
s k- iy ih ng
s 1 iy pd
s I ow v aa k iy ax
s m aa r td
s m aa gd
s n ow b ey s
s n ow f ao 1
s n ow m ae S
s n ow s t ao r m z
s ow
s ( ow Iax ) m aa 1 (iy
ax | y ax )
s ah m w ( ah Iax) n
0.44
0.66
0.33
0.64
0.27
0.99
0.99
0.40
0.35
0.21
0.99
s ah m t ay m
s uw n
s aw n dd
s aw th
s aw th iy s t er n
s ah dh er n m ow s td
Word
stafford
stapleton
PMM Baseforms
s t- ae f er dd
s t- ey p ax 1tf ax n
s t- ey p ax 1tf s ax n
s t- aa r tf ih ng
s t- ao r tf ih ng
s t- ey td rn eh n td
s t- ey td n ax n td
s t- ae tf ax s
s t- aa kd
s t- aa kd v ih 1
s t- ao r m
s t- ao r m iy
s t- uw p ax dd
s t- uw p ih td
s t- uw p ih dd
s ax b er bd
s ah b er bd
s ah m ax r iy
s ah n
s ah n 1 ay td
s ah n iy v ey 1
Weight
0.99
0.51
0.33
0.65
0.33
0.52
0.47
0.98
0.94
0.99
0.99
0.99
0.68
0.12
0.09
0.86
0.13
1
0.99
0.96
0.98
Expert Baseform
s t- ae f er dd
s t- ey p ax 1 tf ax n
sh er
sh ao r
s er f iy ng
s er f iy ng gd
0.69
0.21
0.73
0.25
sh ( er I ao r )
sweden
sydney
syria
systems
s
s
s
s
s w iy dd en
tacoma
s
t
0.93
0.98
0.95
0.77
0.11
0.86
0.99
0.99
0.53
0.46
0.84
0.57
0.14
0.13
0.57
0.11
starting
statement
status
stock
stockville
storm
stormy
stupid
suburb
summary
sun
sunlight
sunnyvale
sure
surfing
tahoe
taiwan
talk
t
t
t
t
tallahassee
tanzania
t
t
t
t
tasmania
w iy dd en
ih dd n iy
ih r iy ax
ih s t ax rri z
s t ax m z
ax k ow rmax
aa hh ow
ay w aa n
ao kd
aa kd.
ae I ax hh ae s iy
ae n z ey n iy ax
ae n z ax n iy y ax
ae n z ax n iy ax
ae z m ey n iy ax
t
t ae z m iy n y ax
s t- aa r tf ih ng
Word
southwestern
spain
spartanburg
special
s t- ey td m ax n td
specifically
s ts ts ts ts ts t-
( ae I ey ) tf ax s
aa kd
aa kd v ih 1
ao r m
ao r m iy
uw p ax dd
s ah b er bd
s ah
s ah
s ah
s ah
m ax r iy
n
n 1 ay td
n iy v ey I
s er f ih ng
s p- ey n
0.96
0.99
square
st
stamford
start
state
states
still
stockholm
stop
storms
strasbourg
s
s
s
s
speeds
spokane
spot
springfield
sprinkles
stuttgart
t
t
t
t
suburbs
summer
sunday
sunny
supposed
k ow m ax
hh ow
w aa n
kd
Weight I Expert Baseform
s aw th w eh s t er n
0.99
s p- aa r tq en b er gd
s p- eh sh ax 1
s p- ey sh eh I
s p- ax s ih f ax k I iy
s p- ax s ih f ax kd 1 iy
s p- iy dd z
s p- ow k ae n
s p- aa td
s p- r ih ng f iy 1 dd
s p- r ih ng k ax 1 z
s p- r ih ng kd ax 1z
s k- w eh r
s ey n td
s t- ae m f er dd
s t- aa r td
s t- ey td
s t- ey td s
s t- ih 1
s t- aa kd hh ow I m
s ih dd n iy
s ih r iy ax
s ih s t ax m z
ax
aa
ay
ao
PMM Baseforms
s aw th w eh s t er n
t ae 1 ax hh ae s iy
t ae n z ax n iy ax
surf
sweater
t ae z m ey n iy ax
switzerland
t- aa k ow I m
t- aa pd
t- ao r m z
tr r ae s b er gd.
s tr r aa s b er gd
sh t uh td g aa r td
s t- uh td g aa r td
s t- ax td g aa r td
sh t uw td g aa r td
s ah b er bd z
s ah m er
s ah n d ey
s ah n iy
s ax p ow z dd
s ax p ax s td
s ax p ow s td
s er f
s w ah tf er
s w eh tf er
s w ih td s er I ax n dd
0.68
0.22
s p- ey n
s p- aa r tq en b er gd
s p- eh sh ax 1
0.82
0.17
s p- ax s ih f ax k 1iy
0.99
0.92
0.99
s p- iy dd z
0.81
0.72
0.27
s p- r ih ng f iy I dd
s p- r ih ng k ax 1 z
0.99
0.99
s k- w eh r
0.78
0.85
s t- ae m f er dd
0.99
0.98
0.98
0.58
s p- ow k ( ae
I ey
)n
s p- aa td
s ey n td
s t- aa r td
s t- ey td
s t- ey td s
s t- ih 1
s t- aa ( k I kd hh) ow
Im
0.39
0.91
0.98
0.62
s t- aa pd
s t- ao r m z
s tr r ( aa I ae ) ( z
) b ( er I ao r ) gd
s
0.17
0.48
0.18
0.09
0.09
0.99
0.90
0.98
0.99
0.69
sh t uh td g aa r td
s ah b er bd z
s ah m er
s ah n d ey
s ah n iy
s ax p ow z dd
0.15
0.15
0.91
s er f
0.51
0.48
0.84
s w eh tf er
s w ih td ( z | s
) er I ax
n dd
tehran
telephone
t ae z m ey n iy y ax
t ae z m ey i y ax
0.11
0.11
t
t
t
t
t
0.49
0.25
0.24
0.49
0.35
0.14
0.28
0.24
0.13
0.10
0.55
t
temecula
eh r aa n
ey r aa n
iy r aa n
eh 1 ax f ao n
eh I ax f ow n
eh 1 ax f ao
ax m eh kd y uw I ax
ax m eh k uw I ax
ax m eh k uw I aa
ax m eh kd y uw I aa,
eh m p ax ch er z
temperatures
t
t
t
t
t
tennessee
t eh m p r ax ch er z
t eh n ax s iy
0.43
0.75
syracuse
s w ih td s ax 1 ax n dd
s ih r ax k y uw z
0.14
0.49
system
s ih r ax k y uw s
s ih s t ax m
0.36
0.96
t
t iy
0.97
t eh 1 ax f ow n
tahiti
t ey hh iy tf iy
0.62
t
t
t
t
t
t
t
0.27
0.98
t eh m ax k uw I ax
taipei
take
talking
t eh r aa n
t eh m p ( er ax I r ax
I ax ) ch er z
t eh n ax s iy
tampa
taos
ax
ay
ey
ao
aa
ae
aa
hh iy tf iy
p ey
kd
k ih ng
k ih ng
m p ax
ow s
t aw s
t aw z
r ax k y uw ( z | s
s ih s t ax m
t iy
t ax hh iy tf iy
0.95
0.66
t ay p ey
t ey kd
t ao k ih ng
0.27
0.95
0.38
t ae m p ax
t aa ow s
0.30
0.16
Word
PMM Baseforms
term
territories
t er m
t eh r ax t ao r iy z
Weight
0.99
0.99
t er m
t eh r ax t ao r iy z
texarkana
t eh kd s er k ae n ax
0.35
t eh kd s aa r k ae n ax
t eh kd s aa r k ae n ax
0.24
t eh kd s ax k ae n ax
0.19
t ay 1 ae n dd
0.64
t ay 1 ax n dd
0.34
th ae ng kd
dh ae td
ax m
0.99
0.97
0.35
dh ax m
dh eh m
dh eh r
dh ey
th ih ng z
th er dd
th er tf iy ax th
dh ih s
dh ow z
th aw z ax n dd
th r iy
th r ax aw td
th r uw aw td
th ah n d er s t- ao r m
th er z d ey
t ax b eh td
t ax b ae td
t ay dd z
t ay m
tf ax
tf uw
t uw
t ow kd y ow
t ow I
t ax n ay td
t ax p iy k ax
t ax p iy k ae s
0.23
0.13
0.96
0.95
0.99
0.95
0.95
0.98
0.91
0.99
0.94
0.65
0.34
0.98
0.99
0.76
0.22
0.99
0.99
0.40
0.24
0.23
0.83
0.98
0.99
0.90
0.09
tornados
t ao r n ey df ow z
0.93
torrence
toulouse
t ao r ax n s
t ax 1 uw z
town
t aw n
thailand
thank
that
them
there
they
things
third
thirtieth
this
those
thousand
three
throughout
thunderstorm
thursday
tibet
tides
time
to
tokyo
toll
tonight
topeka
Expert Baseform
t ay 1 ( ae
I ax
Word
tegucigalpa
) n dd
th ae ng kd
dh ae td
dh eh m
tel
tell
temperature
1p
0.10
0.23
1p
0.19
1p
0.18
1p
0.10
1p
0.09
1p
0.09
0.99
0.99
0.60
think
thirteenth
thirty
thomas
thought
threatening
0.37
0.76
0.10
0.95
0.99
0.72
0.27
0.97
0.43
0.25
0.21
0.99
0.58
0.18
0.11
0.10
0.69
0.17
0.97
0.90
0.09
0.99
0.99
0.99
0.89
0.99
0.95
tenth
terrific
test
texas
than
thanks
the
th ah n d er s t- ao r m
th er z d ey
t ax b eh td
then
t ay dd z
t ay m
tf ( ax uw)
Weight
t ay ow z
t eh g uw s iy g aa
ax
t ax g uw s ax g ae
ao
t ey g uw s iy g aa
ax
t ax g uw s ax g aa
aa m
t ax g uw s iy g aa
aa n
t ax g uw s iy g aa
ax
t eh 1
t eh 1
t eh m p ax ch er
t eh m p r ax ch er
t eh n
ae n
t eh n th
t ax r ih f ax kd
t eh s td
t ax s td
t eh kd s ax s
dh ae n
dh en
dh ax n
th ae ng kd s
dh ax
dh ah
dh iy
dh ih
dh eh n
dh ax n
dh iy z
th ih ng
th
th ih ng kd
th er t iy n th
th er tf iy
t aa m ax s
th ao td
th r eh tq en ih ng
ten
dh eh r
dh ey
th ih ng z
th er dd
th er tf iy ax th
dh ih s
dh ow z
th aw z ax n dd
th r iy
th r uw aw td
PMM Baseforms
these
thing
Expert Baseform
t ax g uw s (
aa 1 p ax
ly| ax) g
t eh 1
t eh 1
t eh m p ( er ax
I ax ) ch er
r ax
t eh n
t eh n th
t ax r ih f ax kd
t eh s td
t eh kd s ax s
dh ae n
th ae ng kd s
dh ( ax I ah I ih
I iy)
dh eh n
dh iy z
th ih ng
through
th r uw
0.92
0.99
0.95
t ( ao r I er ) n ey df ow
z
t ao r ax n s
t uw I uw z
th ih ng kd
th er t iy n th
th er tf iy
t aa m ax s
th ao td
th r eh ( tf ax n I tq en
I td n ) ih ng
th r uw
thunder
thunderstorms
0.99
0.99
th ah n d er
th ah n d er s t- ao r m
0.99
t aw n
tianjin
th ah n d er
th ah n d er s t- ao r m
z
t ih ae n jh ih n
t ow k ( iy
y ) ow
t ow 1
t ax n ay td
t ax p iy k ax
z
0.42
t ( iy
I y)
(eh
I aa
)n
jh ih n
traffic
travelling
trenton
trip
tropical
try
tucson
tulsa
tunisia
tuscaloosa
twelve
twenty
tr r ae f ax kd
tr r ae v ax I ih ng
tr r eh n tq en
tr r ih pd
tr r aa p ax k ax 1
tr r ay
t uw s aa n
t ah 1 s ax
t ax n iy zh ax
0.94
0.99
0.97
0.99
0.94
0.96
0.94
0.95
0.62
t uw n iy zh ax
t ah s k ax I uw s ax
t ah s k ax I uh z ax
t w eh 1 v
ey t w eh 1 v
t w eh nt iy
0.28
0.66
0.20
0.80
0.19
0.98
tr r ae f ax kd
tr r ae v ax 1 ih ng
tr r eh n tq en
tr r ih pd
tr r aa p ax k ax 1
tr r ay
t uw s aa n
t ah 1 s ax
t ( ax I uw ) n iy zh
ax I iy ax )
tide
timbuktu
times
today
toledo
t ah s k ax 1 uw s ax
t w eh 1 v
t w eh nt iy
tomorrow
too
tornado
toronto
t
t
t
t
t
t
t
t
t
ih njh ih n
ih ih n jh ih n
ay dd
ih m b ax kd t uw
ih m b ah kd t uw
ax m b ax kd t uw
ay m z
ax df ey
ax d ey
t ax 1 iy df ow
tf ax 1 iy df ow
t ax m aa r ow
t uw
t ao r n ey df ow
t ax r aa nt ow
0.38
0.16
0.99
0.39
0.21
0.13
0.98
0.84
0.14
0.87
0.10
0.99
0.95
0.90
0.98
t ay dd
t ih m b ax kd t uw
t ay m z
t ax df ey
t ax 1 iy df ow
t
t
t
t
ax m aa r ow
uw
( ao r I er ) n ey df ow
ax r aa nt ow
Word
two
typhoons
PMM Baseforns
Weight
Expert Baseform
t uw
0.99
t uw
t ay f ax 1 ax n z
0.27
t ay f uw n z
0.21
0.16
0.14
0.12
0.96
0.99
uruguay
t ay f ao n z
t ay f uw m
t ay f uw n s ax z
t ay f uw n z
y uw
y uw k r ey n
ah n d er s t- ae n dd
y uw n ax v er s
ah pd
ah p er
ow p er
y ao r ax g w ay
use
usually
y er ax g w ey
y uw z
y uw zh ax I iy
0.49
0.90
0.44
u
ukraine
understand
universe
up
upper
utica
v
vail
valley
vegas
venezuela
vermont
vero
vicinity
vienna
view
virgin
visibility
voice
w
wait
wales
walnut
wanted
warmer
warning
warsaw
y
y
y
y
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
uw zh uw ax 1iy
uw zh 1iy
uw zh ax 1ay
uw tf ax k ax
iy
y ax
ey 1
ae 1iy
ey g ax s
eh n ax z w ey I ax
eh n ax z w eh 1ax
er m aa n td
ih r ow
y ih r ow
ax s ih n ax tf iy
iy eh n ax
y uw
er jh ax n
ih z ax b ih I ax tf iy
0.81
0.99
0.81
0.88
0.11
0.50
0.22
0.16
0.15
0.99
0.86
0.12
0.89
0.97
0.96
0.53
0.42
0.97
0.69
0.29
0.97
y uw
y uw k r ey n
ah n d er s t- ae n dd
y uw n ax v er s
ah pd
ah p er
y uw ( s Iz )
y uw zh ( uw ax I | ax
I w ax 1 ) iy
1
y uw tf ax k ax
v iy
v ey
v ae
v ey
v eh
1
1iy
g ax s
n ax z w ey I ax
united
until
update
urbana
us
v er m aa n td
v ( ih I eh ) r ow
0.99
0.59
v oy s
d ah b ( ax 11 ax ? ) y
( uw I ax)
d ah b y uw
w ey td
w ah iy
w ey
w ey 1
w ah tf ax I z
w ey I ax s
w ey 1 z
w ao 1 n ah d
w aa n ax td
w ao 1 n ax
w ao 1 m ax td
w ao 1 n ax td
w aa nt ax dd
w ah tf ax dd
0.38
0.73
0.11
w ey td
w ao r m er
w ao r n ih ng
0.80
w ao r s ao
turkey
twelfth
twentieth
twin
typhoon
typically
uganda
umbrella
v ih z ax b ih
0.10
0.29
0.25
0.23
0.21
0.25
0.21
t aw n z
tr r ae v ax 1
tr r ae v er s
tr r ih n ax d ae dd
tr r ih p ax 1w iy
tr r oy
ch er oy
tr r ay ih ng
tr r ay
t uw z d ey
t uw n ax s
t uw n ih s
y ( er ao r ) ax g w (
ey ay )
0.99
d ah b ax 1y uw
towns
travel
traverse
trinidad
tripoli
troy
tuesday
tunis
0.98
v oy 6
PMM Baseforms
t ow tf ax 1
trying
v ax s ih n ax tf iy
v iy eh n ax
v y uw
v er jh ax n
0.87
0.99
Word
total
towards
1ax
tf iy
useful
utah
uzbekistan
vacation
vallarta
vancouver
w ey 1 z
velocity
w ao 1 n ah td
venice
vernal
0.17
0.16
0.13
0.83
very
victoria
w aa nt ax dd
vietnam
0.09
0.99
0.71
w ao r m er
w ao r n ih ng
w ao r s ao
vilnius
t ao r dd s iy
Weight T Expert Baseform
0.96
t ow tf ax I
t ( ao r [ ax w ao r I w
0.99
ao r ) dd z
t aw n z
0.98
0.96
tr r ae v ax 1
tr r ( ae I ax ) v er s
0.95
tr r ih n ax d ae dd
0.99
tr r ih p ax I iy
0.99
tr r oy
0.82
0.15
tr r ay ih ng
0.83
0.13
0.95
0.54
t uw z d ey
t uw n ax s
0.33
ch uw n ih s
t er k iy
t w eh I f th
0.12
0.99
0.75
t er k iy
t w eh 1f th
t w eh 1 th
t w eh nt iy ax th
t w ih n
t ay f uw n
t ih p ax k ax 1 iy
y uw g aa n d ax
ax m b r eh Iax
ah m bd r eh 1 ax
ax m bd r eh 1 ax
ah m b r eh I ax
y uw n ay tf ax dd
ax n t ih 1
ah pd d ey td
er b ae n ax
ah s
0.16
0.99
0.99
0.99
0.99
0.99
0.36
t w eh nt iy ax th
t w ih n
t ay f uw n
t ih p ax k ( ax 1I ) iy
y uw g aa n d ax
( ax I ah ) m b r eh 1 ax
ax s
0.38
0.98
uw s f ax 1
y uw t aa
y uw z b eh k ax s t ae
n
v ey k ey sh ax n
v ey k y ey sh ax n
v
v
v
v
v
v
v
v
v
v
v
v
v
v
v
ay aa r tf ax
ay y aa r tf ax
ey y aa r tf ax
ae n k uw v er
ae ng k uw v er
ax 1aa s ax tf iy
ax 1ao s ax tf iy
eh n ax s
er n ax 1
er n ow I
er t n ao I
eh r iy
ax kd t ao r iy ax
ih kd t ao r iy ax
iy eh td n aa m
v iy eh td n ae m
v ih I ax n ax s
v ih n iy ax s
0.21
0.10
0.10
0.99
0.99
0.99
0.99
0.54
y uw n ay tf ax dd
ax n t ih 1
ah pd d ey td
er b ae n ax
ah s
0.97
y uw s f ax 1
y uw t ( ao I aa)
uh z b eh k ax s t aa n
0.89
v ey k ey sh ax n
0.78
0.10
0.60
v ay ( aa I eh ) r tf ax
0.25
0.12
0.73
0.24
0.82
0.17
0.88
0.69
v ae n k uw v er
v ax
1 aa s
ax tf iy
v eh n ( ax I iy ) s
v er n ax 1
0.19
0.10
0.99
0.86
v eh r iy
v ax kd t ao r iy ax
0.10
0.76
0.17
0.48
0.44
v iy ( ax | eh ) td n ( ae
l aa) m
v ih 1 n iy ax s
Word
PMM Baseforms
[ Expert Baseform
w ah z
w uh z
Weight
0.59
0.36
watch
w aa ch
0.93
w aa ch
water
w aa tf er
0.63
w ( aa
df) er
waterloo
w ao tf er
w aa df er
w aa tf er 1 uw
0.19
0.12
0.78
waterville
w ao tf er 1 uw
w aa df er v ih 1
0.21
0.78
wellington
w
w
w
w
w
w
w
w
w
w
w
w
w
aa tf er v ih 1
ey v z
ey n
eh r
ih r er
eh dh er
iy kd
iy kd 1 iy
eh 1 k ax m
ax 1 k ax m
ax 1 k ow m
ax k ah m
eh 1ih ng t ax n
0.21
0.98
0.91
0.89
0.10
0.99
0.97
0.99
0.50
0.18
0.15
0.10
0.88
west
westlake
v eh 1 ih ng t ax n
w eh s td
w eh s I ey kd
0.11
0.99
0.58
w eh s td 1 ey kd
w ah td
w eh n
w eh dh er
w ay td
hh uw
w ay
w ay 1dd w uh dd
w aa 1 dd w uh dd
w ih 1 y ax m z b er gd
w ih 1 iy ax m z b er gd
w ax 1m ih ng t ax n
w ih 1 m ih ng t ax n
w ih n
w ih n d iy ax s td
w ih n d iy z s td
w ih n z er
w ax n z er
w iy n z er
w ih n z
w ih n z ao r
w ih n s er f ih ng
w ih n dd s er f ih ng
w ih n ax p ey gd
w ih n ax p eh gd
w ih n ax p ey kd
w ih nt er
w ih sh
w ih th ih n
0.41
0.99
0.93
0.99
0.99
0.86
0.81
0.77
0.21
0.76
0.12
0.67
0.30
0.98
0.86
0.12
0.36
0.20
0.19
0.10
0.09
0.53
0.46
0.33
0.33
0.20
0.97
0.99
0.38
w ax th ih n
w ax th ax n
0.19
0.18
was
waves
wayne
wear
weather
week
weekly
welcome
what
when
whether
white
who
why
wildwood
williamsburg
wilmington
win
windiest
windsor
windsurfing
winnipeg
winter
wish
within
'
'
'"
w ah z
ao uh ) (tf
Word
virginia
vladivostok
PMM Baseforms
Weight
Expert Baseform
v er jh ih n y ax
v 1 aa df ax v aa s t aa
kd
v 1 ae df ax v aa s t aa
kd
v oy jh er
0.99
0.32
v er jh ih n y ax
v 1 aa df ax v aa s t aa
kd
0.59
v ( oy ax ow y ax J ao
y ax | oy) jh er
v oy ax jh er
w ey k ow
w ah ey k ow
0.33
0.70
0.19
w ey k ow
w ey k ow ax 1
w ey kd.
0.09
0.99
waterbury
w
w
w
w
w
w
w
w
w
w
w
w
w
0.90
0.60
0.16
0.12
0.99
0.99
0.99
0.46
0.41
0.98
0.82
0.09
0.39
watertown
w aa tf er b er iy
w aa df er b er iy
w aa df er t aw n
0.21
0.09
0.35
w ah tf er t aw n
w aa tf er t aw n
w ao tf er t aw n
w ey v
w ey
w ey ey
w ah oy
w ah ey
w iy
w eh r ih ng
w ih r ey ih ng
er ih ng
w eh n z d ey
w iy kd eh n dd
w iy kd s
w eh 1
w ax 1
w er
w eh r
w eh s t er n
w eh td
w iy 1 ih ng
w eh r
w ih ch
w ay td hh ao r s
w ay td hh ao r s s
hh ow l
w ih ch ax t ao
w ih ch ax t aa
0.22
0.15
0.09
0.92
0.52
0.20
0.16
0.09
0.97
0.68
0.10
0.10
0.93
0.97
0.96
0.66
0.29
0.70
0.22
0.94
0.99
0.99
0.99
0.95
0.66
0.12
0.99
0.59
0.28
w ax 1
w ih 1 y ax m z p ao r
td
0.92
0.72
voyager
waco
w ( aa ao uh
df) er 1 uw
)
w ( aa ao uh)
df) er v ih I
(tf
(tf
walla
want
w ey v z
w ey n
w eh r
w
w
w
w
wake
warm
warmest
warnings
warwick
eh dh er
iy kd
iy kd 1 iy
eh 1 k ax m
washington
watches
w eh 1 ih ng t ax n
w eh s td
w eh s ( td
1 | 1 ) ey
kd
w ah td
w eh n
w eh dh er
w ay td
hh uw
w ay
w ay I dd w uh dd
w ih
1y ax
wave
way
m z b er gd
we
wearing
w ih 1 m ih ng t ax n
wednesday
weekend
weeks
well
w ih n
w ih n d iy ax s td
w ih n z er
were
w ih n dd s er f ih ng
w ih n ax p eh gd
w
w
w
ih
ih nt er
ih sh
( ax I ih) (th
n
western
wet
wheeling
where
which
whitehorse
whole
wichita
dh)
will
williamsport
'
aa
aa
ah
aa
ao
ao
ao
ao
ao
aa
aa
aa
aa
I ax
n td
n
rm
r m ax s td
r n ih ng z
r w eh kd
r w ih kd
sh ih ng t ax n
ch ax z
ch ax z s
tf er b eh r iy
0.29
w ey kd
w aa I ax
w aa n td
w
w
w
w
ao
ao
ao
ao
r
r
r
r
m
m ax s td
n ih ng z
w ax kd
w aa sh ih ng t ax n
w aa ch ax z
w (aa I ao I uh
df) er b er iy
) ( tf
w (aa
ao I uh
df) er t aw n
) ( tf
w ey v
w ey
w iy
w eh r ih ng
w
w
w
w
eh n z d ey
iy k eh n dd
iy kd s
eh I
w er
w
w
w
w
w
w
eh s t er n
eh td
iy 1 ih ng
eh r
ih ch
ay td hh ao r s
hh ow l
w ih ch ax t ao
w ( ax I ih ) I
w ih 1 y ax m z p ao r
td
Word
PMM Baseforms
w ax td ih p
Weight_[ Expert Baseform
0.11
wonderful
w ah n d er f ax 1
w aa n d er f ax 1
w ah dd I ax n dd
0.55
woodland
worcester
world
would
wrong
xian
yakima
year
years
yellowstone
yesterday
york
yosemite
your
yukon
zaire
zealand
zimbabwe
w uh dd 1 ax n dd
w uh s t er
w ah s t er
w ao r I dd
w er Idd
w uh dd
r ao ng
r aa ng
sh iy aa n
ch y aa n
sh ae n
sh iy ae n
y ae k ax m aa
y ih r
y ih r z
y ih r ax z
y er z
y eh 1ow s t- ow n
y eh s t er d ey
y ao r kd
y ow s eh m ax tf iy
y ow s ax m ax tf iy
y er
y ao r
y uw k aa n
z ay ih r
z ay y er
z ay iy er
z ay er
z iy I ax n dd
z ax m b aa bd w ey
z ax m b aa bd w eh ey
z ih m b aa bd w ey
Word
PMM Baseforms
w ih I iy ax m z p ao r
td
w ih m b ax 1df ax n
w ih m b ax 1df en
w ih n dd
Weight
0.18
Expert Baseform
0.85
0.12
w ih m b ax 1 df ax n
0.98
w ih n dd
winds
windsurf
windy
winston
w ih n dd z
w ih n dd s er f
w ih n d iy
0.93
0.99
0.99
0.61
0.30
w
w
w
w
wisconsin
w ax s k- aa n s ax n
w ax s k aa n s ax n
w ih th
w ax th
w ih dh
w ah n d er
w ah n d er ax
w ah n d er ih ng
w uh dd z
w ao r kd
w er kd
w ao r th
w er th
w ah aw
w aa ow
w ah ow
w ay ow m ih ng
I ay
w ay
y eh
y ae
y ey ax
y ih r 1iy
y eh 1ow n ay f
y eh s
y ow k ax hh aa m ax
y ow k ow hh aa m ax
y ow k ow hh aa m aa
y ao r kd t av n
y uv
y uw g ow s 1aa v iy ax
y uw m ay
y uw m aa
iy y uw m ax
y uw m ax
z ae m b iy ax
z iy ae m b iy ax
aa m b iy ax
z ih r ow
z iy ow
s ih r ow
z er ax kd
z ao r ax kd
0.74
w ax s k aa n s ax n
w ah n d er f ax I
wimbledon
w uh dd 1 ( ax I ae ) n
dd
wind
0.44
0.51
0.42
0.66
0.30
w uh s t er
0.59
w er 1 dd
0.24
0.98
0.78
w ( uh I ax ) dd
r ao ng
0.19
0.45
0.18
with
sh ( ax y I iy ) aa n
0.11
0.09
0.82
0.96
0.43
wonder
y ae k ax m ax
y ih r
y ih r z
0.38
0.15
0.91
0.95
0.99
0.52
0.19
0.79
0.19
0.92
0.31
0.29
0.23
0.11
0.97
0.52
0.31
0.11
wondering
woods
work
worth
y
y
y
y
eh I ow s t- ow n
eh s t er d ey
ao r kd
ow s eh m ax tf iy
( y uw r I y ao r y er)
y uw k aa n
z ay ( ih r I eh r
z iy I ax n dd
z ax m b aa bd w ey
wow
wyoming
y
yeah
yearly
yellowknife
yes
yokohama
yorktown
you
yugoslavia
yuma
zambia
zero
zurich
w ih n s t- ax n
w ih n s t ax n
ih
ih
ih
ih
n
n
n
n
dd z
dd s er f
d iy
s t- ax n
0.23
0.38
0.24
w ( ax I ih ) (th I dh)
0.23
0.72
0.27
w ah n d er
0.96
0.99
0.65
0.29
0.54
0.45
0.32
0.23
w ah n d er ih ng
w uh dd z
w er kd
w er th
w aw
0.21
0.95
0.68
0.31
0.56
0.20
w ay ow m ih ng
w ay
y ( ae I eh I ey ax)
0.10
0.99
0.99
0.99
0.48
0.24
0.19
0.99
0.98
0.96
0.33
0.33
0.16
0.16
0.68
0.17
0.14
0.35
0.33
0.30
0.66
0.15
y
y
y
y
ih r Iiy
eh I ow n ay f
eh s
ow k ax hh aa m ax
y
y
y
y
ao r kd t aw n
( uw I ax )
uw g ow s 1aa v iy ax
uw m ax
z ae m b iy ax
z ih r ow
z ( er Iao r ) ax kd
Bibliography
[1] 6.345
Automatic
Speech
Recognition,
MIT.
available
from
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/
6-345-automatic-speech-recognition-spring-2003/.
[2] Phonebook, NYNEX isolated words. available from http: //www. ldc upenn. edu.
[3] PronLex, COMLEX English
http://www.ldc.upenn.edu.
pronunciation
dictionary.
available
from
[4] L. R. Bahl, S. Das, P. V. deSouza, M. Epstein, R. L. Mercer, B. Merialdo, D. Nahamoo,
M. A. Picheny, and J. Powell. Automatic phonetic baseform determination. In Proc.
ICASSP, Washington, DC, USA, 1991.
[5] J.R Bellegarda. Unsupervised, language-independant grapheme-to-phoneme conversion
by latent analogy. In Speech Comm, 2005.
[6] Maximilian Bisani and Hermann Ney. Open vocabulary speech recognition with flat
hybrid models. In Proc. Europeen Conf. on Speech Communication and Technology,
2003.
[7] Maximilian Bisani and Hermann Ney. Joint-sequence models for grapheme-to-phoneme
conversion. Speech Commun., 50(5):434-451, 2008.
[8] Hung-An Chang and James R. Glass. A back-off discriminative acoustic model for
automatic speech recognition. In Proc. INTERSPEECH, 2009.
[9] Stanley F. Chen. Conditional and joint models for grapheme-to-phoneme conversion. In
Proc. European Conf. on Speech Communication and Technology, 2003.
[10] Ghinwa Choueiter. Linguistically-Motivated Sub-word Modeling with Applications to
Speech Recognition. PhD thesis, MIT, 2009.
[11] Ghinwa F. Choueiter, Mesrob I. Ohannessian, Stephanie Seneff, and James R. Glass. A
turbo-style algorithm for lexical baseforms estimation. In Proc. ICASSP, 2008.
[12] Michael Collins. Discriminative training methods for hidden markov models: Theory
and experiments with perceptron algorithms. In Proc. EMNLP, 2002.
[13] V. Zue et al. Jupiter: A telephone-based conversational interface for weather information. IEEE Transactions on Speech and Audio Processing, 8(1):85-96, 2000.
[14] K. Macherey F. Wessel, R. Schluter and H. Ney. Confidence measures for large vocabulary continuous speech recognition. IEEE transactions on Speech and Audio Processing,
2001.
[15] James Glass. A probabilistic framework for segment-based speech recognition. Computer
Speech and Language, 17:137-152, 2003.
[16] A. Gruenstein, I. McGraw, and I. Badr. The wami tollkit for developing, deploying, and
evaluating web-accessible multimodal interfaces. In Proc. ICMI, 2008.
[17] J. Hakkinen, J. Suontausta, S. Riis, and K.J Jensen. Assessing test-to-phoneme mapping
strategies in speaker independant isolated word recognition. In Speech Comm, 2003.
[18] Timothy Hazen, Lee Hetherington, Han Shu, and Karen Livescu. Pronunciation modeling using a finite-state transducer representation. Speech Communication, 46(2):189-201,
2005.
[19] I. Lee Hetherington. An efficient implementation of phonological rules using finite-state
transducers. In Proc. EuroSpeech, 2001.
[20] Lee Hetherington. The mit finite-state transducer toolkit for speech and language processing. In Proc. ICSLP, 2004.
[21] Sittichai Jiampojamarn and Grzegorz Kondrak. Online discriminative training for
grapheme-to-phoneme conversion. In In Proc. INTERSPEECH, 2009.
[22] R.M Kaplan and M. Kay.
put.Linguist, 1994.
Regular models of phonological rule systems.
In Com-
[23] Antoine Laurent, Paul Deleglise, and Sylvain Meigner. Grapheme to phoneme conversion
using an smt system. In In Proc. INTERSPEECH,2009.
[24] Xiao Li, Asela Guanawardana, and Alex Acero. Adapting grapheme-to-phoneme conversion for name recognition,. In Proc. ASRU, 2007.
[25] et al. M. Riley. Stochastic pronunciation modelling form hand-labelled phonetic corpora,
1999.
[26] D. MacAllester, L. Gillick, F. Scattone, and M. Newman. Fabricating conversational
speech data with acoustic models: A program to examine model-data mismatch. In
Proc. Spoken Language Processing, 1998.
[27] Benoit Maison. Automatic baseform generation from acoustic data. In Proc. Eurospeech
2003, 2003.
[28] Y. Marchand and R.I Damper. Multistrategy appraoch to improving pronunciation by
analogy. In Comput. Linguist., 2000.
[29] Ian McGraw, Chia-ying Lee, Lee Hetherington, Stephanie Seneff, and James R. Glass.
Collecting voices from the cloud. In Proc. LREC, 2010.
100
[30] V. Pagel, K. Lenzo, and A.W Black. Letter-to-sound rules for accented lexicon compression. In In Proc International Conference on Spoken Language Processing, 1998.
[31] A. Park, T.J Hazen, and J. Glass. Automatic processing of audio lectures for infonation
retrieval: vocabulary selection and language modeling. In Proc. ICASSP, 2005.
[32] Brian Roark, Murat Saraclar, and Michael Collins. Corrective language modeling for
large vocabulary asr with the perce ptron algorithm. In Proc. ICASSP, 2004.
[33] Stephanie Seneff. Reversible sound-to-letter/letter-to-sound modeling based on syllable
structure. In Proc. HLT-NAACL, 2007.
[34] Tilo Sloboda and Alex Waibel. Dictionary learning for spontaneous speech recognition.
In Proc. ICSLP, Philadelphia, PA, USA, 1996.
[35] Oriol Vinyals, Li Deng, Dong Yu, and Alex Acero. Discriminative pronunciation learning
using phoentic decoder and minimim-classification-error criterion. In Proc. ICASSP,
2009.
[36] Stanley Wang. Using graphone models in automatic speech recognition. Master's thesis,
MIT, 2009.
101