41
Representation of Consonants in the Peripheral Auditory
System: A Modeling Study of the Correspondence between
Response Properties and Phonetic Features
Richard Scott Goldhor
Technical Report 505
February 1985
d at
j
;
····
ore0
Massachusetts Institute of Technology
Research Laboratory of Electronics
Cambridge, Massachusetts 02139
/jy
I
Representation of Consonants in the Peripheral Auditory
System: A Modeling Study of the Correspondence between
Response Properties and Phonetic Features
Richard Scott Goldhor
Technical Report 505
February 1985
Massachusetts Institute of Technology
Research Laboratory of Electronics
Cambridge, Massachusetts 02139
This work has been supported in part by the National Science Foundation Grant MlCS 81-12899
II_
_II
_ _
_
t
I·I-
2
REPRESENTATION OF CONSONANTS IN THE PERIPHERAL AUDITORY SYSTEM:
A MODELING STUDY OF THE CORRESPONDENCE BETWEEN
RESPONSE PROPERTIES AND PHONETIC FEATURES
by
RICHARD S. GOLDHOR
Submitted to the Department of Electrical Engineering and Computer Science on February 1, 1985 in partial fulfillment of
the requirements for the Degree of Doctor of Philosophy in
Bioelectrical Engineering.
ABSTRACT
This thesis describes a model of the peripheral auditory
system, and discusses certain properties of its response to
speech signals.
The work was motivated by a theory regarding
the role of the peripheral auditory system in the perception
of speech.
The theory is that the peripheral auditory system
transforms speech signals into neural firing patterns with
properties which correspond to the phonetic content of the
signal.
Whereas the phonetic features of speech are encoded
in the acoustic waveform in complex and often undecipherable
ways, this theory suggests that the representation of speech
in the discharge patterns of the auditory nerve correspond in
a more direct and straightforward manner to those underlying
phonetic features.
Two testable hypotheses are implied by
this correspondence theory.
The
first
is that cochlear
transformations to speech signals result in auditory neural
firing patterns whose apparent properties are significantly
different from the apparent properties of the acoustic signal.
The second
is that particular properties
of the neural
response to speech reflect particular phonetic features of the
stimulus.
To test the correspondence theory a computer model of the
peripheral auditory system was constructed.
The model consists of three stages, each of which effects a transformation
on the acoustic input signal.
The spectral analysis stage
--
"" --li·I
" I*Lb-~--------
ABSTRACT
3
consists of a bank of filters whose characteristics reflect
The transducthe frequency response of the basilar membrane.
signal
tion stage models the cochlea's transformation of
rate.
The
adaptation
neural
firing
intensity to expected
stage produces a temporal transformation that mimics neural
The output of the model
adaptation to sustained stimulation.
is the expected firing rate of a representative set of auditory fibers in response to the input signal.
The hypotheses regarding auditory neural response are
A number
tested by studying the model's response to signals.
of simple signals exhibiting speech-like spectral structure,
onset and offset characteristics, and durational properties,
are used to demonstrate the model's behavior and to test the
Then two series of speech
validity of the first hypothesis.
stimuli, one of which spans the perceptual boundary between
stop consonants and glides, and the other of which spans the
boundary between voiced and unvoiced intervocalic stop consonants, are used to test the second hypothesis.
The results tend to support the correspondence theory of
They demonstrate that acoustic
peripheral auditory function.
patterns of speech-like signals are transformed by the peripheral auditory model into auditory patterns with quite difThey also show that certain perceptual
ferent properties.
judgements of the tested speech signals by human subjects conmodel's
the
of
properties
with measurable
form closely
These measurable properties correspond to changes
response.
in the overall expected firing rate in the mammalian auditory
nerve in response to the speech signals.
Thesis Supervisor: Kenneth N. Stevens
Title: Lebel Professor of Electrical and Bioengineering
4
ACKNOWLEDGEMENT
It is both pleasant and appropriate that
this doctoral dissertation should end, and its
with an acknowledgement of the many people
I did not write this
encompassed its creation.
the writing of
reading begin,
whose support
thesis alone.
I acknowledge with deep gratitude the support, patience,
wisdom, and encouragement of Prof. Kenneth Stevens, my thesis
advisor. His support made it possible to begin this work, his
faith sustained it through the difficult middle stages, and
whatever is best and most true in the final result reflects
His extraordinary qualities as a
his insight and wisdom.
speech scientist, teacher, and advisor made my doctoral studies a unique and wonderful experience.
Both of my readers contributed in important ways to my
Prof. Campbell Searle nurtured my
education and research.
enthusiasm,
early interest in auditory modeling with his
He was always available
informed opinions, and good' ideas.
Prof.
and interested when I had a new wild idea to expound.
Thomas Weiss taught me much of what I know about the auditory
The precision, clarity, and thoroughness of his
system.
teaching, research, and writing is an inspiring example.
I thank Christine Shadle and Stephanie Seneff, my fellow
graduate students, for their support and encourgement. Among
other things, each semester on the day we had to petition to
be removed from the degree list they were kind enough to drown
their sorrows with me over lunch.
I thank Dr. Dennis Klatt for his advice on speech and
I thank Marie Southwick and Keith
signal processing issues.
North, without whom life as we know it in the Speech Communication Group would be impossible, for their efficient and wilI thank Drs. Edith Maxwell, Karen
ling help, and good cheer.
Landahl, Patti Price, and Helen Simon for their kind permission to make use of the speech tokens and perceptual data disThanks to Dr. Price also for her
cussed in Chapter Three.
help wrestling alligators.
-
7
1-
.·
----
-.
-.1-------·-·----·l··-·r·------
------
II
Acknowledgement
5
I thank Dr. Francis Ganong, Ray Kurzweil, and my many
friends and colleagues at Kurzweil Applied Intelligence, for
encouraging my studies and for believing for so long that I
really would be done in another three months.
I thank my parents for instilling in me, by their
ple, an appreciation of the importance and honor of
scholarly work.
examdoing
I thank my wife, Pamela, for her unstinting love and support during the long days and nights of my graduate education.
That that work has been successfully completed is due in very
large measure to her offering of time and attention to me and
to our children, at some considerable cost to her own scholarOur children Lucy, Robbie, and Andrew have
ly activities.
I thank
lived with this thing called a thesis for many years.
them for their patience, and for celebrating its completion
with us.
This work was partially supported by an NSF grant, and by
a C.J. Lebel Fellowship.
ad majorem Dei gloriam
t
B
----
--
6
BIOGRAPHICAL NOTE
Richard
Scott
Goldhor
Champaign, Illinois.
of the University of
was
born
on
March
26,
1951,
in
He attended the University High School
Illinois, and received his undergraduate
and graduate education at MIT.
He earned a
and a B.S. in Electrical Engineering in 1973,
B.S. in physics
and an M.S. and
E.E. in Electrical Engineering and Computer Science
While at MIT he served as a member of the Medical
in 1976.
Advisory
Board and the Committee on Educational Policy.
From 1978 to 1983 he was the senior partner in a consulting firm, doing software development for speech synthesis and
recognition systems.
During 1980 and 1981 he was a Research
Associate at MIT's Center for Policy Alternatives, where he
studied and wrote about technology transfer between universities and industry.
He is currently the Assistant Director of
Research at Kurzweil Applied Intelligence, Inc.
Mr. Goldhor is a member of Sigma Xi
sional societies.
He holds one patent.
and numerous
profes-
PUBLICATIONS
1.
1983,
"University-to-Industry Technology Transfer: A Case
Study,"
12(3).
2.
1983,
Goldhor,
R.S.
and
R.T.
Lund,
Research
"A Speech Signal Processing System Based on a Peri-
pheral Auditory Model,"
Proceedings of the
International Conference on Acoustics, Speech,
Processing,
3.
1982,
1980,
1983
IEEE
and Signal
Boston.
"Bringing up UNIX--One User's Experience,"
Software List,
4.
Policy,
1(4);
"University
Transfer,"
Boston.
The UNIX
January-March 1982.
to
Industry
Advanced
Technology
IEEE Engineering Management Conference,
Biographical Note
7
5.
1976, "Bound and Quasibound States of Alkali-Rare Gas
Molecules," GolPhor and Pritchard, The Journal of Chemical Physics, 64(3).
6.
1972, "A Scanned Infrared Light Beam Touch Entry System,"
Ebeling, Goldhor, and Johnson, Proceedings of the International
Symposium
of
the
Society
for
Information
Display, San Fransisco.
_
_
1___
___
PC
8
TABLE OF CONTENTS
........................................
2
.........................................
4
Biographical Note
......................................
6
Table of Contents
.......................................
8
.........................................
14
Abstract
Acknowledgement
List of Figures
CHAPTER 1:
INTRODUCTION: PERIPHERAL AUDITORY RESPONSE AND
THE ANALYSIS OF SPEECH
..
1.1
MOTIVATION AND OVERVIEW
1.2
THE ROLE OF PERIPHERAL AUDI TORY MODELS IN
SPEECH ANALYSIS
.................................
27
1.3
THE NATURE OF PERIPHERAL AUDITORY RESPONSE
32
1.4
PERIPHERAL AUDITORY RESPONSE AND THE ISSUE OF
INVARIANCE AND VARIABILITY IN SPEECH
......
TORY
ODEL
21
IN
......
39
...............
43
CONSIDERATIONS IN CONSTRUCTING AND USING THE
PERIPHERAL AUDITORY MODEL
.......................
46
IMPLEMENTATION
47
FIGURES AND FIGURE CAPTIONS FOR CHAPTER 1
CHAPTER 2: PAM: A PERIPHERAL AUDITORY MODEL
2.1
2.2
..................................
II
9
Table of Contents
2.3
THE PREPROCESSOR
............
47
2.3.1
Signal Acquisition
..........................
47
2.3.2
Signal Preemphasis
..........................
48
2.4
THE PERIPHERAL AUDITORY MODEL
2.4.1
The Spectral Analysis Stage
2.4.1.1
...................
49
.................
50
Characteristics of the Frequency
Transformation
.......................
53
....
57
......................
58
Implementation Issues
2.4.1.3
Examples of The Filter Bank Response
The Transduction Stage
2.4.2.1
50
...................
2.4.1.2
2.4.2
Characteristics of the Amplitude
Transformation
..........................
59
.....
61
......
65
Selecting Values for the Transduction
Parameters
..............................
71
2.4.2.2
Constructing the Transduction Model
2.4.2.3
Analysis of the Transduction Model
2.4.2.4
...................
72
2.4.2.5
Implementation Issues
2.4.2.6
Examples of the Transduction Stage
..................... ...........
Response
73
........................
74
Characteristics of the Temporal
Transformation
..........................
74
2.4.3
The Adaptation Stage
2.4.3.1
_
....................
_I
_I·__
_
1_
Table of Contents
2.4.3.2
The single time-constant model
Analysis of the STCAM
...............
76
2.4.3.2.2
Selecting Values for the Circuit
...............
Elements in the STCAM
78
Response of the STCAM to Simple
.............................
Signals
78
The Multiple Time Constant Adaptation
Model
...................................
82
2.4.3.3
...............
2.4.3.3.1
Analysis of the MTCAM
2.4.3.3.2
Selecting Values for the Circuit
...............
Elements in the MTCAM
88
Response of the MTCAM Circuit to
......................
Simple Signals
9
2.4.3.3.3
2.4.3.4
Implementation Issues
2.4.3.5
Complete Example of the Adaptation
.......................
Stage Response
...................
POSTPROCESSORS: MEASURING PROPERTIES OF THE
...................................
PAM RESPONSE
96
97
.......................
99
2.5.2
Two Smoothing Filters
2.5.3
Averaging across Channels
SOME OTHER AUDITORY MODELS
1_
95
97
A Masking Detector
_llq
84
..........................
2.5.1
2.6
76
2.4.3.2.1
2.4.3.2.3
2.5
..........
_111
..................
......................
99
10
___
II__
11
Table of Contents
CHAPTER 3:
3.1
PAM RESPONSE PATTERNS AND PHONETIC DISTINCTIONS
INTRODUCTION: USING THE PAM TO STUDY PERCEPTUAL RESPONSES TO SPEECH
3.2
......
154
..............................
154
..................
155
....................
157
............................
16g
PAM Processing Procedure and Param...................................
eters
161
...................
162
............
165
....................
167
......................
170
.....................
171
..............................
171
Background
3.2.1.2
Experimental Procedure
3.2.1.3
Results and Analysis
The PAM Response
3.2.2.2
PAM Response Patterns
3.2.2.3
A Possible Auditory Analysis
3.2.2.4
Statistical Analysis
VOICED/UNVOICED EXPERIMENT
3. 3.1
1501
.....................
The original experiment
3.2.2.1
.......
154
3.2.1.1
3.2.2
.....
...........................
STOP/GLIDE EXPERIMENT
3.2.1
3.3
10 9
...............
FIGURES AND FIGURE CAPTIONS FOR CHAPTER 2
The original experiment
3.3.1.1
Background
3.3.1.2
Experimental Procedure
. .
t.. . . . .. .. . .
172
II
I
--
1--
·-·-1----·IIIII---·lllll(·-·II·XII
l-C-_l.
--
1_1
.1-_--_111·
- -- _- ----
Table of Contents
3.3.1.3
3.3.2
Analysis
...............................
The PAM Response
3.3.2.1
3.4
12
174
............................
177
PAM Processing Procedure and Parameters
....................................
178
...................
178
3.3.2.2
PAM Response Patterns
3.3.2.3
A Possible Auditory Analysis
............
..........................
CONCLUSION
181
189
FIGURES AND FIGURE CAPTIONS FOR CHAPTER 3
..............
193
CHAPTER 4: EXTENSIONS AND FURTHER EXPERIMENTS
4.1
EXTENSIONS TO THE PERIPHERAL AUDITORY MODEL
4. 1.1
Extending the Spectral Analysis
4. 1.2
Extending the Transduction Stage
4. 1.3
Extending the Adaptation Stage
4.2
Further Temporal Effects
240
..............
246
...........
246
....................
247
...............
247
................
250
"Slit-Split" Contrast
4.2.1.1
The
4.2.1.2
The "Shop-Chop" Contrast
4.2.1.3
Speaking Rate, Auditory Response, and
Phonetic Perception
--
1 El P"L-·.·
------
237
............
FURTHER AUDITORY PHONETIC EXPERIMENTS
4. 2.1
_
Stage
236
.....................
------·--- ··
11·1___-----_
?51
13
Table of Contents
4.2.2
Further Investigations of Auditory Correlates tolVoicing in Stops
...................
Auditory Correlates to Place of Articula-
4.2.3
tion in Stops
...............................
FIGURES AND FIGURE CAPTIONS FOR CHAPTER 4
Bibliography
.....................
...............
II
I
_____lll__sj___l________I______)_
254
259
267
.....................
__
_j
252
I
IC_1_
14
LIST OF FIGURES AND TABLES
FIGURES AND TABLES FOR CHAPTER 1:
Figure 1.1
The human external, middle, and inner
ear
Figure 1.2
.......................................
The structure of the cochlea
...............
44
45
FIGURES AND TABLES FOR CHAPTER 2:
Figure 2.1
Block diagram of preprocessor, PAM, and
...............
117
Magnitude of the frequency response of
.....................
the preemphasis filter
118
postprocessor
Figure 2.2
Figure 2.3
Block diagram of the Peripheral Auditory
Model
Figure 2.4
Table 2.1
Figure 2.5
Figure 2.6
......................................
Graph of the magnitude of the frequency
response of three adjacent PAM filter
...............................
bank filters
120
List of center frequencies of the PAM
...........................
bandpass filters
121
Gain and phase of even numbered PAM
.......................
filter bank channels
122
Gain and phase of even numbered PAM
filter bank channels
_
^ _
_11_11__
119
------
_1__11_------ -.
.......................
_
_-
123
List of Figures and Tables
Figure 2.7
Magnitude of frequency response of indi......................... 124
vidual PAM filters
Figure 2.8
The composite frequency response of the
............................ 125
PAM filter bank
Figure 2.9
Impulse response of PAM filter bank
........
126
Figure 2.10
PAM filter bank response to two sine
waves
................... ...................
127
Spectral slices of the PAM filter bank
...................................
response
128
PAM transduction stage and its derivation
.......................................
129
.......
131
......................
131
Figure 2.11
Figure 2.12
Figure 2.13
Four possible transduction functions
Figure 2.14
DC transfer functions
Figure 2.15
DC transfer functions for the four
transduction functions shown in Figure
.......................................
2.13
132
PAM transduction stage response to two
.................................
sine waves
133
Comparison of the PAM filter bank and
...............
transduction stage responses
134
Figure 2.16
Figure 2.17
_
15
.... 135
Figure 2.18
Single time constant adaptation circuit
Figure 2.19
Response of the STCAM to a step onset and
offset
..................................... 136
Figure 2.20
Demonstration of the constant incremental
..........
gain characteristics of the STCAM
_
I
____________1________1_1___111
137
II
_ _
1__5__11
List of Figures and Tables
Figure 2.21
Figure 2.22
Figure 2.23
Figure 2.24
16
Origins of nonlinearities in the PAM
response
.................
..................
138
Demonstration of forward masking in the
responses of the STCAM
............
.........
139
Another demonstration of masking
phenomenom, and the effect of the intensity of the masking signal
.................
140
Multiple time constant adaptation circuit
.......................................
141
Figure 2.25
Adaptation circuit with two time constants
..................................... 142
Figure 2.26
Responses of the circuit shown in Figure
2.20
.......................................
143
Response of the' MTCAM to signals with
ramp onsets of various slope
...............
144
Effect of previous signal levels on the
size of the response of the MTCAM to onsets and offsets
...........................
145
Figure 2.27
Figure 2.28
Figure 2.29
Demonstration of the dynamic range of the
transient and steady state response of
the MTCAM
.............................. .... 146
Figure 2.30
PAM adaptation stage response to two sine
waves
......................................
Figure 2.31
147
Comparison of the transduction stage
response, and adaptation stage response
at various times
........................... 148
Illl··CI·l--·-·-···--a-----
__I
I
d(
List of Figtires and Tables
Figure 2.32
17
Masking postprocessor response to two
sine wave test signal
.............
........ 149
FIGURES AND TABLES FOR CHAPTER 3:
Figure 3.1
Figure 3.2
Table 3.1
Figure 3.3
Figure 3.4
Figure 3.5
Figure 3.6
Figure 3.7
Figure 3.8
Schematic diagram showing synthesis
parameters of stimuli in STOP/GLIDE experiment
...................................
201
Waveforms of stimuli with rapid and slow
onset times
......................
202
Percent STOP judgements for each
stimulus
..............
....
203
Percent stop judgements as a functi on of
onset time
.......................
204
Boundary duration as a function of syllable duration
....................
205
PAM response to stimulus with 15 msiec onset and 299 msec duration
........
206
PAM response to stimulus with 15 msiec onset and 87 msec duration
.........
207
PAM response to stimulus with 60 ms(ec onset and 299 msec duration
........
208
PAM response to stimulus with 60 msiec onset and 87 msec duration
.........
.
Figure 3.9
_
I-·-----·
·-----
.
.
.
209
PAM response to stimulus with 30 msec onset and 299 msec duration
_
.
---I.----
... t..........
11-
LI LBl________l___sS_Y___·
18
List of Figures and Tables
Figure 3.10
Table 3.2
PAM response to stimulus with 30 msec on...................
set and 87 msec duration
211
Location of onset and offset pulses in
.........
smoothed composite response signal
212
...........
213
Figure 3.11
Smoothed composite PAM responses
Table 3.3
Summary of multiple regression analysis
......................... 214
of stop/glide data
Figure 3.12
Phonetic boundary of the onset measure as
......... 215
a function of the offset measure
Figure 3.13
Spectrograms of the four original "rabid"
..........................
tokens
...........
216
Waveforms of stimuli with shortest and
........
longest closure and vowel durations
217
Audiograms of the two groups of sub......................................
jects
218
Figure 3.14
Figure 3.15
Figure 3.16
Percent of "p" responses
....................
219
Figure 3.17
Interpolated voiced/unvoiced phonetic
.................................
boundaries
220
PAM response to stimulus with 160 msec
..................
vowel and 35 msec closure
221
PAM response to stimulus with 160 msec
.................
vowel and 125 msec closure
222
PAM response to stimulus with 220 msec
..................
vowel and 35 msec closure
223
Figure 3.18
Figure 3.19
Figure 3.20
I
_
_II i _1__1_____1_1_11_*___11··11---11
--
-
I_--·-*
II
19
List of Figures and Tables
Figure 3.21
PAM response to stimulus with 220 msec
................. 224
vowel and 125 msec closure
Figure 3.22
PAM response spectra of stop bursts
Figure 3.23
Output of masking post-processor for
stimulus with 160 msec vowel and 35 msec
o-1
Figure 3.24
Figure 3.25
Figure 3.26
Figure 3.27
.
Table 3.4
Figure 3.29
Output of masking post-processor for
stimulus with 160 msec vowel and 125 msec
closure
........................
227
Output of masking post-processor for
stimulus with 220 msec vowel and 35 msec
closure
........................
228
Output of masking post-processor for
stimulus with 220 msec vowel and 125 msec
closure
........................
229
Three representations of a stimul us per..............
ceived as "rapid"
230
..
Figure 3.30
I_ _ _
....
Three representations of endpoint stimu.............................
li
231
Linear fit of decline of masking during
stop closure: "Young" analysis
232
Relationship between perceptual a nd masking boundaries: "Young" analysis
233
Linear fit of decline of masking during
stop closure: "Old" analysis
234
Relationship between perceptual and mask.............
ing boundaries: "Old" analysis
235
.e.e.
Table 3.5
225
226
*...
Figure 3.28
........
_
___
__
.....
I
_
·
~I
List of Figures and Tables
FIGURES AND TABLES FOR CHAPTER 4:
Figure 4.1
Figure 4.2
Schematic diagram of one channel of a
channel vocoder system
.............
.......
.
.
.
.
261
262
Figure 4.3
Schematic diagram of a single channel of
the proposed PAM phase vocoder-based
transduction stage
......................... 263
Figure 4.4
PAM response to stimulus perceived as
"split"
..................................
264
Three representations of "slit/split/slit" stimuli
...............................
265
Two representations of the spectral shape
of the burst and second glottal pulse
following voice onset for /ba/ and /da/
266
Figure 4.5
Figure 4.6
·
Slope of phase response of auditory
fibers
......
.... .....
......
...
IRI-·-b
I1(·I
L--
-
·
__
I
--_
__
_
_
1
_·
21
CHAPTER 1
INTRODUCTION:
PERIPHERAL AUDITORY RESPONSE AND THE ANALYSIS OF SPEECH
1.1
MOTIVATION AND OVERVIEW
This thesis describes a model of the peripheral auditory
system (PAS), and discusses certain properties of its response
to
speech
regarding
signals.
the
role
The
of
perception of speech.
the
the
signal.
encoded
which
Whereas
in the
pherable ways,
in
peripheral
by
auditory
a
system
theory
in
the
signals
into
correspond
the
to
phonetic
neural
the
firing patterns
phonetic
features
of
content
speech
of
are
acoustic waveform in complex and often undecithis
terns with which
correspond
motivated
It states that the peripheral auditory
system transforms speech
properties
was
This theory will be referred to as the
correspondence theory.
with
study
a
theory suggests
that
speech is represented
more
underlying phonetic
direct
and
features.
It
the
discharge
in the auditory nerve
straightforward
is
pat-
manner
to
the peripheral auditory
system, and in particular the cochlea,
that brings about this
correspondence.
The correspondence
theory does
not
assign
any phonetic
detection or discrimination task to the PAS, nor does it imply
that the PAS is especially "tuned" to speech, or
ferently
~~I~
to
...
~
speech
sounds
I
than
to
other
~1
sounds.
reacts difRather,
_~_1~~·11~~-_--
it
Introduction
22
suggests that human speech has
advantage
of
the
characteristics
system; and that the neural
the
basis
for
the
takes
developed in a way that
of the
peripheral auditory
which form
signal characteristics
identification
and
discrimination
of
at
least some phonetic contrasts are established by the PAS, and
are apparent in the response patterns of the auditory nerve.
The PAS effects the transformation between the representation
of
speech
as
an
acoustic
pressure
argues
that
the
form
of
important to speech perception as
perception
tions
of
relies
on
relationships
that
its
is
as
and that speech
and peculiar transformaprofiles,
intensity
acoustic
transformation
its nature,
the particular
shapes,
spectral
this
and
The correspondence
representation as a neural firing pattern.
theory
wave
signals
undergo
and
as
temporal
they
travel
through the peripheral auditory system.
From
the
correspondence
point
of
theory
to
view
states
speech
is
different--representation
than
response
which
to
attempt
to
signal
alternative
recognize
of
pattern
that
the
a
the
as
the
peripheral
auditory
better--not
merely
acoustic
speech.
representations
recognition,
By
waveform
implication
short
term
from
such
Fourier
transforms and linear predictive encoding,t to the extent that
they do
not
incorporate
peripheral
auditory transformations,
are deficient as representations on which to attempt phonetic
analysis.
-
-T
--··
I
..
That is,
in the peripheral auditory representation
---..·3-1)
------
_______
Introduction
23
and
discriminable,
are more
features
phonetic
represented
in
fewer dimensions, than they are in such alternative representations.
The correspondence theory depends on two testable assumptions
the apparent properties of acoustic and audi-
regarding
tory representations
of speech.
"Apparent"
the literal
sense of appearing, or visible.
perties
meant
are
appreciable,
is
used here
in
By apparent pro-
surface-level
features
of
a
representation, or simple relationships between such features.
The first assumption is that cochlear transformations of
speech signals result in auditory neural firing patterns whose
properties are significantly different from the properties of
signals
acoustic
spectrogram-like
or
their
analysis
representation
classical
techniques.
This
is
a
by
necessary,
though not sufficient, condition for the correspondence theory
to
be
true:
clearly
if
the
patterns
of
auditory
neural
response are insignificantly different from, say, the patterns
of wideband spectrograms,
the correspondence theory cannot be
true in any important way.
The
second assumption
is
that
particular
properties
of
the neural representation of speech signals reflect particular
phonetic
features
of the
stimulus.
If
this
were
not
true,
then even though the PAS might radically transform the signal
properties
of
speech,
the
decoding
of
phonetic
_1_1_
information
-·-------^_1_1_
Introduction
24
from the auditory nerve
response would still appear to depend
wholly on the activjity of the higher auditory centers
brain.
In that case, perhaps any peripheral representation of
speech that did not actually delete
the
in the
acoustic
central
signal
nervous
could be
system.
The
information contained
in
interpreted equally well by the
correspondence
theory,
on
the
other hand, implies that the PAS plays a substantial and vital
part in the perception of speech.
To test this
theory a computer model
auditory system was
constructed.
of the
peripheral
The model consists of three
stages, each of which effects a transformation on the acoustic
input
of
signal.
filters
response
models
whose
of
the
expected
The spectral
the
basilar
rate.
The
that
tained
The
stimulation.
of a
reflect
membrane.
transformation
poral -transformation
firing rate
stage consists of a bank
characteristics
cochlea's
firing
analysis
The
of
adaptation
mimics
the
transduction
signal
stage
neural
set
of
stage
intensity
produces
adaptation
output of the model is
representative
frequency
to
a temto
sus-
the expected
auditory
neurons
in
described
in
response to the input signal.
This
Peripheral
Chapter Two.
Auditory
The model is
Model
(PAM)
is
primarily based on published elec-
trophysiological data of firing patterns observed in the auditory nerve.
tic
These data describe the neural response to acous-
stimuli,
1_11
and
_11_
the
ways
in which
-- ---II._IC--lll
neural
firing
patterns
-------- )1/-1__·_·_11111_1_1_
__1_----·--
I-·----
25
Introduction
change
and
as
function of the onset characteristics, duration,
a
Only
intensity of the stimuli.
those
features of
peri-
pheral auditory response which were thought to play an essential
role
in
the perception
have
been
modeled.
No
of consonant-like
attempt
been made
has
reproduce all aspects of the peripheral
as the
speech
to
sounds
accurately
auditory system, such
frequency response of the outer and middle ear, or the
many nonlinear phenomena associated with the basilar membrane.
PAM
the
Similarly,
not
is
a physiologically
be
to
intended
valid model of the peripheral auditory system.
The
two
hypotheses
regarding
nature
the
of
auditory
neural response are tested by studying the model's response to
In Chapter Two a number of simple signals exhibiting
signals.
speech-like spectral structure, onset and offset characteristics, and durational properties, are used as stimuli to demonstrate the model's
first
behavior and
The
hypothesis.
to
test
the validity of the
of
correspondence
patterns
of
peri-
pheral auditory response with patterns of phonetic perception
is
in
demonstrated
described
response
which
Chapter
relate
measurable
perceptual
to
patterns
consonants
and glides.
boundary
O
span the
stimuli
between
r
voiced
In
and
-----^-"-
experiments
properties
judgments
trasts elicited from human subjects.
the speech
Two
Three.
of
of
the
phonetic
PAM's
con-
In the first experiment,
perceptual boundary between
the second,
unvoiced
are
the
stimuli
intervocalic
stop
span the
stop
con-
Introduction
26
sonants.
In both experiments it is shown that an analysis using a
single measurable property of the output of the PAM can do as
well as, or better than, a classic "trading relation" analysis
based
on
such
as
two
or
onset
more
time
measurable
and
acoustic
syllable
signal
duration.
properties
Evidence
is
presented that two well known and very general properties of
neural
firing
patterns,
namely
spontanteous
firing
in
the
absence of stimulation, and adaptation of mean firing rate in
the presence of sustained stimulation, may play an important
role
in
conforming
peripheral
auditory response
to phonetic
content.
The
results of
correspondence
these
theory of
experiments
peripheral
tend
to
auditory
support
function.
the
The
results show that acoustic patterns of speech-like signals are
transformed
by
the
peripheral
auditory model
patterns with quite different properties.
into
auditory
They also show that
certain perceptual judgements of the tested speech signals by
human subjects conform closely with measurable properties of
the model's response.
These measurable properties correspond
to changes in the overall expected firing rate
in the mam-
malian auditory nerve in response to the speech signals.
The final chapter discusses limitations of the peripheral
auditory model, and suggests improvements that should enhance
_
--PP
11
1·~_·~--__~-~-
27
Introduction
list of known acoustic
a
also presents
It
studies.
further
phonetic effects for which perceptual data are available, and
be
would
which
interesting
in
re-analyze
to
auditory
the
domain.
1.2
THE ROLE OF PERIPHERAL AUDITORY MODELS IN SPEECH ANALYSIS
study of
The
phonetics
particularly true
since
the
in
invention
et al.,
(Koenig
spectrograph
This has been
for experimental data.
on acoustic instruments
1946
This
1946).
largely
relied
traditionally
has
of
sound
the
instrument
has
proved to be of such remarkable utility and longevity that its
particular form of output, the sound spectrogram, might almost
Indeed, almost all dis-
seem to show speech in its True Form.
cussions
of
acoustic
phonetics
observable properties of
on spectrograms: bursts,
as
speech
descriptive
as
patterns
terms,
displayed
formant peaks, stop gaps, voice bars,
Even the advent of sophisticated computer-based
and so forth.
signal
visual
use,
processing
done surprisingly little
systems has often
to change the fundamental terms
in which speech is discussed.
A popular use of computers for speech analysis consists of the
of
production
digital
whose
spectrograms
characteristics
are
modeled after their analog predecessors.
At
the
same
signals
has
been
analysis,
7a----
a.
the
time
that
accepted
basis
Rsa;l-xl-a
for
this
as
of
representation
the
natural
most models
--.-
of
form
speech
acoustic
for
has
--------.
--
speech
been
--
the
I-
`---..-"1*
Introduction
28
human speech production system.
been
advanced
by
such
works
Production-based analysis has
as
Fant's
acoustic theories of vowel production
the
predictive
LPC
is
of
realm
a
of
(LPC)
has
1961).
techniques,
linear
become
based
of analysis
(Stevens et al.,
analysis
computer-based
encoding
method
Theory
(1960) and research by Stevens and others on
Speech Production
In
Acoustic
on
particularly
the
popular.
representation
of
speech as the product of a linear time-varying system excited
by
a
either
or
quasi-periodic
reviews of LPC
see Makhoul
and Wolfe,
1972;
(For
source.
random-noise
or
Rabiner
and
Schafer, 1978).
These production-based models
stantially successful
teristics
of
waveforms.
story.
Although
cant amount of
speech
have been
sub-
in predicting and analyzing the charac-
speech
only half the
of
This,
however,
spectrograms
information about
speech,
encompasses
reveal a signifi-
they
often
display
properties that are not perceptually relevant, while obscuring
important, and perceptually clear, aspects of the speech signal
1982).
(Klatt,
The patterns of the spectrograph
the speech patterns which humans hear.
form different
spectrographic
Speech
patterns may
are not
signals which
sound the same to
human listeners, and other signals which appear imperceptibly
different
on
the
spectrogram
remains
true
may
that
elicit
speech, is
It
through the
response patterns of the peripheral auditory sys-
_
I·_
I
only
phonetic
responses.
__I_
_
different
perceived
C__I
__-_IIIYX-·LIIZ·i---F·-
Introduction
Given
our
current
represents
minus
30
of
of
ignorance,
the
auditory
a regioni of manageable complexity.
the
secondary
state
auditory
fibers
nerve
branch
out
in
the
nerve
From the ter-
cochlear
nucleus
in many directions
to
both within the CN and to other auditory nuclei
(CN)
locations
in the brain.
In addition to these ascending (afferent) fibers, a variety of
descending
creating
(efferent)
complex
fibers
feedback
terminate
loops.
To
in
the
further
of
response
nerve.
response
patterns
patterns,
exhibited
by
much
the
less
the
fibers with a
uniform
neurons
nucleus,
complicate
situation, the central auditory system contains
variety
CN
in
than
the
the
auditory
Thus the auditory nerve represents a pathway of rela-
tive simplicity before
the
so-far
overwhelming
complexity
of
CNS interconnection.
(This is not
the
PAS.
On
the
to
say that
contrary,
no
the
feedback circuits
PAS
is
well
efferent fibers originating higher up in the
exist
supplied
nervous
in
with
system.
Since almost all auditory nerve studies have been performed on
anesthetized animals whose
state prevents
the
efferent audi-
tory neurons from firing, little is known about the role these
fibers, and the feedback loops they support, might play in the
normal perception of speech.)
A fourth reason to study the output of
many
of
the
transformations
that
take
the
place
PAS
there
is that
are
the
result of non-neural processes: most prominently the frequency
-
"""I----
-I-------------
Introduction
29
tem.
It is through this system, consisting of the outer, mid-
dle,
and inner ear, that some transformation of the acoustic
pressure waves of speech sounds reach the central nervous system.
In particular, it is through the firing patterns of neu-
rons in the auditory nerve that the CNS receives all
information
about
the speech signal
of its
it must interpret.
Any
information that is not somehow encoded in that auditory firing
pattern
is,
ipso
facto,
first reason for directing
not
perceivable.
attention
some
This
is
the
it
to the PAS:
is
the input to the auditory perceptual system of the brain.
The second reason for attending to the patterns of auditory neural response has to do with experimental practicality:
the
response patterns
Using
micropipettes
variety of animals
directly
record
terminate
on
auditory
of the
implanted
in
(most commonly
the
firings of
sensory
cells
the
nerve
are
auditory
cats),
it
is
representative
located
accessible.
nerve
of
possible
neurons
within
the
a
to
that
cochlea.
Through the pioneering work of such researchers as Kiang and
his
associates
(Kiang, Watanabe,
electophysiological
Thomas,
and
Clark,
1965),
measurements have become relatively
rou-
tine, and an extensive amount of data regarding the discharge
patterns of single auditory neurons to a wide variety of both
non-speech and speech-like stimuli are now available.
There
is
a third reason
why the
auditory
nerve
is
an
attractive place to study auditory response to speech stimuli.
i·-
1
-----
CII---·I(·IIL·.
_
1----------
1__11
31
Introduction
response
of the
and middle
outer
ear,
macro- and micro-
the
mechanical properties of the basilar membrane,
While
cells.
receptor
the
of
properties
trochemical
and the
elecit
is
in the speech sig-
clear that further radical transformations
take place in the central nervous system, these transfor-
nal
mations may be expected to differ in kind from the transformations observable in the PAS.
An
important
of
understanding
is
deafness
whose
due
problems
clinical
of
but
sensory
direct
adequate
unless
characteristics
to
to
restore
electrical
models
an
hearing
to
people
cochlea.
stimulation
of
The
the
are being steadily over-
of
peripheral
the
of
from
signals in the
in the
loss
cochlear implants
auditory nerve via
come,
ability
result
of speech
representation
the
is the
auditory nerve
that may
potential benefit
the
signal
auditory
processing
system
are
developed, the resulting auditory sensations will continue to
be
highly
artificial,
and
of
little
use
in
understanding
speech.
A final reason to focus on the response patterns of
is
PAS
that there
is
no
evidence to suggest that the PAS in
man is specialized for the perception of speech.
of
the
characteristics
fundamental
Indeed, most
of PAS response appear to
be qualitatively similar for a wide range of mammals.
investigation
help
answer
of the
the
the
role
question
Thus an
of the PAS in speech perception can
of
the
extent
to
which
speech
_1
lo
32
Introduction
depends
perception
mechanisms,
perceptual
speech-specific
on
auditory
and to what extent it is carried out by more general
mechanisms.
In addition, of course, any aspect of speech per-
ception that depends on the peripheral auditory system is certo
tain
be
language
auditory response can help separate
pheral
studying
Therefore
independent.
peri--
indepen-
language
dent from language dependent aspects of speech perception.
THE NATURE OF PERIPHERAL AUDITORY RESPONSE
1.3
This section contains a review of the major structures of
peripheral
the
transformation
contribute to the
auditory neural
the
and
system,
auditory
ways
in which they
stimulation into
of acoustic
The contribution of some of these
response.
structures to peripheral auditory response is well represented
in the PAM.
or
Other structures are represented less precisely,
not at all.
The degree of
representation
in the PAM will
be mentioned in the discussion below.
Figure 1.1
and
middle,
inner
ear.
Acoustic
funneled through the outer ear
pressure
to the
linkage
stapes
with
cochlea.
filled
__
ossicles
of these
a
As a
cavities
piston-like
result,
the
in
are
set
of the cochlea.
··L-·-CL
-___..+-·l-L------
air are
-- I.
The mechan-
base-plate
the oval
traveling waves
and membranes
outer,
eardrum, where they are
drives
motion
in
waves
converted into motion of the middle ear ossicles.
ical
the
structure of
the anatomical
shows
of the
window of the
up
in
fluid-
The resulting
33
Introduction
motion of the cochlear membranes stimulate sensory cells ener(the eighth cranial nerve), which
vated by the auditory nerve
of
centers
auditory
higher
the
to
information
carries
the
brain.
of
acoustic
free-field
things,
other
Among
the
for
(see,
signal
Shaw,
example,
enhance
selectively
pinnae
frequency
between the
ear
speech
Mellert,
and
(Mehrgardt
outer
for
range
sound
be
can
signals:
For
1977).
source and the head
modeled
as
a
the
mid-
roughly
2
in
frequencies
enhance
canal
ear
linear
a
fixed
of
the
the
1974).
high-
The concha
frequency sounds originating in front of the head.
and
of
amplification
direction-dependent
and
frequency-
are primarily ones
ear
of the outer
The transformations
to
high7
to
KHz
orientation
listener,
time-invariant
the
filter
with roughly bandpass characteristics.
The middle ear also acts
frequencies
as a bandpass
in the range of 200
enhancing
to 200 Hz (Zwislocki, 1975).
action of the ossicular
in the
A nonlinear element is present
filter,
muscles, which contract in response to loud (approximately 8
dB
SL)
reducing
thereby
sounds,
through the ossicular chain.
to
action
serves
tially
damaging
this
protect
levels
reflex plays
an
the
the
of
transfer
Presumably this acoustic reflex
delicate
of stimulation.
important
role
in
inner
It
ear
is
-_I___-----·-·----------·-·---··
from poten-
unlikely that
acoustic
intensity levels typical of conversational speech.
________
pressure
response at
II
Introduction
34
The transformations
of the outer and middle
represented in the PAM in a principled way.
6
dB/octave
roll-off
of voiced
Instead, a simple
speech
sounds
reflex of the ossicular
The acoustic
quency.
not
used to counteract the
filter is
first-difference preemphasis
ear are
at high
muscles
fre-
is not
represented at all.
Sound waves
to the
a
in
results
transferred
of
motion
piston-like
vestibule
are
the
the
inner
in
the
oval window of the
stapes
cochlea.
vibratory
to
The
resulting
of
pattern
surrounding the scala media,
fluid
motion
including
ear
in
via
the
displacement
structures
the
the basilar membrane.
This sequence of events is shown schematically in Figure 1.2A,
where
the
cross
sectional
duct
coclear
is
shown
of
structure
more detail in Figure 1.2B.
of
the
basilar membrane
the cochlea
membrane
to
its
respond
vibration.
This
apex,
as
the
an
unrolled
cochlear
is
shown
The
in
Because the width and flaccidity
steadily
increases
different points
to
preferentially
structural
duct
tube.
from the base of
along
different
variation
the basilar
frequencies
provides
the
of
primary
frequency analysis of the peripheral auditory system.
The PAM equivalent to the frequency analysis of the basilar
membrane
is
a
filter
bank
with
2
channels,
roughly
corresponding to 20 equally-spaced locations along the basilar
membrane.
Although the basilar membrane exhibits a number of
nonlinear effects, it is not known how important those effects
---_I - - - - - -1----I--I_-·
11Ow
____
Introduction
35
The PAM filter bank is com-
are to the perception of speech.
pletely linear.
The sensory organ- of the auditory system in the organ of
on the medial surface of the basilar membrane.
Corti, mounted
Figure 1.2C shows the structure of this organ,
three
cells.
in
of outer hair
rows
These cells
surface of each
rod-like cilia.
acoustic
its
of
these transducer
turn,
each
through
which
causes
can
be
cells
the
of
electrochemical
measured
as
rows
the
upper
of
stiff
haircell
body drags
scala media, causing
the
This
in
the
angular displacement,
changes
changes
are
changes
rocks in response to
rigidly-attached
fluid
into
from
Emerging
potential.
cilia to pivot about their base.
in
motion
As the basilar membrane
stimuli,
cilia
transduce mechanical
electrical
internal
single row of inner hair
and
cells
including its
within the haircell,
the
haircell
receptor
potential.
The PAM contains a very simple model
to electrical
transduction.
The model
of this
mechanical
is a minimal model, in
the sense that it simply posits a smooth monotonic saturating
nonlinear
the DC transfer
driven by
plicity
of
the
I
and
model,
of
output,
·I---.
of various amplitude.
it
displays
haircell
behaviour
·-
and
and derives
function of such a transduction device
sine waves
characteristics
nature
input
relationship between
of
at
rate-level
this
least
I-----------------3
Despite the simfour
important
relationships.
transduction
when
model
will
The
be
-
-II-
Introduction
36
discussed at length in Chapter Two.
Each inner haircell of the organ of Corti
is
innervated
by as many as a dozen- or more afferent auditory nerve fibers.
The
outer
haircells
efferent fibers.
are
innervated
almost
exclusively
by
Since a particular afferent fiber terminates
at the base of only a single haircell on the basilar membrane,
each
fiber
is
frequencies,
brane
Each
is
preferentially
and
the
reflected
channel
of
expected
firing
haircell
located
stimulated by a single
frequency analysis of the
in
the
firings
of
the
PAM
rate
of
an
afferent
at
one
of
twenty
is
the
intended
range of
basilar mem-
auditory
to be
fiber
neurons.
a model
of
terminating
equally
spaced
the
at
a
locations
along the basilar membrane.
Neural
transmitters
function
firings
by
of the
are
generated
by
the
release
of
neuro-
an inner haircell.
The rate of release is
receptor
in
potential
the
haircell,
and
a
is
reflected in the firing rate of the afferent neurons terminating at that haircell.
Little is known about the role of outer
haircells and efferent neurons in mammalian auditory systems.
Art
and
Fettiplace
(1984)
have
strong stimulation of efferent
central
nervous
system
seems
shown
that,
fibers at
to result
in
the
turtle,
their origins in the
in
elevated
auditory
thresholds and a broadening of the frequency response of associated afferent fibers.
_
_-_
__
I
____sC_____III_·LI___-------11--·111^
_I
_
1_1
II_
_
37
Introduction
electrical
spike-like
random,
are
firings
Neural
discharges, and can be described using the statistics of point
to the quasi-periodic changes
responses
These neural
events.
in the internal potential of the haircell (caused by acoustic
the short term
stimulation) can be characterized in two ways:
average
number
of
firings
for
a given
level of
stimulation,
and the temporal distribution of those firings relative to the
phase of the stimulation.
be
used
to study
an
in response to
discharges
of neural
Post stimulus time (PST) histograms
the
effect of
stimulus
parameters, such as
both
intensity, and duration, on
frequency,
can
stimulus
acoustic
rate
average
the
of firing and the temporal characteristics of the response.
Some of the more obvious
response
(for
rate
include:
different
correlated
of
range;
longed
in
stimulation.
in
represented
spontaneous
of
absence
threshold
of
a
increased;
is
of
the
average
firing
rates
stimulation;
a
response
as
the
limited
dynamic
in firing rate at high stimulus intensifiring
of
adaptation
an
of
the
the
stimulation
a saturation
and
ties;
range
in
neurons)
variation
intensity
a
characteristics
All
the PAM,
of
and
rate
these
will
be
in
to
response
phenomena
discussed
are
in
prowell
detail
in
Chapter Two.
The
limited
saturation
stimulus
in
dynamic
the
firing
intensities
I-^-
-
range
of
rate
of
below those
-I·I-.
·
neural
many
auditory
typically
-
firings,
found
and
neurons
in
the
at
speech,
lo
Introduction
have
38
led many researchers
particularly of vowels,
rate
(see,
for
to doubt
that spectral information,
can be adequately encoded
instance, Sachs
and Young,
1979).
in
An attrac-
tive alternative makes use of the synchrony of neural
to a particular phase of the
firing
firings
stimulating acoustic wave.
Neu-
rons terminating at locations along the basilar membrane that
respond preferentially to frequencies below about 3 KHz show a
statistical preference for firing during a particular phase of
basilar
membrane
motion.
This
preference
can
ultimately be
traced to a non-symmetrical relationship between the orientation
of
cilia
haircell
displacement
receptor
above 3 KHz.
and
potential.
the
This
resulting
preference
change
is
in
not
the
seen
possibly because of capacitive low-pass filter-
ing effects in the haircell.
A
variety
proposed
and
1981, 1984b;
of
generalized
implemented
(Young
Seneff, 1985).
tory response
data
synchrony
shows
and
measures
Sachs,
have
1979;
been
Delgutte,
Applying these measures to audi-
that
the degree
of phase
synchrony
increases as a function of stimulus intensity, beginning below
the
threshold
beyond
the
saturated.
of
increased
intensity
at
It has been
firing
which
shown
rate,
and
expected
that the
continuing well
firing
spectra
rate
of vowels
has
is
well represented by such synchrony measures, albeit with significant differences
models
_
I
of
auditory
-- · ·-
from acoustic spectra.
synchrony
· ·--
· ·---
P--····----·--·I
are
It is clear that
important,
___.___
and
_111
active
39
Introduction
research continues.
The PAM does not model
That aspect
auditory synchrony.
of auditory response was omitted from the model, which is primarily intended to model temporal response patterns resulting
from
the
of
stimulation
PAS by
sounds.
consonant-like
Such
sounds often include substantial high frequency energy, typi-
sounds,
shape
and
and
prominent
less
have
cally
are characterized by abrupt
amplitude.
However,
a
changes
strategy
vowel-like
than
peaks
spectral
in
spectral
expanding the
for
PAM to include neural synchrony, and other methods of increasing the effective dynamic range of the model, are discussed in
Chapter
Four.
appropriate
for
These
extensions
would
make
the
PAM
more
investigating peripheral auditory response to
vowels.
1.4 PERIPHERAL AUDITORY RESPONSE AND THE ISSUE OF INVARIANCE
AND VARIABILITY IN SPEECH
The relationship between phonetic features and observable
properties of the
speech
of acoustic phonetics.
signal is a major issue in the study
One
theory of speech
perception--the
theory of acoustic invariance--states that phonetic perception
depends on the detection by the auditory
system of invariant
properties in the acoustic signal of a speech segment (Stevens
and Blumstein, 1981), properties which are directly related to
the distinctive features which combine to specify the identity
---------
--
------11111111··C-·-··-·3···1141·llrll
·
-
-
..I---.-·--------111-----
------·------ r--------
--_-----r--Pi
lo
Introduction
40
of a word.
However, despite decades of work aimed at articulating a
"unified
theory"
of the
acoustics and phonetics of speech, it
is still difficult in many cases to explain the way in which
the
features
distinctive
which
the phonetic percep-
describe
tion of speech are encoded into measurable properties
speech waveform:
ments
such as
using acoustic instru-
properties observable
the
spectrograph.
On
of the
the
contrary,
acoustic-
phonetic studies have shown that, if such invariant properties
do exist, they at least do not correspond to discrete
attri-
traditional
time-
butes
in
located
frequency-amplitude
Acoustic
onset
properties
rate
or
particular
regions
representations
such
duration
as
of
of
of
formant
bursts,
the
acoustic
frequency
or
etc.,
not
do
signal.
amplitude,
represent
invariant correlates of phonetic features.
Therefore any invariant correlates must take the
more
properties
complex
of
the
acoustic
signal,
for instance, the place and manner of articula-
tion affects the entire spectral
tion of
shape, as well as
the evolu-
that shape throughout some interval of time.
become clear that many, and perhaps
contrasts,
consonants,
such
as
or
the
tinuant consonants,
_
presumably
In the case
spread over particular time and frequency ranges.
of consonants,
form of
the
all, Iistinctive phonetic
contrast between
contrast between
are
It has
voiced
continuant
affected by the
and
and
unvoiced
noncon-
relative strengths of
_.___
I
r
Introduction
41
several--sometimes many--acoustic properties.
voicing,
been
for instance,
identified
(Edwards, 1981).
which
over
In the case of
a dozen acoustic properties have
are
salient
to
stop
recognition
Much work has been dedicated to cataloging
and studying the nature of these "trading relationships."
The transformations of the peripheral auditory system are
of particular interest in light of the difficulty of identifying
acoustic correlates
of
phonetic
features.
arises, what are the auditory correlates
in speech?
be
Are there auditory
identified
with
phonetic
The
question
of phonetic identity
response properties which
features?
These questions,
can
and
investigations into the nature of auditory response to speech,
suggest
a
new point
of view on
variability in speech.
theory of auditory
the
issue
of
invariance
This point of view might be
invariance, and
and
called
a
could be summarized in the
following steps:
_____
1.
invariant phonetic perception develops
from variable
speech signals as the result of transformations which
occur to the signals as they travel through the peripheral and central auditory systems;
2.
these transformations result in neural response patterns
which correspond to the phonetic pattern of linguistic
information in the signal;
3.
these "correlating transformations" may well
different points in the auditory system for
occur at
different
1
Introduction
42
phonetic features;
4.
the proper qu4stion is not so much whether invariance
exists, but where in the auditory system the "correlating
transformations" are located for each phonetic feature.
To what extent do the
transformations
of
the peripheral
auditory system serve as the "correlating transformations" out
of which invariant
phonetic
perception
emerges?
This
represents a first step in answering that question.
_
____
----- - .
--1-1
thesis
43
FIGURE CAPTIONS FOR CHAPTER 1
FIGURE 1.1
The human external, middle, and inner ear.
From Moore
and Lindsey and Norman (1972), by permission.
(1982)
FIGURE 1.2
The structure of the cochlea. From Pickles (1982), by permission.
A:
The cochlea shown as an unrolled tube.
Acoustic stimulation results in vibration of the stapes in the oval window.
This vibration sets up traveling waves in the basilar membrane
and the other structures which surround the scala media.
The afferent
B:
A cross-sectional view of the cochlear duct.
fibers of the auditory nerve terminate at haircells in the
organ of Corti, which sits on the basilar membrane.
Almost all of the
C:
A detailed view of the organ of Corti.
afferent fibers of the auditory nerve terminate on inner hairThe cilia of these haircells project into the scala
cells.
media, and may touch the tectorial membrane.
-
-·
----III--LC
·
--·CIIP*-
I
FIGURE 1.1
44
It
0
05
__
__I
1_1
Isls__
_1II
FIGURE 1.2
A
fl%
B
C
cftcukr
lamnt
CIl
(rod of cort,)
----
------
45
I
46
CHAPTER 2
PAM: A PERIPHERAL AUDITORY MODEL
2.1
CONSIDERATIONS IN
AUDITORY MODEL
CONSTRUCTING AND USING
THE
PERIPHERAL
This chapter describes a model of the peripheral auditory
system.
The
reproduces
model,
some of
malian
auditory
First,
it
does
which
the
system.
not
nor of the central
the
activity
Second,
it
the
focuses
be
referred
to
as
the
PAM,
response characteristics
of
the
mam-
The
include
ear,
of
will
PAM
is
a model
limited
of the
auditory system.
inner
ear,
and
in
two
outer
and
ways.
middle
The PAM only models
the
auditory
on a limited number of phenomena
nerve.
in these
parts of the auditory system.
In
the
normally
experiments described
used
in
in this
thesis
the
PAM is
conjunction with a preprocessor and a post-
processor that take the place of,
and perhaps
in
a primitive
way imitate, the parts of the auditory system that precede and
follow the inner ear.
quency
shaper
to
A preprocessing filter acts as
assist
in
the
alignment of
speech
within the limited amplitude
"window" of the PAM.
postprocessing
out
filters
pick
A
certaint properties
response patterns of the PAM so that they can be more
examined.
This
distinction
is
shown
in Figure 2.1.
a
fre-
spectra
set of
of
the
easily
Whereas
the characteristics of the PAM are based on psychoacoustic and
_II _
_
II_
I__Pl____s______l_______mCUllnLIIIIT
_________·_________IIXI···--·-··-··-
C
PAM: A Peripheral Auditory Model
the
47
characteristics
physiological
data,
processor are
the result of
of
experimental
the pre- and post-
necessity
or
tion.
The preprocessor is a typical speech processing
but at
least one of
and
ventional,
the postprocessors
hopefully
somewhat
is based on
invenfilter,
an
uncon-
treatment
provocative,
of
the PAM output.
2.2
IMPLEMENTATION
The PAM has been implemented as
C
language.
The
programs
a
use
make
set of programs
of
a
general
in
signal
the
pro-
cessing package called SigPak, and have been implemented under
The execution of the PAM
two operating systems, UNIX and VMS.
programs,
and
the
them, was
carried
acquisition
out
of
the
signals
used
to
test
on a variety of general purpose comput-
ers, including a PDP-11/60, a VAX-11/750, and an LSI-11/23.
2.3
THE PREPROCESSOR
Signal Acquisition
2.3.1
The
speech
digitized
from
in
signals described
analog
tapes
or
precision
12,987 Hz.
signals
sample
values,
thesis
synthesized
(1980) speech synthesis program.
bit
this
The
and
a
digital
were
using
low-pass
filtered
at
6.4
sampling
KHz,
Klatt
signals have
frequency
(sample period of 77.000 microseconds).
were
the
either
using
14
of
The analog
a
Rockland
11
PAM: A Peripheral Auditory Model
752A
programmable
elliptic
48
filter
with
a
frequency
response
that falls off at 115 dB/octave above the cut-off frequency.
2.3.2
Signal Preemphasis
Before
being
submitted
to
normally preprocessed by a high
the
PAM,
speech
signals
are
frequency preemphasis filter.
The input-output equation of the filter is:
y(n) = Gain * (x(n) - x(n-l)/Zero)
where x(i) and y(i) are the i'th input and output samples, and
Gain and Zero are
adjustable parameters.
This filter has the
system function:
H(z) = Gain * (1.0 - 1.0/(Zero*z))
The
magnitude
of
the
shown in Figure 2.2,
frequency
response
of
this
filter
for Gain = 0.78 and Zero = 0.80.
is
As can
be seen, this filter gives a gentle boost to mid and high frequency components of the input signal.
The magnitude of the
frequency response at 100 Hz, which is the center frequency of
channel 1 in the PAM,
is 20 dB less than the magnitude of the
frequency response at 6400 Hz, which is
of channel 20 in the PAM.
was
the
center
frequency
The use of this preemphasis filter
found necessary to compensate
for the
6 dB/octave
inten-
sity loss in typical speech signals, combined with the limited
-
~-
-
---
--
~-----7~~~~Ir-'
----
PAM: A Peripheral Auditory Model
dynamic
range of
the
PAM.
49
(As will be
seen,
the
high
fre-
quency channels of the the PAM provide an inherent 6 dB/octave
boost
to
high
frequency
broadband
signals
filters' increasingly wide bandwidths.
in
order
to
keep
the
higher
of
the
The additional amplif-
ication of this preprocessor is required
range
because
in the mid frequency
formants
of many
voiced
speech signals within the dynamic range window of the PAM.)
2.4
THE PERIPHERAL AUDITORY MODEL
The components of the peripheral auditory model are shown
in Figure 2.3.
The input to the PAM is a preprocessed signal
representing the waveform of a speech
oval window of the cochlea.
20 filter-bank-like
(or other) sound at the
The output of the PAM is a set of
signal channels.
Each channel models the
expected firing rate of a representative auditory neuron.
The
neurons represented by the PAM channels have identical tuning
characteristics,
The
20
equally
auditory
thresholds,
channels
spaced
correspond
intervals
neurons,
the
20
and
to
along
spontaneous
neurons
the
channels
within different frequency bands,
firing
terminating
basilar
respond
frequency
of
a particular channel
response
of
the
to
signal
IIC
20
Like
energy
centered at equal intervals
The frequency
roughly corresponds
auditory
system
to
the
to
the
channel's
center frequency, as derived from psychoacoustic data.
_I_
at
membrane.
along the Bark frequency scale (Zwicker, 1961).
response
rates.
I
50
PAM: A Peripheral Auditory Model
As shown in Figure 2.3, the PAM consists of three stages.
Each stage effects a transformation on the speech signal: the
spectral
produces
stage
amplitude
an
a
produces
transformation
frequency
transduction
the
analysis;
frequency
cochlear
a
to
similar
stage
analysis
the
models
that
transformation
ear's conversion of signal intensity to average neural firing
rate; and the adaptation stage produces a temporal transformaThe intent
tion that mimics neural adaptation to stimulation.
is
in this model
to reproduce a
few
selected
characteristics
response of the peripheral auditory system, which were
of the
considered most important in determining the PAS's response to
consonantal
could be
stage
in
(Chapter
sounds.
expanded.)
detail,
The
Four
following
discusses how this model
discuss
will
sections
each
and specify the auditory phenomena that are
being modeled.
The Spectral Analysis Stage
2.4.1
The
bank,
spectral analysis
with
bandpass
cochlear-like
stage
filter
consists
of a
characteristics
frequency mapping.
Twenty
filter
linear
that
filters
a
produce
are
used
to
cover the frequency range of DC to approximately 6500 Hz.
2.4.1.1
Characteristics of the Frequency Transformation
The PAM bandpass
derived
are
filter spacings and frequency responses
from psychophysical
data.
The magnitude
_.__
_
I
__
_C_____II__I___C·_IJ1l___l_1lslll··)
__
of
the
_1_1 _ssY
PAM: A Peripheral Auditory Model
51
frequency response of three adjacent filters are shown in Figure 2.4.
1.
The important features of these filters are:
Center frequencies occur at approximately integral values
of frequency in Bark.
This places each filter at approximately one critical bandwidth from its neighbors on both
sides.
Table 2.1 lists the center frequencies of these
filters in Hz and Bark. As can be seen, 1 Bark intervals
correspond to approximately equal intervals below 1KHz,
and to 1/6th octave intervals for frequencies above 1
KHz.
Note that the number of channels that have been
used
(20) is not based on auditory data: mammalian
cochleas contain several thousand inner hair cells, each
with a different characteristic frequency.
The number of
channels was choosen to minimize computational complexity
while maintaining a reasonably smooth composite frequency
response between 10 and 6000 Hz.
2.
The 3dB bandwidth of each filter is 1 Bark, or one critical bandwidth (Zwicker, 1961).
This corresponds to a
center frequency to 3 dB bandwidth ratio of 6 for filters
with center frequencies above 1 KHz.
3.
The tops of the filters are parabolic for gain measured
in dB, corresponding to the Gaussian-shaped (for gain
measured on a linear scale) masking curves measured by
Patterson (1976).
4.
The low frequency filter skirts fall off at 10 dB/Bark,
and the high frequency skirts fall off at 25 dB/Bark.
For frequencies above 1200 Hz or so, this corresponds to
60 dB/octave and 150 dB/octave respectively.
For frequencies below 1000 Hz, 30 dB/octave and 60 dB/octave are
typical values for low and high frequency skirts. These
skirt characteristics are
derived
from masking
data
(Zwicker, 1970).
5.
The phase response of the filters is linear in Hz, with a
group delay which is identical across all channels. This
choice of a linear phase characteristic is very close to
PAM:
A Peripheral Auditory Model
52
that reported by Pfeiffer and Kim (1975), who measured
the phase and amplitude response of populations of auditory neurons as a function of their characteristic frequency.
Their phase responses come very close to being
linear in Hz, with a slope that is a function of characteristic frequency.
Equalizing the slopes is equivalent
to adjusting the group delay so that the peak response to
bursts occurs at the same time in all channels.
Figures
2.5 and 2.6 present two different views of the phase and
magnitude components of the frequency responses of evennumbered channels.
6.
The gain at each filter's center frequency is equal to
one.
Since the bandwidth increases linearly with center
frequency (in Hz), channels with center frequencies above
1 KHz have an effective 6 dB/octave preemphasis for wide
band signals.
Figure 2.7 shows the magnitude of the frequency response of the individual filters, and Figure 2.8
shows the composite frequency response, both with and
without the preemphasis filter.
7.
Since the bandwidth of the filters is a function of
center frequency (for filters above 1 KHz), high frequency filters provide good temporal resolution at the
expense of spectral
resolution, while
low frequency
filters provide good spectral resolution at the expense
of temporal resolution.
The temporal resolution of the
filters can be inferred from Figure 2.9, which shows the
magnitude of the impulse response of each filter.
As
Searle (1979) and Klatt (1979) point out, the auditory
variation in temporal and frequency resolution produces
conditions favorable for formant tracking at low frequencies and good temporal resolution at high frequencies.
The frequency response of the band pass filters is based
on
psychophysical
masking
data
rather
data--that is, neural tuning curves such a
Kiang
_
(1965).
_
The psychophysical
1_1
than
physiological
those presented by
shapes were used because of
_11-
PAM: A
their
data
Peripheral Auditory Model
53
greater simplicity and uniformity.
that
the
Bark
frequency
scale
It
is
is
from masking
derived.
One
Bark
corresponds to one critical bandwidth of masking noise, beyond
which
increasing
bandwidth no
effectiveness of a masking
This
critical bandwidth
low,
and
is
frequency
generally
scale
longer
serves
to increase
the
signal on a pure-tone test signal.
is wider at high frequencies than at
assumed
reflecting
to
the
correspond
change
of
to
a
"natural"
characteristic
response frequency with position along the basilar membrane.
Because most of the post-processing used in these experiments
involves
recombining
all
of
the
PAM
channels,
unlikely that any of the results reported here depend
exact shape of the PAM filters.
it
is
on the
On the other hand, both basi-
lar membrane and auditory fiber tuning curves exhibit a sensitive,
sharply-tuned
"tip"
(Kiang,
1965;
Khanna
and
Leonard,
1982) that could provide a much greater frequency selectivity
than psychophysical
masking curves
indicate.
This
frequency
selectivity might well be important to model in any investigation of the auditory response properties of vowels.
2.4.1.2
Implementation Issues
Throughout
the
frequency transformation
stage,
the
fol-
lowing conversion formula was used to convert from Hz to Bark:
X = H * 0.01;
= 1.5 + H *
____
__lj14*C___l_____ll
.007;
for H <= 500.0.
for 500.0 < H <= 1200.0
Ii
PAM: A Peripheral Auditory Model
= -32.6 + 6.0 *
Here
X
is
a
particular
corresponding
all
values
frequency
of
H,
and
54
for 1200.0 < H
logl
1 (H);
frequency
in Hz.
in
Bark,
This formula
deviates
from
and
its
which
differs
deviation
from
is
less
than
Zwicker's
is
the
is continuous
Zwicker's
Schroeder's
values by more
tabular
for
values
Throughout this
(1961) by less than 3% between DC and 10 KHz.
range
H
formula
than 5%
(1979)
for
fre-
quencies less than 200 Hz or more than 7 KHz.
Another
of the
advantage
formula
is
that
it
provides
an
explicit center-frequency-to-bandwidth ratio of 6 to 1 at high
for
frequencies
filters,
like
ours,
have
that
1
Bark
bandwidths.
Each bandpass
by
convolving
signal.
The
quency
filter
the
channel's
impulse
response,
were calculated
calculated
in the
from the
impulse
impulse
response
whose
filter bank was
was
magnitude
response
phase
specifications
response
was
with
calculated
and
from
and
the
the
input
fre-
characteristics
presented
windowed
implemented
above.
The
smoothed with
a
256 point Hamming window before being used for filtering.
The
frequency
band
between
DC
and
6493.5
Hz
(sampling
rate of 12987 divided by 2) was spanned using a 256 point DFT.
result, the phase and magnitude of the
As a
of
each
center
---·--L···---····-·11-ilill
-
_
of
the
filters
was
specified
frequency response
every
5.7
Hz.
frequencies of the 20 channels were adjusted to
The
fall on
I _____l_______l_^_a_______m__sr_·Dll
PAM: A Peripheral Auditory Model
55
multiples of this frequency.
The
magnitude
of
the
frequency
response
of
each
PAM
bandpass filter can be expressed analytically as a function of
distance from the center frequency of the channel.
lytic
expression
is
independent
quency is measured in Bark.
in
and
dB,
the
of
skirts:
the
number when
With frequency in Bark
magnitude function has
straight
channel
The ana-
low
and gain
a symmetric parabolic
frequency
skirt
fre-
rises
top,
at
10
dB/Bark, and the high frequency skirt falls at 25 dB/Bark.
In
the PAM, values below -60 dB were approximated as
straight skirts were
zero.
fitted smoothly to the parabolic center.
As a function of DelX, distance in Bark from the center
quency, the
The
fre-
filter attenuation in dB was calculated using the
following formula:
A = -1.92 + 10.0 *
= -12.0 -
25.0 *
(DelX + 0.4);
for DelX <=
(DelX -
for 1.0
1.0);
= -12.0 * DelX * DelX;
The phase of the
filter
can
distance
trast
also
from
with
be
the
the
frequency
expressed
center
for -0.4 < DelX < 1.0
response of each PAM bandpass
analytically
frequency
magnitude
-0.4
<= DelX
of the
function,
the
as
a
function of
channel.
phase
In con-
function
is
independent of channel number when frequency is represented in
Hz.
With frequency in Hz, the phase function is linear: as a
function
of DelF,
distance
in Hz
from
the
center
the phase is calculated using the following formula:
frequency,
I
PAM: A Peripheral Auditory Model
56
P = -PI * DelF / (SamplingRate / DFTSize)
this choice of slope provides a group delay equal to 1/2
size
of
the
impulse
response:
128
samples,
or 9.856
the
msecs.
This means that the peak response to an impulse occurs in all
channels
at
a
delay
of
one
half
the
width
of
the
impulse
response, as evident in Figure 2.9.
The
impulse response of each PAM
using the
following procedure.
phase
around
functions.
the unit
Then
a
was
calculated
First, the value of the fre-
quency response of each channel was
spaced points
channel
calculated at 256 equally
circle
high-pass
from
filter
the magnitude and
was
simulated
by
setting to zero the DC and 50.7 Hz magnitude components of the
frequency
response
of
each
channel.
A
256-point
complex
impulse response was then calculated for each channel using an
inverse DFT, as follows:
I(n) = H(n) * (sum (for i = 0 to 255) of T(n, i))
T(n, i) = F(i) * W(i*n)
where I(n) is the n'th sample of the impulse response, H(n) is
the n'th
coefficient
of a
256-point Hamming window,
the i'th complex DFT coeffecient,
is
is the value of a
point on the unit circle
in the complex plane, at an angle of
(-2*pi*k/256)
I,
numbers.
radians.
Figure
2.9
the PAM filter bank
I
and W(k)
F(i)
_jl I_
_
I_
_I
shows
T,
F,
and
the resulting
W
are
impulse
all
complex
response of
(not including the high frequency preem-
_
_ _
_a__U___P____·_al___
PAM: A Peripheral Auditory Model
57
phasis of the preprocessing stage).
The PAM filter bank itself
is
implemented by convolving
the input signal to the PAM with the impulse response for each
channel.
Each output sample of each channel is the result of
convolving
the
most
recent
samples
256
of
the
input
signal
with the 256 samples of the channel's impulse response.
the
response
impulse
values
sample
are
the
complex,
Since
filter
bank output values are complex.
Examples of The Filter Bank Response
2.4.1.3
As
part
of
the
test
procedure
for
the
PAM
spectral
analysis stage, a test signal was constructed that consists of
the sum of two sine waves of equal
of
one
sine
wave
is
405.8
quency of channel 4.
Hz,
or
amplitude.
precisely
The
the
frequency
center
fre-
The frequency of the other sine wave is
1674 Hz, or precisely the center frequency of channel 12.
signal
is
amplitude
preceded
at
the
by
onset
50
msecs
of
silence.
increases smoothly over
amplitude remains constant for 100 msecs,
and
The
The
signal's
The
5 msecs.
then decreases
smoothly to zero over 5 msecs.
Figure
PAM
2.10
shows
filter bank to this
right.
the magnitude
signal.
Time
response
of the
increases from left to
Low frequency channels are at the top, and high
quency channels at the bottom.
I
of the
_
_II
I
_I
s
_sl___l__l
fre-
Channel center frequencies are
I_
__C______I___1_____II____
_
I
58
PAM: A Peripheral Auditory Model
and to the nearest Hz
shown to the nearest Bark on the left,
on the right.
Below the output response is shown the
same
the
time
to
filter bank has approximately
Since the
scale.
signal,
input
a 10 msec delay, the input signal has been delayed by an equal
so that the input and output signals are time aligned.
amount,
filter bank clearly
The
resolves
the
two
components
of the
test signal.
Figure 2.11 shows a single spectral slice of the response
The ordinate
in Figure 2.10.
shown
of the
it
in dB,
is gain
should be
high
frequency preemphasis,
pass
filters.
low
high
verifies
figure
basic
the
and
frequency
tails
the
effect
the
shape of
of
band
"formants"
of. the
low frequency slopes of the bandpass filters, and
reflect the
the
The
similar to Figure 2.4, with which
This
compared.
is gain
The ordinate of the
in linear units, similar to Figure 2.10.
bottom panel
top panel
tails
frequency
reflect
the
steeper
high
frequency
slopes of those filters.
The Transduction Stage
2.4.2
The transduction
stage of
the
PAM contains
the
primary
nonlinear component in the auditory model.t As shown in Figure
consists
it
2.12A,
saturating memoryless
of
an
envelope
nonlinearity,
detector
which
__
_
_
_
_
I
____p_____L__C___bll__-----·
is
followed
by
a
used to convert
11_11_1
PAM: A Peripheral Auditory Model
59
the detected envelope amplitude into the proper output value.
This
system
is
analogous to the transduction of basilar mem-
brane motion by hair cells.
band
pass
filter
channel
stage.
As
applied
independently
indicated
The
input
signals
in
Figure
to
each
to this
generated
2.3,
this
by
stage
the
is
the
previous
transformation
filter bank channel,
using
is
an
identical detector and nonlinearity for each channel.
2.4.2.1
The
Characteristics of the Amplitude Transformation
amplitude
characteristics
of
the
transduction
stage
are motivated by electrophysiological rate-level data relating
mean neural
firing
rate
to
stimulus
intensity.
The
charac-
teristics of the rate-level data that have been modeled are:
1.
Some auditory neurons exhibit spontaneous firing in the
absence of input stimulation.
The spontaneous firing
rate can range from 0 to approximately 25 percent of the
adapted saturated firing rate (Liberman, 1978).
2.
Auditory neurons exhibit a range of relative thresholds
(input amplitudes above which the output exceeds the
spontaneous by a certain percent) of 30 to 40 dB (Liberman, 1978).
3.
A strong inverse relationship exists between spontaneous
rate and relative threshold (Liberman, 1978).
The population of low threshold fibers is generally co-extensive
with the population of fibers exhibiting a range of spontaneous rates greater than 1 spike per second, while the
population of fibers with a spontaneous rate less than 1
spike per second exhibits a wide range of relative thresholds.
--
lr
I
I"-··-·------
----
PAM: A Peripheral Auditory Model
In a region between threshold and saturation, auditory
fibers respond to exponential increases in the amplitude
of stimulation. with roughly linear increases in firing
rate.
The width of this region, which can be considered
4.
a measure of the dynamic range of the fiber, is typically
20 to 30 dB, but can be as large as 60 dB for fibers with
high thresholds (Liberman, 1978; Evans and Palmer, 1980).
2.4.2.2
Constructing the Transduction Model
this
In
transduction
section we will
stage
static memoryless
linear
from
an
envelope
detector
nonlinearity, and we will
raise
six
followed
show how the
considerations,
conclusion from each consideration in turn.
sion explain and
of
the
by
a
non-
in a well-motivated
function itself can be constructed
To do this, we will
way.
construction
the
justify
drawing
a
These six conclu-
justify our transduction model.
Consider the specifications, listed above, of the desired
characteristics
the
of
the
auditory
Each
of
them
DC characteristics
them specify any phase rela-
None of
system.
state
the steady
specifies something about
of
stage.
transduction
tionships between the acoustic stimulus and the resulting firing
rate.
(As mentioned
in the
Introduction,
the PAM was not
designed to model auditory synchrony.)
Conclusion: The
specifications
correctly establishing
the
above
steady-state
can be
satisfied by
DC characteristics
of
the PAM.
_
___
_1__ 11_·_1_
_
__·______r__r___pq-----
PAM: A Peripheral Auditory Model
Consider
shown
use
the
of
61
As
signals.
transduced output
the
Figure 2.3, the output of the transduction stage is
in
adaptation
further processed by the
see,
As we will
stage.
the specifications for this later stage call for the degree of
adaptation to depend only on the long-term average firing rate
of
the
channel
fastest
time
implying
tion
in
constant
that only
stage
that high
response
low
will
output
to
that
sustained
be
will
stimulation.
is
involved
15
The
msecs,
frequency components of the transducWe will
adaptation.
affect
of
frequency components
also
the output will be
see
passed
through the adaptation stage essentially unaffected.
Furthermore,
the
expected firing rate.
output
of
the
PAM
is
defined
to
be
Although this term has not be precisely
defined, it can be taken to mean the average firing rate over
In fact, the post pro-
a period of at least one or two msecs.
cessors that are used smooth the output signal even more.
Conclusion:
postprocessors,
The following stage
of the PAM,
the
stage may
the PAM
depend only upon the low frequency components
Taken in conjunction
of the output of the transduction stage.
with
and
first conclusion,
this
include a low frequency
means
that
the
transduction
removes
signal
input signal
to the
filter that
components above 100 or 200 Hz.
Consider the characteristics
transduction stage.
__
--
of the
For each channel, this signal is the out-
- i··lsli ·.
I
I
PAM:
A Peripheral Auditory Model
put
a PAM band pass
of
three-dB-bandwidth
the
nels,
filter center
output
periodic
of one Bark.
PAM
signal
is
filter
these
of
a
an approximation, the
as
can
channel
center
the
than
smaller
considerably
at the channel
have
filters
For all but the lowest chan-
Therefore,
frequency.
each
of
All
filter.
bandwidth
filter
62
as
represented
be
a
frequency, modulated by
a slowly varying envelope signal:
O(t) = A(t)
where O(t)
A(t)
is
f
and
complex-valued,
the
is
complex-valued output signal of the channel;
is the
the
* exp(i*pi*f*t)
slowly
of
frequency
center
varying,
the
envelope
It
Hz.
in
channel
signal;
is
important to realize that the frequency components of A(t) are
limited to
filter.
the bandwidth of the
one half
Thus
for
low
channels
frequency
while
quency components below 50 Hz,
channel's
band
pass
has
fre-
only
A(t)
frequency chan-
for high
nels A(t) may contain components as high as 500 Hz.
Conclusion:
each
for
The
channel
signal
input
can
be
approximated
stage
transduction
the
to
slowly modulated
by a
sine wave.
Consider
system
shown
the configuration
in Figure
transduction stage
even
like
simplier
to
2.12B.
shown
Note
in Figure
transduction
investigate.
for an
If
model
we
idealized transduction
can
_
1__----··1·1·--)·Il··lll--C-----
the PAM
it
is
properties
we
would
that
show
-_
_
not
Instead,
2.12A.
whose
is
this
that
I
~
the
_
_
an
DC
_l___~___1
PAM: A Peripheral Auditory Model
63
characteristics of this idealized model has desirable properties, and meets all of our specifications, the model's extreme
its use--at
simplicity justifies
least until we add some more
What we will attempt to
specifications that it can not meet!
do
is
to
that
show
the
under
conditions
output
and
input
described above, the PAM model in panel A is equivalent to the
idealized model in panel B.
This idealized model consists only of a static memoryless
input
signal
a
is
way
of
family
a
collapsing
into a single function T(i).
functions B + T(i)
output that will result from applying the
to
sign'al A*sin(f*t).
input
sinusoidal
a
B to
constant bias
the
Adding
nonlinearity.
saturating
the
nonlinear
of
Consider the
transduction
model
Because the
ideal-
ized model involves a memoryless nonlinearity, the output signal
contain
will
quency of the
components
input,
and
at
the
at
DC,
For
harmonics.
at higher
fre-
fundamental
such
an
input signal, the output characteristics of the model are completely specified by a set of transfer
the
amplitude
There
is
of
each
a DC transfer
functions,
one
for
component
function,
each harmonic.
to
functions
the
and
The
a
input
amplitude
of AC
set
shape
relate
that
A.
transfer
of these
func-
tions do not depend on the frequency of the input signal: only
on its amplitude A.
Conclusion: For the input signals we are dealing with in
the
__I
PAM,
the
____1_II____
DC
characteristics
of
the
Cml
response
__
_
of
11---1_--
the
-·1--.
II
PAM: A Peripheral Auditory Model
64
idealized transduction model depend only
on
the magnitude
of
A(t), and the bias B.
Consider the system shown in Figure
2.12C.
This
confi-
guration is a method for determining the DC transfer functions
of
the
input
idealized
amplitude
labeled
over
transduction model,
A
and
"AVERAGE"
the
a
function
constant offset bias
measures
one period of the
as
the
input
average
output
sine wave.
of
B.
the
The box
of
the
model
Figure 2.13 shows
a
number of different transduction functions that might be used
idealized model.
in this
Figure
2.14
shows
the DC
transfer
functions that result from using the raised hyperbolic tangent
Each
function.
line
in Figure 2.14 corresponds to the use of
a different bias B.
The lines are
B
We
that
were
used.
will
labeled with the values of
consider the
characteristics
of
these transfer functions in more detail below.
Figure
2.15 shows
the DC
transfer
functions
that
result
from using each of the transduction functions of Figure 2.13.
Although specific details of the
functions
change
from panel
to panel, the broad characteristics of the functions, such as
their dynamic range,
their saturation value,
tive
very
threshold,
underlying
erated.
are
transduction
The
one
insensitive
function
exception
to
from
this
transfer functions generated from the
zero lower asymptotes.
·-
1-
---11--11
to
the
which
is
and their
that
shape
they
relaof
were
the
gen-
none of the DC
step function have
non-
The reason for this will be discussed
__
_ICI
PAM: A Peripheral Auditory Model
55
in a moment.
Conclusion:
transduction
The
function
specific
does
shape
not have
of
the
a strong
underlying
effect on the
shape of the resulting DC transfer function.
Consider again the implementation of the PAM transduction
stage shown in Figure 2.12A.
tion used in the model
from
the
instance,
envelope
test
setup
one
of
detector
the
Assume that the nonlinear func-
is a DC. transfer
shown
in
curves
extracts
function
Panel
C of that
shown
in
the
short
amplitude from the filter bank output.
Figure
term
generated
figure:
for
2.14.
The
channel
signal
This amplitude is then
transduced using the appropriate DC transfer function.
Thus
the PAM transduction stage synthesizes the low frequency component of the
idealized model's output signal from the input
signal's amplitude and the appropriate DC transfer function.
Conclusion: The PAM transduction model, when used with a
calculated DC transfer function, will generate a slowly varying output signal that is the DC component of the output of
the idealized transduction model.
2.4.2.3
Analysis of the Transduction Model
In the previous section the underlying nonlinearity used
-
in the idealized model was referred to as
the
"transduction
function".
PAM
transduction
II
The
P---·
nonlinearity
-- ·----·-------------
used
in
the
l1
PAM: A Peripheral Auditory Model
is
the DC
is
stage
referred
in
this
and
that
emphasized
in
use
the
2.12B
main-
should
idealized
to generate transfer
functions
made
is
of
Only
model.
PAM
the
stage,
by
generated
idealized
the
model
are the PAM equivalent of physiological rate-level curves.
with those
closely
tions
the
on
characteristics
compare the
a moment we will
curves, but
first
dependence
of the
the
be
is used to process speech.
functions
transfer
the
the
transduction
PAM
shown in Figure 2.12A,
The
is
It
as
be
will
sections.
that
use
or
function"
designations
These
only
the
"transfer
following
model shown in Figure
for
the
as
nonlinearity".
"amplitude
tained
either
idealized model, and
function of the
transfer
to
66
it
characteristics
In
functions
of our
is worth while examining more
of the DC
shape
of the
transfer
underlying
func-
transduction
functions that are used to generate them.
at
Looking again
shown
functions
have
asymptote
of two.
for
an
All
region"
function varies
asymptote
of
the
zero.
around
from a small
the
to
we
asymptote
negative
input value of
"transition
tive
a
2.13,
same
transduction
of potential
examples
Figure
in
functi6ns
the
can
see
that
of
zero,
and
of
all
a
the
positive
functions have an output of one
function also has
Each
zero,
a
amount
amount
region
(say 0.1)
the
below
within
a
finite
which
above the
positive
the
negaasymp-
tote.
_I
_
_
_
__1______1_________1______
_
_Dls·_l
67
PAM: A Peripheral Auditory Model
The
provide
selected
to
negative
asymptote
functions
negative and
of the
value
of
have
ciently small input
value
a
were
functions.
The
transfer
resulting
that
ensures
zero
themselves
will
transfer
"normalized"
asymptotes
positive
suffi-
for
zero
of
signal amplitudes and large negative bias
values B (see Figure 2.14, B = -16, input amplitudes less than
20
dB).
of
asymptote
positive
The
two
ensures
the
that
"saturation value" of the resulting transfer function (output
level for large input amplitude) will be one (see Figure 2.14,
any curve, input amplitudes greater than 80 dB).
the
Changing
value
simply adds
transduction
function
asymptote of
the resulting
value
the
the
of
transfer
positive
these
functions:
changes
effect
the
only the vertical
to
the
of
asymptote
an offset
the negative
functions, while changing
asymptote
saturation value of the transfer
of
negative
the
of
adds
functions.
dynamic
to
offset
an
neither
However,
range
scale and offset
of
of
the
the
transfer
those
func-
tions.
Similarly, changing the horizontal scale of the transduction
(the width of the transition region,
function
the dynamic range of the transduction function),
the
horizontal
(the
offset
relative
threshold)
and hence
only changes
of
the
gen-
erated transfer functions, and-not their dynamic range.
_
161 - Y---
iapl aaa
a-
--i--r
1l
PAM: A Peripheral Auditory Model
68
The only two remaining aspects of the transduction
tions
func-
that might effect the shapes of the resulting transfer
functions are the shape of the nonlinearity, and the DC offset
(the operating bias B) of the input sine wave.
In the ideal-
ized model, only bias values less
than or equal to zero yield
meaningful
The
transfer
functions.
effect
of
varying
the
operating bias is shown in Figure 2.14, using a raised hyperbolic tangent
function as the transduction nonlinearity .
effect of varying the
is
shown in Figure
shape of
Each curve in these figures is the
2.15.
resulting
nonlinearity
amplitude
nonlinearity
transduction
the
The
from
a
transduction
given
nonlinearity and a given operating bias B.
Figure
transfer
functions
level
curves.
fall
outside
increases
the
the
in
common
all,
for
operating bias
relative
transition
threshold
threshold between the B = -16,
This
makes
with
sense,
because
B
input
region,
and
values
are
reach
transduced
_
I__
bias
transfer
the 12 dB change
in
-64, and following curves).
left
into
greater than zero.
the
resulting
doubling
the
ratethat
of
the bias means that the
input amplitude must be twice as large before
tive
DC
values
doubling
the
the
observed
region,
(observe, for example,
function by 6 dB
of
characteristics
much
have
First of
of
the
that
shows
2.14
side
output
of
values
the most
the
posi-
transition
significantly
PAM: A Peripheral Auditory Model
Once the
input
it continues to
than those
region,
grow until,
needed
the
amplitude
69
reaches
the
transition
region,
for input amplitudes much larger
to
reach
the
positive
half
of
right
the
side of
input
cycle
the
transition
is
transduced
into values close to two, while the negative half of the input
cycle
is
transducted
into
values
close
to
zero.
Thus
for
input amplitudes much larger than the width of the transition
region, any transduction
function looks
like a step function.
The average output value of one input cycle
is the saturation
value of the transfer function, which approaches one.
For operating bias values within the transition region of
the transduction nonlinearity, the transduced DC output value
for a sine wave of zero amplitude is greater than zero.
This
"quiescent value" of the DC transfer function corresponds to a
nonzero
spontaneous
quiescent
value
value
of the
of
firing
the
rate
in
transfer
transduction
function
auditory
function
at the
is
neurons.
The
precisely
the
operating bias
B.
This quiescent value is the sole source of the final non-zero
spontaneous rate in the output of the PAM.
tion stage
transforms
The later adapta-
the value of this quiescent level, but
does not, in itself, give rise to any spontaneous output.
The
presence of a transition region ensures that there are values
of B which result in transfer
cent values.
(In Figures
functions with non
quies-
2.13 and 2.15, only the step func-
tion does not have a transition region.)
_
zero
I
I
PAM: A Peripheral Auditory Model
Note
quiescent
that
all
values
of
the
have
76
transfer
approximately
functions
the
same
with
nonzero
relative thres-
hold, while all of the transfer functions with higher relative
thresholds have a quiescent value equal or very close to zero.
This
satisfies
item
three
in
our
original
list
of
desirable
characteristics.
Thus changing the operating bias
on
the
shape
varying B
of
the
from zero
transition region
value of
resulting
to
much
causes
the resulting
transfer
less
the
B has a
than
relative
transfer
manner that relative thresholds
distinct
function.
the
In
fact,
half-width of
threshold
function
effect
the
and quiescent
to vary in
and spontaneous
the
same
rates of audi-
tory fibers are seen to vary.
The dynamic
as
the
range
of
range of
input
tion varies between,
between
its
a
transfer
amplitudes
say, 10
quiescent
and
function
defined
transfer func-
and 90 percent of
the difference
saturation
values.
shown
The
in
dynamic
Figure
2.11
around 20 to 30 dB.
The
function
one
remaining
that
might
with a variety
cates
shown
-
be
for which the
range of each of the transfer functions
is
can
- --
of
be
characteristic
is
varied
possible
its
of
the
shape.
transduction
transduction
Experimentation
nonlinearities
that the basic characteristics of the transfer
in
Figure
_-_ _
2.15
----
are
- _
relatively
--
insensitive
____
indi-
functions
to
the
__
_
IC
PAM: A Peripheral Auditory Model
Changing the transduction non-
specific nonlinearity chosen.
transfer functions around their
dent
of
amplitude
is
so high
that
of
shape
shape,
any
resulting
the
of
shape
The
level
saturation
function
transduction
the
the
threshold.
around
functions
transfer
effect on
an
linearity only has
71
indepen-
is
the
because
the
input
function with
transduction
a
transition region of the size shown in Figure 2.13 essentially
acts as a step function.
2.4.2.4
Selecting Values for the Transduction Parameters
We have shown that the only characteristics of the idealized transduction model that significantly affect the shape of
the
resulting transfer function is
the presence of
a
transi-
tion region in the transduction function, and the value of the
offset bias B.
The raised hyperbolic
transfer
function shown
in Figure 2.13 was selected as the standard transduction function in the PAM.
This
values B to select,
leaves
the
question of how many bias
and what their values (and thus the rela-
tive thresholds and quiescent values of the resulting transfer
functions)
should
be.
Delgutte
variations
in relative threshold can
models
a
as
principled
way
to
has
(1982)
be
extend
range of the model, by assuming that
rons have different operating points,
employed
the
1
1 6-~~~~~~~~~~~~~~~~~~~-
in
apparent
different
that
auditory
dynamic
auditory
neu-
and modeling the output
of an ensemble of neurons with a weighted
.W
suggested
sum of three curves
1l
PAM: A Peripheral Auditory Model
72
with different thresholds and dynamic ranges.
For the sake of
simplicity, however, the PAM was implemented using only a single
modeling
function,
transfer
a
single auditory rate-level
curve.
A value of -1.1 was selected as the standard PAM bias
value.
This
bias
results
in a quiescent response of 20 per-
cent of the saturation value, and a dynamic range (as measured
between
The
10 percent and 90 percent of maximum) of 22 dB.
resulting amplitude nonlinearity is shown in Figure 2.14.
Implementation Issues
2.4.2.5
The DC transfer functions generated by the raised hyperbolic
tangent
function
discussed
above were calculated for a
set of bias values and stored in a data
file.
The value
of
each transfer function was calculated at one dB steps of input
signal amplitude, between -20 and 100 dB (re 1.0).
When
the
PAM
is
processing
the
signals,
amplitude
non-
linearity function for the bias value chosen (typically -1.1)
is read
from the data file and stored in an array.
For each
output channel of the filter bank, the magnitude of the chanto dB
nel response is calculated and converted
resulting amplitude
value
is
amplitude nonlinearity array.
then used as
an
(re 1.0).
index
The
into the
The linearly interpolated value
of the transfer function at that amplitude becomes the output,
for that channel, of the transduction stage of the PAM.
I __
~~
_
~~~~
_
1II
--------_-____ _______
PAM: A Peripheral Auditory Model
The input signal
is
the transduction stage.
samples/second,
Nyquist rate
or
73
down-sampled by a
facter
of six
in
The resulting sampling rate is 2164.5
.462
msecs/sample.
This
rate
is
the
for the envelope signal of the highest channel,
which has a pass band that is 1082 Hz wide at the -3dB level.
Since all of the lower channels have narrower pass bands, they
continue to be oversampled at this rate.
2.4.2.6
Examples of the Transduction Stage Response
Figure
2.16 shows
the response of
the
combined
spectral
analysis and transduction stage to the test signals described
earlier.
the
Note
The most obvious
significant
that
channel
the
quiescent
feature of the response pattern is
output
transduction
amplitudes
(for
stage
effectively
to a "log magnitude" scale.
the apparant width of the two "formants"
2.16
than
zero amplitude input).
in Figure
2.10,
and
converts
As a result,
is greater
the onsets
the
in Figure
and offsets appear
faster.
Figure 2.17 compares a spectral
nitude
of the PAM
tion stage.
has been
slice of the output mag-
filter bank to the output of the transduc-
In the top panel
converted to dB.
the output of the
filter bank
In the bottom panel the output of
the transduction stage is shown in linear units.
The limited
dynamic range of the transduction stage is clearly evident.
_
"*
---l--·CIII1·--------I
·
H
PAM: A Peripheral Auditory Model
The Adaptation Stage
2.4.3
The
74
adaptation
amplifier
level
of
subtracts
which
input signal.
input
stage
The amount
in
the
the
PAM
acts
as
a time-varying
of
offset
recent past.
a
constant-gain
DC offset
depends
Negative
on
from the
the
output
average
levels
are
clipped to zero, since the output of this stage represents the
expected
firing
rate
of
auditory
neurons.
As
indicated
in
Figure 2.3, a separate adaptation amplifier is associated with
each channel of the PAM,
and it is
the amplitude
of stimula-
tion in each channel that determines the instantaneous
of
channel's
that
amplifier.
However,
the
offset
inherent charac-
teristics of all of the channel amplifiers are identical.
Characteristics of the Temporal Transformation
2.4.3.1
The temporal transformation models the decay of auditory
neural
firing rates over
tic stimulation.
time in response to sustained acous-
The characteristics of adaptation that have
been modeled are:
1.
The maximum firing rate for an abrupt amplitude onset is
approximately 2.5 to 4 times the adapted maximum, which
is around 200 spikes/second (Smith and Zwislocki, 1975;
Delgutte, 1980).
2.
A range of adaptation time constants can be observed in a
These vary from so called "rapid" adaptasingle neuron.
tion, with time constants in the 5 to 15 msec range (Delgutte, 1980), to short term adaptation, with time con1975;
Zwislocki,
and
(Smith
ms.
40
around
stants
__
Ii
-
-
PAM: A Peripheral Auditory Model
75
Delgutte, 1980), to slower adaptations, often referred to
as fatigue or long term adaption (Kiang et al., 1965;
Young and Sachs, 1973; Abbas and Gorga, 1981) with adaptation times in the hundreds of msecs up to many seconds
or even minutes.
r
3.
A step increase in signal intensity results in a step
and the magnitude of the
rate,
in
firing
increase
response increment is independent of the degree of adaptation (Smith and Zwislocki, 1975).
4.
Following the abrupt offset of an intense signal of sufspontaneous
cause
adaptation,
to
duration
ficient
activity is initially absent, and then increases until it
reaches its nominal level.
The time constant of this
recovery is somewhat longer than the time constant of
adaptation (Harris and Dallos, 1979), and although different animals exhibit variations in both adaptation and
recovery time constants, the ratio of recovery to adaptation time may be fairly constant across species (Harris
and Dallos, 1979).
5.
A
6.
For a level signal with an abrupt onset, the ratio of
onset firing rate to adapted firing rate is constant
and
rate
from both)
spontaneous
subtracting
(after
independent of input level (Smith and Zwislocki, 1975;
Smith, 1979).
-·1--C-·IIIIPI I
of sufficient intensity and duration to cause
adaptation can mask another signal which follows the
offset of the first signal within a period of time on the
This masking
order of the recovery time of the haircell.
takes the form of a lowered firing rate in response to
the second signal, or even a complete lack of response to
that signal.
signal
--
L--·--"·9"c--··1·-·-LI-------------
I
PAM: A Peripheral Auditory Model
2.4.3.2
76
The single time-constant model
A circuit that exhibits most of the desired properties is
shown in Figure
tion model
2.18.-
(STCAM).
This
is a single time constant adapta-
The independent
(input)
variable
is
the
voltage Vin and the dependent (output) variable is the current
IoUt through the diode.
The input to this circuit is the out-
put of the transduction stage of one channel of the PAM, and
the output of this circuit is the final output
nel of the PAM.
for that chan-
Except for the diode, the circuit is a linear
time invariant high pass filter.
2.4.3.2.1
The
Analysis of the STCAM
incremental
resistance
of
the
circuit
to
increase in Vin is the parallel resistance R a
1
as
current
capacitor Ca charges
up,
the
incremental
Rs
.
a
step
However,
through
the diode falls off to its steady state level with an "adaptation" time constant:
Ta = C a * Ra
The steady state value of Vc, the voltage across C a,
is equal
to Vin: all of the current flows through R s and the diode, and
Iout is equal to Vin / R s .
mental
Note that the tsteady state incre-
current is smaller than the peak transient incremental
current by an amount
I
_
I
77
PAM: A Peripheral Auditory Model
fa
=
Ra
/
(Ra
+
Rs)
The constant fa' which will be referred to as the adaptation
ratio,
is
important
an
which
ratio
repeatedly
occurs
throughout the analysis of this model and its
successor, the
multiple time constant adaptation model.
If Vin is suddenly reduced, Vout, the voltage across the
If Vin is reduced so that
diode, is reduced.
Vin
<
Vc * Ra / (R a + R)
Iout
becomes
discharge through R
no longer
In that case the diode
Vout will become positive.
conducts,
= Vc * fa
zero,
and
the
and Ra in series.
capacitor
begins
to
The time constant of
this "recovery" is:
Tr = Ca * (R s + R a)
This recovery time constant is longer than the adaptation time
which
constant,
correctly
models
described in item four above.
related
to
the
physiological
data
The recovery time constant is
time
adaptation
the
constant
by
the
adaptation
ratio:
Ta = Tr *
fa
It would be interesting to see if this predicted relationship
really exists between the
rates
incremental and steady state firing
auditory neurons,
in
_I
_
_rl(lllllllllra
and
the
adaptation
and
recovery
____
I
PAM: A Peripheral Auditory Model
78
times of those same neurons.
The
diode
remalins
turned
off,
and
the
output
from
the
model is zero, until the capacitor has discharged sufficiently
that
Vc < Vin / fa
at which point Vu
t
becomes negative, the diode begins to con-
duct, and Vc completes its discharge through Ra alone.
2.4.3.2.2
STCAM
Selecting Values
The values of Ca, R,
for the
and R
Circuit
Elements
in
the
are completely specified by
the desired values for the adaptation time constant, the adaptation ratio, and the overall gain of this
Based
on
physiological
data,
an
stage of the
appropriate
adaptation time constant is 40 msecs, and 0.3
tion
unity
ratio.
in the
only, and
channel
The
overall
PAM.
leave
output
the
These
gain
is
for
the
for the adapta-
arbitrary,
values model
value
PAM.
short
and
is
set
to
term adaptation
response in normalized units, such that a
value
of
1.0
represents
the
maximum
adapted
expected firing rate of the neurons modeled by that channel.
2.4.3.2.3
Figure
Response of the STCAM to Simple Signals
2.19
stant adaptation
_
shows
the
circuit
response of
the
single
time
con-
to an input stimulus whose envelope
11
·11_---11--
1
II
I.niiijCllp/sZa)l····-·i·--··-i
PAM: A Peripheral Auditory Model
consists of a
quiescent
level
79
followed by a
step onset
lowed by a step offset back to the original quiescent
Panel A shows the input signal, which
fol-
level.
in the PAM would be the
output of the transduction stage, while Panel B shows the output
of
adaptation model
the
itself.
computer model
(The
of
the adaptation circuit has been initialized with the capacitor
charged to the full voltage it will achieve in response to the
quiescent
input
level
that
increases
sharp
is
in a
transient
state level
level.
its
Thus
steady
the response
state
higher
falls
than the
circuit to
When
response.)
step-wise manner,
which
of the
the
input
the circuit responds with a
off
exponentially
original
level.
to
a
steady
When the
input
returns to its quiescent level, the response falls to zero for
an
interval
We
shall
state",
during which
refer
to
the
because during
time
state
this
the capacitor
of
interval
the relatively high input that
ing,
the
output
current
zero
builds
output
the
preceded
back
is
up
discharging.
as
the
input
it.
"masked
is masked by
Following
to
its
mask-
pre-stimulus
level.
The
dotted
line
in
Panel
B
shows
the
value
that
Iout
would have at each point in time if the diode were to be short
circuited for an instant.
In other words, when the
conducting, the current through it is equal to
Iout =
(ViWRs ) +
(Vin-Vc)/Ra
__ _
i
diode
is
I
PAM: A Peripheral Auditory Model
The
dotted
the
diode
line
value
this
shows
80
is not conducting.
that
during
even
As we will
see,
time
when
this value can
be thought of as a hypothetical baseline quiescent response on
top of which incremental responses to stimuli are built.
Figure
adaptation
circuit:
constant
is
signals
input
of eight
its
incremental
A
series
consists
signal
Each
shown.
gain.
of
initial step onset followed by a secondary step increase.
initial
interval between the
ranges
from 0
secondary
ing
of
response
are
stage
equal.
and
follow-
Panel
A
shows
the
Panel
B
shows
the
As can be seen, the size of
response of the adaptation stage.
the incremental
amplitudes
their
stage,
transduction
the
that
so
adjusted
transduction
the
The
The intensity of the primary and
to 280 msecs.
steps are
an
increase
secondary
and the
onset
the
of
property
important
an
demonstrates
2.20
adaptation stage to the
response of the
secon-
dary step is the same regardless of the time delay between the
the
adap-
tation circuit is not an automatic gain control circuit:
if it
primary and secondary
were,
the
effect
together,
primary
to
a
demonstrates
step
and
stimulus
with
after the circuit had adapted to the
Figure 2.21 is intended
linearity
__
in the
PAM.
The
_C__1
would
the
two
be
to
steps
second
first
reduce
the
occurring
step occurring
step.
the sources of
to elucidate
PAM
that
the circuit would respond
the
with
stimulus
a
to
than
of the
secondary step,
response to the
differently
This
step.
non-
responses to eight signals are
I
IT_
--
81
PAM: A Peripheral Auditory Model
in
stage
input to the transduction
shows the
Panel A
shown.
dB, panel B shows the output of that stage, and panel C shows
All eight signals have an
the output of the adaptation stage.
intensity
initial
of
dB.
-30
50 msecs
At
the signals.
in
increments
increase to -6 through 30 dB in 6 dB
is
there
a step
seven of
One of the signals continues at -30 dB.
By com-
paring Panels B and C, it can be seen that the nonlinearity in
the
onset
transient
is
On
stage.
transduction
due
the
solely to the
of the
masking
period
the
hand,
other
saturation
which follows the offset of the step increase is solely due to
Note that the more intense the preced-
the adaptation stage.
ing stimulus, the longer the duration of the masking period.
Figure
masking
the
demonstrates
2.22
response of the adaptation circuit.
The
stimuli.
of
output
Panel A, and the output
The
dotted
response level
of
offset
response
in
line
as
the
by
followed
stimulus,
intense
the
B
it builds
masking
falls to zero
there
no response
is
the
probe
to
in
adaptation
stage
in
Panel
B.
the hypothetical baseline
up
back
to
the
zero.
for a period of time,
is
to
_.·
Following the
adaptation
circuit's
and then builds
During this time, its
greatly
reduced.
In
fact,
the probe signal occuring 10
msecs after the offset of the masking signal.
_
probe
shown
stimulus,
at all
msec
is
shows
stimuli
1
stage
back up to its initial quiescent level.
response
of
transduction
of the
Panel
The input is a 150 msec
series
a
the
in
effect
11_1
Only
after 250
PAM: A
msecs
Peripheral Auditory Model
of
recovery
intensity
to
the
time
does
probe
82
the
signal.
circuit
response with
It
be
can
seen
full
that
the
reduced response is due to the negative level of the hypothetical baseline quiescent response,
upon which all
incremental
responses are built.
Figure
2.23 shows the effect of masker
degree
of
higher
amplitude
of
masking
turn results
probe
following
signals
cause
amplitude on the
signals.
greater
As
adaptation,
expected,
which
in
to probe signals that occur
in smaller responses
within the recovery period of the adaptation circuit.
2.4.3.3
The Multiple Time Constant Adaptation Model
Although the single time constant model meets many of the
criteria set forth for the adaptation stage,
it does not exhi-
bit the variety of adaptation time constants that can be seen
in
auditory
electophysiological
constant is set around 4
If
data.
the
STCAM's
time
msecs it can be used as a model of
short term adaptation, but then it does not model either rapid
or
long
term
adaptation.
Obviously,
a model
with multiple
adaptation time constants is required.
Presumably the
pairs
to the circuit
way to do
this
in Figure 2.18.
is
to
following
two
conjectures
__1---·1171.
additional
RC
There are many ways of
adding these additional circuit elements.
the
add
We have made use of
from a survey of electropysio-
--._-.
-·I-·--·
__
I___
___C
PAM: A Peripheral Auditory Model
83
logical data in choosing our method:
1.
Before and during adaptation, all RC pairs appear to act
in parallel: the response rate associated with each time
constant appears to be added together.
To put it another
way, as the transient period for rapid, short, and long
term adaptation is in turn exceeded, a certain portion of
the
response
rate--associated
with
those
adapted
portions--is subtracted from the overall response, until
at last only the steady state portion of the response
remains.
In circuit terminology, the resistors of the RC
pairs appear to be in parallel.
2.
During periods of recovery, all RC pairs appear to act in
series: the shape of the response deficit appears to be
the sum of the deficits associated with each of the time
constants.
As each of the adaptation portions recover,
the response rate is boosted toward the spontaneous rate.
In circuit terminology, the capacitors of the RC pairs
appear to be in series.
Based on these two hypotheses, we have augmented our sin-
gle
time constant model as
source
and diode remain
resistor
Rs
shown in Figure 2.24.
unchanged.
through which the
steady
There
is
three RC
element pairs
implemented with as
are
many as desired,
adaptation time constants.
still
state current I
generating the nonzero steady state response
Although
The voltage
shown,
of the
a
s
shunt
flows,
circuit.
the model can be
to model
any number
of
The element values must be chosen
such that
Ta 0
= R
C0
is the fastest adaptation time constant,
I
_
_I
__
I_
_
_
Ii
84
PAM: A Peripheral Auditory Model
Ta
1
= R
1
*
1
is the second fastest, and so forth.
Analysis of the MTCAM
2.4.3.3.1
Consider a version of the MTCAM with n+l
Let R n
constants.
resistor and capacitor that
be the
and C n
adaptation time
produce the longest time constant,
and C i be the
and R i
ments which produce the i'th time constant.
ele-
Similarly, let I i
and Vc i be the current through R i and the voltage across C i ,
and Is
be the
model
can
Then the operation of this
current through R s .
be
considering
by
understood
in Vin.
response to a step increase
incremental
its
The incremental response
is determined by replacing the capacitors with short circuits.
Then the incremental current flow through the diode equals the
through R n ,
increment in Vin divided by R
lel.
the
Thus
relative
in paral-
and R s,
contributions of each time constant,
and fa' the relative size of the steady state response to the
transient
response,
is
determined
solely
values of the resistors in the circuit.
by
relative
the
In all of the
cases
we shall discuss, the transient associated with each time concontributes
stant
equally
the
to
overall
response.
Conse-
quently all of the adaptation resistors have the same value.
current
As
response
to
the
flows
through
increment
the
adaptation
in Vin, the
resistors
capacitors
adaptation
_
______
_
__
llsllli--
-1..------
I_
in
---IIC·l
PAM: A Peripheral Auditory Model
85
charge up, with the voltage across C 0
C 1,
etc.
The capacitors
charge
increasing fastest, then
up until
them equals the
voltage drop across the
tion resistor.
(e.g.,
C
charges
drop across it, equals the
happens,
current
stops
the voltage
next highest
up until Vc0 ,
adapta-
the voltage
voltage drop across R 1 .)
flowing
across
When that
through that RC element pair,
except for a small negative current as the voltage across the
next highest resistor begins to drop in response to its capacitor charging
up.
highest capacitor,
Eventually, Vc n,
voltage
across
the
equals Vin; the voltages across all of the
other capacitors equal zero;
are
the
carrying current;
Rs
none of the adaptation resistors
is carryihg all of the current;
and
Vout , which equals Vc n - Vin when the adaptation resistors are
not conducting, is zero.
If now Vin suddenly drops
a
voltage
decrease
zero.
drop
across
in Vin
is
the
large
(becomes smaller),
adaptation
enough,
Vout
Vc n
resistors.
can
be
creates
If
driven
the
above
This occurs when
Vin < VCn * fa
where
fa
=
/ (Ra + Rs)
Ra is the parallel combination of all of the adaptation resistors:
Ra = Rs I1
-----------
r?qrrrar---
R1
...
Rn
--·
PAM: A Peripheral Auditory Model
When Vin
and
falls
below
the diode
this
86
threshold, Vout
stops conducting.
becomes
positive,
As long as the diode is not
conducting, C n will discharge itself through the series combination of R s and Ra
As before,
turn, until
each of the capacitors will charge up in its
its voltage equals the voltage across the adapta-
tion resistor above it in the ladder.
slowly as
the time
tion,
that RC pair charges up.
constant of charging
because
the
resistance
is
Then it will discharge
During recovery, however,
longer than
through which
the adaptation resistor R i plus R s
during
Ci
is
in series.
adapta-
charged
is
Therefore the
recovery time constant Tr i will be longer than the corresponding adaptation time constants Tai:
Tr i = C i * (R i + R s ) > Ta i = C i * R i
To demonstrate these relationships, consider the two time
constant circuit
shown in Figure 2.25.
Figure 2.26 shows the
way this circuit responses to a signal with a step increase in
intensity followed some time later by a step decrease.
signal is shown in Panel A of Figure
tialized
so that Vc 1
= Vin, and Vc
2.26.
The model
= 0, at t = 0.
Such a
is ini-
Thus dur-
ing the first 50 msecs, the circuit is in steady state, all of
Iout flows through R s,
and Iout = I s
(Panel B).
1-11
---
.-
87
PAM: A Peripheral Auditory Model
Is
conducting,
is
diode
Indeed,
to be proportional to Vin.
continues
the
(at t = 50 msecs),
step increase in Vin occurs
When the
it
must
now
current
However,
be.
as long as
begins to flow through the adaptation resistors, resulting in
Since C
the current spikes visible in Panels B, C and D.
relatively
I0
small,
quickly
falls
When this
match the voltage drop across R 1.
(at
about t = 60 msecs),
the
decreasing
of
level
Vc 0
slightly negative at this stage.
it
adapted,
At
Vin.
equals
this
state is
to
reached
goes
I0
that
Note
D).
continues to rise until
Vc 1
the
rises
circuit
is
almost
fully
and almost all of the current is flowing through R s.
its
When Vin drops back to
happen.
things
several
point
Vc 0
as
fall again, tracking
begins to
(Panel
I1
to zero
is
diode to
stop conducting.
initial
level
Vout becomes
C1
now
(at t = 200 msecs),
causing
positive,
starts
the
through
discharging
the series combination of R s and Ra, the parallel combination
Once again, Vc 0
of R 0 and R 1.
drop
across
msecs)
as
R1 ,
although
during
it
adaptation,
quickly charges to
takes
about
because
R0
twice
and
Rs
the voltage
as
long
are
now
(20
in
series.
After t = 220 msecs, a negligible amount of current flows
through R,
and C 1 must continue discharging through R 1 and R s
in series.
When Vc 1
equal to zero, the
has
dropped
diode begins
sufficiently
that Vout
is
to conduct again, and I s once
more is proportional to Vin.
_
II
---·l"-CI '-C
IC
IL--"-
II(-·IIP(C-----
i
I!
88
PAM: A Peripheral Auditory Model
Note that at this point Vc
slightly faster rate, because
Vc 1
for
Values
Selecting
2.4.3.3.2
MTCAM
Vin,
equals
a
at
but
the diode effectively shorts out
is concerned.
as far as the discharge current
Rs
C1
still larger than Vin.
is
until
discharge
to
continue
will
1
the
Elements
Circuit
the
in
The desired steady state gain between the input and output of
multiple
the
adaptation
constant
time
the
determines
circuit
value of Rs:
R s = Vin / Iout
in the
For use
that
the
PAM, the overall
dimensions
implicit
of
so
gain has been set to one,
the
values
output
of the PAM
For
channels are "maximum steady state expected firing rate".
is
that channel that
firing rate for
an expected
indicates
0.5
of
value
example, a
one half of the maximum possible
steady
expected
firing
state firing rate.
transient
During
rates
can
go
The
desired
lishes
the
size
step
state
increase
response
value
of the
in
to
of
relative
same
the ratio between R s and Ra
-
------------
the
adaptation
incremental
Vin,
that
onsets,
to
2 or 3 times the maximum steady state
as high as
rate.
a
responses
ratio
peak transient
to
increase.
the
fa
response to
incremental
This
ratio
estab-
steady
determines
the parallel resistance of all
~~~"1~11-"1_ --- C-- ------
II__I___
of
LI_r
PAM: A
89
Peripheral Auditory Model
the adaptation resistors:
Ra = (fa / (1-fa))
* Rs
.3 for the PAM.
The adaptation ratio is normally set to be
The values of the adaptation resistors in the circuit are
determined by the desired partial
difference between the
the
RC pair to
contribution fi of the i'th
size
of the transient
response to an onset and the steady state response to the same
onset:
Ri = Ra / fi
fi = Ii / Ia
where
Ii
is
the
incremental
current
through
R i,
and
Ia
the
incremental current through Ra in response to a step onset.
In the PAM, we normally choose to use equal contributions
RC
from each
pair,
so
time constant model.
Rs =
that fi =
1/3
for
standard
the
three
This gives us, finally:
1.0
* (f
Ri = R
/ (1-f
/
fi
= 1.0 * (0.3 / (1 - 0.3)) / (1/3)
= 1.286
Once the values of the resistors in
the
adaptation
cir-
cuit have been chosen, the values of the capacitors are deter-
1___
_
_
_
PAM: A Peripheral Auditory Model
90
mined by the desired adaptation time constants.
If
the
i'th
desired time constant is Ta i , the correct value for the i'th
capacitor i is:
C i = Ta
i
/
R
i
For most of the use of the PAM discussed in this
ranges
for modeling
short
term
rapid adaptation;
term adaptation;
adaptation.
ranges
for
et al.,
5 to 15 msecs
adaptation time constants were used:
of
1965;
1975; Smith, 1979;
2.4.3.3.3
values
adaptation
Young
to
60 msecs
for
modeling
and 200 to 500 msecs for modeling long
These
modeling
40
study, three
and
Sachs,
are
within
in
auditory
1973;
the
appropriate
neurons
Smith
and
(Kiang
Zwizlocki,
Delgutte, 1980; Abbas and Gorga, 1981).
Response of the MTCAM Circuit to Simple Signals
Figure 2.27 shows the response of the multiple time constant adaptation circuit to signals with ramp onsets of variIn this
ous slopes.
figure three
signals,
and the
responses
of the transduction and adaptation stages to them, are overlaid.
Each signal has a ramp onset from an
-24 dB to +24
dB.
60
Signals
msecs
for
initial value of
The durations of the ramps
1,
2,
and
3
are 5, 2,
respectively.
150
and
msecs
after the beginning
of the onset, each signal abruptly drops
back
second
to -24
after a
dB.
A
5 msec gap.
_
After
repetition
the
of
each
signal
begins
second offset of each signal
_1____11_11__11_______
_.___ C-·91
---·
PAM: A Peripheral Auditory Model
91
there is another gap, this time of 30 msecs duration, followed
by
a third repetition
100 msecs
nal.
B
of the
signals.
The
following gap is
long, and precedes a final repetition of
each
Panel A shows the input to the transduction stage;
the
output
of
that
stage;
and
Panel
the
C
output
sigPanel
of
the
adaptation stage.
The
model
responds
differently
as
onset,
produces
the
strongest
the
different
onset
Signal 1, with the fastest
can be seen in Panel C.
rates,
to
response
from
the
adaptation
stage in Repetition 1, before any adaptation takes place.
The
slower onsets produce significantly lower peak responses,
and
of
the
the
peaks
onset
later
occur
at the
output of
in
However,
time.
the
adaptation
the
stage
slopes
show much
less
variation than the slopes of the inputs:
all three onsets are
very steep,
step-like.
due
both
to
approximately
the
limited
parallel,
dynamic
and
range
of
the
This
is
transduction
stage, and the high-pass filter characteristics of the adaptation stage.
The responses to Repetition 2 are very different from the
responses to Repetition 1.
repetitions
As
all
a
result,
three
leaves
the
the
peak
signals,
The very short gap between the two
adaptation
responses
circuit
are
strongly
significantly
but proportionally much
lower
1--the fast rise time signal--than for Signal 3.
adapted.
lower
for
for
Signal
In fact, the
peak response rates to Repetition 2 are approximately the same
_II___
__lbal·
__II_
___
PAM: A Peripheral Auditory Model
for the three signals.
for Signal 3
The peak response rate in Repetition 1
(no previous
approximately equal to
for
Signal
1
92
adaptation;
60 msec
rise
time)
is
the peak response rate in Repetition 2
(significant
previous
adaptation;
5 msec
rise
time).
The gap between Repetitions
recovery
time
is
sufficiently
2 and
3
is
30 msecs.
long that the peak
response to
Signal 1 is somewhat higher that the peak responses
nals 2 and 3.
(The reason
due
circuit,
which
recover.)
response
to
all
the
first
three
for
Sig-
for the relatively strong response
to Signal 1 is that its peak is
is
This
to
signals
is
to the
still
rapid adaptation
However,
the
substantially
peak
lower
than for Repetition 1.
Before Repetition 4 the gap is 100 msecs, and significant
rapid
and
short
term
recovery
three previous repetitions
adaptation,
has
occurred.
have caused
from which the model has
However,
substantial
long
not yet recovered.
result the peak responses are still less
than
for
the
term
As a
Repetition
1.
In
PAM
examining the
responses
to
them, we
relationship between
the
transition
signals
are
the PAM
between
two
shown in
Figure
interested
responses
speech
in
to
events,
2.28,
and the
considering
speech
and
signals
the
ways
the
at
in
which those responses depend on the intensities of the signals
___1_C_____I__I_1_1__I^---·-·-
I--
1_1__----·---·11·---·1···-1_1_;·n^l·-
93
PAM: A Peripheral Auditory Model
Two
that immediately precede and follow the transition point.
signals
are
in
shown
con-
a LOW level (-24 dB); a MID level
from three levels:
structed
are
signals
These
2.28.
Figure
To make the figure clearer,
(6 dB); and a HIGH level (18 dB).
the LOW level for Signal 1 is actually at -21 dB, and for Sigis
2 the MID level
nal
Panel
A
shows
the
input
B
and Panel C shows
shows the output of the transduction stage;
the output of the adaptation stage.
Panel
stage;
transduction
the
to
dB.
16.5
level
HIGH
and the
dB
4.5
All possible transitions
between preceding and following LOW, MID, and HIGH signal levels can be found in Figure 2.28.
Beginning on the left in Panel C, consider
transition.
PAM
The
transition
a
such
to
response
the LOW->MID
sharp peak because of the rapidity of the transition.
shows
a
t =
At
90 msecs, a visual comparison can be made between the response
to
a
peak
MID->HIGH
transition
and
a
response to the HIGH onset
The
LOW->HIGH transition.
is greater when it
is
from a
LOW level than from a MID level, because of the adaptation to
the MID level.
After 40 msecs, the response to the HIGH level
has fallen off to the point that
to
initial
transient
response
the
LOW->HIGH
transition
as
transition,
the
HIGH->HIGH
it is not stronger than the
the LOW->MID transition.
is
greater
(at
transition
the
than
t
=490
Just
MID->HIGH
msecs)
is
lower than either LOW->HIGH or MID->HIGH.
*-r*O--*arp--
----
-·-^Y·L-X-·IIII.--1
111---··161(·111I····P--_.li-_.
-----
-I_---.
_II_----
-----
I
94
PAM: A Peripheral Auditory Model
to see
is possible
it
In the middle of Panel C (t = 290 msecs)
the difference between the PAM response to a LOW->MID
Because of adaptation,
transitionsand a MID->MID transition.
the response to the MID level is substantial different depending
upon
LOW->MID
whether
in
results
transition
was
level
previous
the
a
stronger
A
MID.
or
LOW
than
response
a
Furthermore, at t = 490 msecs, it can be
MID->MID transition.
seen that the response to a HIGH->MID transition is even lower
than for MID->MID.
Figure 2.29 shows another characteristic of the
PAM:
its
range
than
its
on
the
transient
steady
order
response
response,
state
of
10
for
are
signals
nal
3 has
times
rise
has also been
(Smith and Brachman, 1980).
fibers
All three have rise
shown in Figure 2.29.
a maximum
Signal
stage
intensity of
shown
in
shown
is
and
B;
stage is shown in Panel
is
with
1 has a maximum intensity
Signal 2 has a maximum intensity of 50 dB; and Sig-
transduction
stage
dynamic
characteristic
This
and fall times of 20 msecs.
of 25 dB;
larger
signals
to 40 msecs.
observed in auditory nerve
Three
a
exhibits
in Panel C.
The
the
input
The
dB.
Panel A;
the
output
output of
the
to
to
the
that
adaptation
steady state response to the
However,
three signals are equal.
80
the
transient
response to
t
Signal
1 is
only 85%
This phenomemom
transduction
I-I
-_-- 111-
is
of
due
the
transient response to Signal
to the
stage, which
limited
dyhamic
range
of
3.
the
(as can be seen in Panel B) has the
-.----- illli(ls ·
______
111 _IPI
IIII-PII-----CI_
PAM:
transitions of the more intense sig-
effect of sharpening the
nals,
so
that
their
effective
shorter than their acoustic
the
95
A Peripheral Auditory Model
peakiness
of
the
Note also
circuit.
wider the
rise
times
sources.
This
transient
responses
the more
that
PAM response to
it:
are
in
of
intense the
another effect
substantially
turn increases
the
adaptation
signal
is
the
of the limited
dynamic range of the transduction stage.
2.4.3.4
In
Implementation Issues
the
implemented
three
using
standard
Circuit element values
parameters
time
PAM a
are
adaptation
constant
equation
difference
calculated
techniques.
from specifications
electrophysiological domain: maximum
in the
is
model
of
steady
state firing rate; ratio of maximum transient to steady state
and
rate;
adaptation
time
charge on each capacitor)
PAM channel,
and
Vin,
the
voltage
are maintained
are updated
These input sample
values
source
State
constants.
variables
separately
(the
for each
for each sample of each channel.
are used as
shown
in
the
Figure
sampled values of
2.24.
In test mode
the output of the circuit can be set by the user to be any of
the
voltages or currents of the MTCAM, but in normal use the
output is taken to be Iou t,
_I
II
the current through the diode.
I1
96
PAM: A Peripheral Auditory Model
Complete Example of the Adaptation Stage Response
2.4.3.5
Figure 2.30 shows the response of the complete PAM to the
two-sine-wave test signal discussed earlier in this chapter in
Many
the sections on the filter bank and transduction stages.
analysis;
transformation
PAM
important
the
of
output;
spontaneous
non-zero
are
evident:
frequency
masking;
adaptation;
The effects of adaptation
and recovery of spontaneous output.
are shown even more clearly in Figure 2.31, which, in Panel B,
from Figure 2.30 before, during,
taken
slices
spectral
shows
The locations of the spectral
and following the test signal.
shown
slices
in
2.31
Figure
are
marked
in
2.30
Figure
by
arrows above channel 1.
The spectral slice taken at 55 msecs (before the onset of
the
signal)
test
ately
following
shows
the
a
level
onset,
the
channels
Immedi-
output.
spontaneous
near
the
"formant"
frequncies, which are positioned on channels 4 and 12, respond
strongly, duplicating the response
of the
transduction
shown in Panel A, albeit with a higher amplitude.
stage
As the sig-
nal continues, the size of the PAM response declines, until it
reaches a near steady state value at 140 msecs, 80 msecs after
The curve for
the onset.
after the
171 msecs
offset of the test
shows
Masking
signal.
several channels on either side of the
time passes,
response
I
P
L
these
level,
channels
with
the
the output
1 msec
is evident for
formant channels.
As
begin to recover their quiescent
channels
closest
to
____________
the
formant
1____1·_1__1_____
97
PAM: A Peripheral Auditory Model
peaks recovering most slowly.
POSTPROCESSORS: MEASURING PROPERTIES OF THE PAM RESPONSE
2.5
sampling
rate
microseconds.
The
The
one
is
sample
channel
per
in
signal
the
of
value
signal.
a 20-channel
To review, the output of the PAM is
462
every
channel
each
corresponds to the expected firing rate of an auditory neuron
at
terminating on a haircell
tions
along
the
basilar
one
of
membrane.
includes
no mechanism
for
Thus
explicit
loca-
spaced
the PAM models
the
firing rate,
into expected
transformation of acoustic signals
but
equally
20
property
or
feature
detection.
This section describes some simple postprocessing filters
that
can be applied to the output of the PAM.
These filters
have the effect of simplifying and extracting some properties
of
PAM
the
They
response.
used,
are
in
combina-
different
tions,
in the two experiments described in the next chapter.
2.5.1
A Masking Detector
The
PAM models
the
response
non-zero spontaneous firing rates.
of
auditory
the
duration,
offset
these
of
stimulation
fibers
exhibit
of
with
Such neurons normally fire
even in the absence of acoustic stimulation.
ing
neurons
However, follow-
sufficient
intensity
a complete absence
·----------rr--------
of
--
and
firing
I..._..
PAM: A Peripheral Auditory Model
followed by a second period during which
for a period of time,
firing occurs, but
spontaneous
at
lower
a
period
recovery
this
During
stimulation.
98
rate
before
than
be
can
fiber
the
said to be in a masked state.
In the output of the model, a channel sample value
is
less
firing
fiber
rate represents an expected
than the quiescent output
rate that
is
by
modeled
the
channel,
ipso
and
for
rate
spontaneous
than the
less
that
the
represents
facto,
a
filter was designed to detect
masked state.
A postprocessing
such a state.
This filter operates on each channel of the PAM
output independently, and produces a signal with the same sam-
filter
The possible output values of this
as the input.
pling rate
are
and
1
0.
The
channel
the
between
relationship
input and output is as follows:
if x(n) is less than THRESHOLD;
otherwise.
y(n) = 1
= 0
input
that
value,
corresponds
In
output
output
be
set
value.
anywhere
A
I
use,
with
PAM
zero
between
value
definition
response
.2, THRESHOLD was set to
_ __
is
x(n)
the
n'th
THRESHOLD is a value
lower
to a more conservative
normal
and
value,
channel.
for a particular
quiescent value of
I__
n'th
reasonably
can
quiescent
state.
the
is
where y(n)
_I 11
the
signals
.01.
the
THRESHOLD
of
of
and
masked
having
a
99
PAM: A Peripheral Auditory Model
2.5.2
Two Smoothing Filters
The width
of the
impulse
and
responses,
therefore
the
effective time windows, of the bark filters in the filter bank
vary from 10 msecs
msecs
for
the
for the low frequency
frequency
high
channels to
resolution
This
channels.
1 or
2
is
useful for many purposes, but needs to be smoothed to obtain
or
one
over
averaged
rates
firing
expected
more
glottal
pulses: times on the order of 10 to 20 msecs.
Two smoothing filters were used on the output of the PAM.
convolve the output of each channel with a half
Both filters
Since the left half of the hamming window is
hamming window.
used,
the
effect of the convolution is to smooth the channel
response, weighting recent samples within the window more than
earlier
ones.
In
the
first
experiment
Chapter 3 (the STOP/GLIDE experiment),
the
20
smoothing window is
(the VOICED/UNVOICED
both
the
cases,
msecs.
the
of
be
described
in
the
non-zero width of
the
second experiment
the width
experiment),
output
In
to
smoothing
is
5 msecs.
filters
is
In
down-
sampled to one sample per msec per channel.
Averaging across Channels
2.5.3
In both experiments, the
output
average
the
because
there
_
-
were
---.
from
no
rC
all
final postprocessing step is to
the
grounds
r--r
·----"
channels.
for
treating
was
done
channels
dif-
This
II
PAM:
A Peripheral Auditory Model
each
from
ferently
treatment was
found necessary.
are
postprocessing)
no
and because
other,
The group
delay of
identical,
so
that
Furthermore, the preprocessing
of the PAM
that
ensure
filter
and the
the output
all
chan-
proper, and
events
multi-channel
such as glottal pulses occur at the same time
range
sophisticated
more
the PAM processing
(including preprocessing,
nels
100
in each channel.
limited dynamic
channel
from each
is
weighted approximately equally.
Thus
in both
experiments,
by
derived
signal,
channel
the
final
msec
and
rate,
sampling
value of either firing rate
ing
2.6
singlechannel
signal has a
The output
the
a
expected
whole-nerve
(experiment 1)
of mask-
or degree
auditory nerve.
in the
(experiment 2)
models
is
individual
the
adding
values and dividing by their number.
1
output
SOME OTHER AUDITORY MODELS
The field
of auditory modeling has
Furthermore the
tory.
long
a
and
rich his-
label "auditory model" has been applied
to systems with a wide variety of design goals and performance
In
characteristics.
this
section
discuss
we will
some
audi-
tory models that are related to the PAM by their design objectives, response characteristics, or
discussed
considered
are
to
important
be
in
examples
their
intendqd use.-
own
right,
of many other
have been implemented or proposed.
--
--
I·l)·llllll····L·sl111111·1·1··1111
can
also be
auditory models
Most, but not
__
-·--*
and
The systems
all,
that
of the
_____·__·______1_11_11110_1_1_
teristics
are
characbut
Most,
auditory system.
peripheral
of the
or psychological
physiological
on
based
are
systems
all,
1 1
Peripheral Auditory Model
PAM: A
not
intended to be used in studying auditory response to
speech.
The
developed
system
designed
extract
to
information
chosen
tuning
curves.
Each
wide dynamic range envelope detector
logarithmic amplifier.
(low pass
filter
system
bank
auditory
followed by
a
and
a
filter)
These elements play much the same role
as the transduction stage in the PAM.
this
octave
1/3
to match
is
is
system
in this
filter
bandpass
resulting
the
employing
bank
filters whose characteristics were
fiber
from
analysis
filter
postprocessing
with
speech
frequency
analog
an
by
phonetic
The
response pattern.
performed
of
preprocessing
cochlear-based
Rayment
and
Jacobson,
Searle,
is an example of a system that combines
1979)
(Searle et al.,
by
represents
The digitized output of
Searle's
model
of
the
"information flowing down the eighth nerve to the brain."
The
second
abstraction
half
algorithms
of
this
which,
consists
system
for
CV
input
of
feature
signals, measure
the values of such psychophysically motivated features as high
frequency rise time, voice onset time, and spectral peak location.
In their
describe a recognition
article, Searle et al.
experiment in which features abstracted from CV input signals
were subjected to a statistical discriminant analysis in order
to measure
1
the discrimination
_
_
accuracy of the entire system.
__
_
_I_
_
__
I
hl
102
PAM: A Peripheral Auditory Model
Following training oh 148 syllables, the system identified the
consonant in a further 148 syllables with an overall accuracy
rate of 77 percent.
system
This
1979).
to
attempts
speech
also
signals,
such
jective duration.
is
qu.ency scale
auditory model
analog
front
The
properties
psychoacoustic
loudness, pitch,
as
that
(Zwicker et al.,
postprocessor.
measure
directly
Paulus
an
combines
recognition
digital
a
and
is
in structure
similar
Terhardt,
Zwicker,
by
described
with
is somewhat
system that
A
roughness,
end
of
and sub-
An analog filter bank based on a Bark fre-
followed by halfwave
rectification, smoothing
with low pass filters, and conversion to the log domain.
The
resulting set of "raw" signals are combined in various ways to
the
produce
and
loudness
called
measures
psychoacoustically-related
roughness both within
"specific"
loudness
and
each
of
signal
frequency band
roughness),
and
(so-
overall
As part of this process the
("total" loudness and roughness).
raw signals are fed through a nonlinear circuit which combines
a fast non-adapting onset response to a step signal with a two
time constant decay following signal offset.
Signal qualities
related to timbre are measured by calculating three moments of
the specific-loudness patterns.
The resulting numbers appear
to measure spectral balance, and perhaps "peakiness".
The postprocessing
described
_
as
_ I
in this
"standard
II
-- _
system consists of
pattern
__
recognition
· __·--·11_-_1._-_1_--
III
-------
what
are
procedures."
_
I____
PAM:
unremarkable, perhaps because the
reported, most of which are
and
Chistovich
to
appears
analysis,
The
response
of
high
system
frequency
a
uses
filters,
pass
filter
bank.
adaptation
circuits
an
the
with
tapped
The
the
the amount
period
of
of energy
an
preceding
perception
in
The
time.
the PAM adaptation
minus
a
by
divided
the
input
circuit
stage, in
whose
input
signal.
value
The
adaptation model over the first.
_
_
_
Two
to
a
the
validity
equals
upon
preceding
the
energy
of
an
to be
similar
be
confirm
and
such
depends
during
upon
low
to
fed
output
output equals
researchers
the
the
appears
depends
support
experiments
signal
order
one appears
value
whose
that the
is
adaptation.
in which
coefficient
This
rectified
signal
1974 article:
low
of
to implement the
halfwave
neural
circuit
other
offset
is
slopes
second
of
resulting
models
in
proposed
output
fre-
the
and bandwidths.
filter,
between each
taps
Each
including
sequence
series-connected
(Chistovich
stages
stage models
time delays,
skirts,
automatic gain control
input
fibers,
auditory
circuit which
are
analysis
frequency
transformed.
nonlinearly
adaptation
and
transduction,
frequency
includes
that
system
what
implemented
have
colleagues
her
all-analog
1974).
et al.,
quency
an
be
"preliminary state".
in a
reported to be
system is
are
experiments
recognition
various
of
results
Accuracy
and
103
A Peripheral Auditory Model
to
input
in
the
that
speech
this
second
PAM: A Peripheral Auditory Model
104
These researchers and others
Physiology
in Leningrad
have
1982).
the
combined
Pavlov
of
1974; Chisto-
One of the characteristics of this cen-
tral model is that it detects onsets and offsets
pheral
Institute
this peripheral model
(Zhukov et al.,
with a central auditory model.
vich et al.,
at
frequency channels
in
such
a way as
in the peri-
to mark phonetic
segment boundaries.
Another approach to auditory modeling is direct manipulaA group of researchers at KTH in Stock-
tion of FFT spectra.
holm have described this approach
Blomberg et al.,
1982;
and Granstrom,
in several
1982,
with a standard FFT spectrum, a series
papers
(Carlson
Beginning
1984).
of transformations
can
produce spectra that represent successive approximations to an
"auditory spectrum".
These transformations include converting
to a Bark
frequency scale, converting the amplitude
from a Hz
scale from dB to phons or sones, and
pseudo-spectrum
in
which
the
finally, converting
of
height
each
point
to a
the
along
spectrum represents the width of the surrounding region whose
response
point.
sharp
Thus
spikes
quencies
model
is
"dominated"
a
vowel
at
the
would
includes
the
spectrum
formant
dominate
an
by
wide
unspecified
nominal
would
appear
surrounding
technique
L
11-
as
a
because
frequencies,
tion.
_
frequency
regions.
at
that
number
these
A
of
frefinal
for modeling adapta-
PAM: A Peripheral Auditory Model
These researchers
tion
system to
for
tions
used a standard isolated word recogni-
test the utility of these various representa-
male
standard
The
and
especially
model
The
lowest
accuracy
In
rates.
than any of the
better
reporting
recogni-
consonant
for
had
adaptation
included
which
recognition.
consonant
and
representation was
FFT
tion.
vowel
female
representations,
auditory
105
this work,
one
the
of the
authors
emphasize that it is by no means proven that auditory modeling
provides the best
at
recognition,
which
techniques
from which to attempt speech
representation
least
using
have
proven
the
standard
pattern
with
successful
matching
and
FFT-
LPC-
based representations.
An important issue in
which
model
the
example
of
is
a model
intended to be physiologically valid.
that
physiological
knowledge
presented
Allen
by
uditory modeling is the extent to
is
intended
auditory
about
(Allen
to directly
and
Sondhi,
haircells
1979;
An
incorporate
is
Allen,
the
one
1983).
This model combines a cochlear frequency analysis stage with a
haircell
to
model
cilia
ions
transduction stage.
the
variation
displacement,
across
and
the haircell
of
the
The transduction stage attempts
haircell
receptor
resulting
wall.
In
potential
diffusion
the model
of
this
with
calcium
diffusion
current is taken as a measure of the expected firing rate of
auditory
fibers
terminating
at
the base of the haircell.
addition to testing the model's
_
1.
response to
-
--- ----
simple test
In
sig-
PAM: A Peripheral Auditory Model
106
nals, Allen is using it in combination with a central model to
study auditory response to speech signals (personal communication).
Any
auditory model,
transduction
stage
with
such
the
auditory fibers demonstrate
problem of modeling
as
the
PAM,
limited
same
that
includes
dynamic
range
(20 to 30 dB) must deal
a
that
with the
speech with its much wider dynamic range.
Delgutte solves this problem in his auditory model by using a
parallel combination of three transduction nonlinearities with
differing
thresholds
and
dynamic
ranges
(Delgutte,
1982).
These nonlinearities model fibers with correspondingly varying
response characteristics.
three nonlinearities
is
The composite dynamic
to 80 dB.
close
uses a large number of bandpass
closely
many
of
reproduce
the
other
the
shapes
models
The
range of the
Delgutte model
filters with shapes that more
of
auditory tuning
discussed
here.
curves
than
model
also
The
includes a two time constant adaptation stage that is used to
model rapid and short term adaptation.
The models discussed above calculate the expected
mean)
firing rate of auditory neurons in response to acoustic
stimuli.
mary
In a model developed by Seneff (1983, 1985) the pri-
intent
response
to
is
to
reproduce
the
synchronicity
stimulus 'waveform for
frequencies
half of the speech range (below about 3 KHz).
includes
II
(i.e.,
I
a
peripheral
and
_ CC
a
I__
central
_
model.
of
in
auditory
the
lower
Seneff's system
The
peripheral
PAM: A Peripheral Auditory Model
model
uses
a bank of bandpass
have frequency
responses
The
each
output
compressor"
limit
of
which
the maximum
filters which were designed to
similar
filter
uses
107
a
signal
is
two
to
auditory
passed
time
level.
through
constant
The
tuning
two
an
AGC
time
curves.
"envelope
circuit
to
constants are
approximately three and forty msecs, so these compression circuits
can
envelope
be
said
to
model
compression the
rectified,
so
that
the
neural
signal
adaptation.
in each channel
response
is
Following
is
halfwave
always nonnegative.
The
resulting signal is taken to be a model of the fine time firing rate of fibers in the auditory nerve.
Seneff's central model makes use of a number of "Generalized
Synchrony
Detector"
measures the degree
particular
each
element
of periodicity
reference
auditory
tuned
fiber
to
elements.
frequency.
of
In
response signal
the
fiber's
Each
its
of
these
elements
input signal
Seneff's
central
do
a
good
job
of
characteristic
representing
frequency.
vowel-like
over a wide range of input amplitudes, and
noise.
the
Seneff
same kind
also describes a scheme
of GSD elements
as
model
is processed with a GSD
resulting pattern, described as a "pseudo spectrogram,"
to
to a
speech
The
seems
signals
in the presence of
for using an array of
a pitch detector
that
has
psychoacoustically appropriate response characteristics.
The output signals produced by all of the auditory models
we
_
I
have
discussed have values which represent,
I
to one extent
I
PAM: A Peripheral Auditory Model
or
another,
the
probability
developed by Lyon
which
represent
discharge
time.
in
This
a
1t8
of
neural
firings.
the
presence
particular
bit pattern
haircell
Lyon
model
typically
with a
(1)
or
fiber
is
copied
models
absence
at
a
as
50
of
usecs.
of
model
a
neural
moment
the output of
Allen's
firing
of
a
in
"pri-
input the output of
transduction
more
than
model.
2200
neurons
With bit-stream
signals
auto-correlation and cross-correlation can
be
performed with simple
by
strong
implication,
operations,
after
()
particular
produced as
the
sampling period
such operations
a
(j982, 1984) the output is a stream of bits
mary auditory neuron model" which takes as
a
In
logic
in
elements
the central
in turn, can be
used
in
Lyon's
nervous
to model
model
system.
and,
These
such pyschoacoustic
functions as pitch detection and binaural lateralization.
The models
variety
All of
of
the
discussed
auditory
these models,
that
they
are
with
limited
above
modeling that
including
partial
models
objectives
demonstrate
and
is
the PAM,
of
from
a
the
richness
and
currently underway.
suffer
complex
incomplete
from
system,
data.
the
fact
designed
Neverthe-
less, the field of speech perception is likely to be considerably advanced
through
the
use
of
such
models
to
study
peri-
pheral and central auditory response to speech.
7 ____
_
__·_s_____~
109
FIGURE CAPTIONS FOR CHAPTER 2
FIGURE 2.1
The
Block diagram of preprocessor, PAM, and postprocessor.
shapes the speech spectra to fit within the
preprocessor
amplitude window of the PAM, while the postprocessor measures
particular properties of the PAM response pattern.
FIGURE 2.2
Magnitude of the frequency response of the preemphasis filter.
The vertical axis shows
The frequency axis is linear in Hz.
The diagonal line shows the frequency response
linear gain.
of a perfect 6 dB/octave preemphasis filter.
FIGURE 2.3
The spectral
Block diagram of the Peripheral Auditory Model.
The following
analysis stage is a 20-channel filter bank.
stages are shown for only one channel of the filter bank.
Each channel has a similar and independent set of transduction
and adaptation stages.
FIGURE 2.4
magnitude of the frequency response of three
The horizontal axis shows
adjacent PAM filter bank filters.
frequency shift from the center frequency of the center chanThe vertical axis shows magnitude in dB of frenel in Barks.
These curves apply for any of the PAM
quency response.
bandpass filters, -except for the lowest and highest channels,
where the frequency responses are truncated at DC and the
Nyquist frequency.
Graph
_
of
the
__
I_
I
_
1_1
111---------
11
Figure Captions for Chapter 2
110
TABLE 2.1
List of center
frequencies
of the
PAM
bandpass
filters.
The
center frequencies nominally appear at integral Bark values
between 1 and 20, but are adjusted so that they fall on the
closest multiple of 50.7 Hz, the frequency spacing of the
256-point DFT that is used to construct the impulse response
of the filter bank channels.
FIGURE 2.5
Gain
and
phase
of
even
numbered
Frequency is shown in Barks,
the magnitude characteristics
PAM
filter
bank
chahnels.
and gain is linear.
of all of the filter
Note that
banks are
frequency scale, while
identical on this
phase curves increase with channel number.
the
slope
of
the
FIGURE 2.6
Gain
and
phase
of
even
numbered
PAM
filter
bank
channels.
Note that the
Frequency is shown in Hz, and gain is in dB.
bandwidths of the filters increase with channel number on this
frequency scale, while the
stant.
slopes of the phase curves are con-
FIGURE 2.7
Magnitude
of
frequency
response
of
individual
PAM
filters.
Notice that on
Frequency is shown in Hz, and gain is in dB.
this scale, the filters appear almost symmetrical, although
the low frequency slopes rise at 10 dB/Bark, and the high
quency slopes fall off at 25 dB/Bark.
fre-
FIGURE 2.8
The composite frequency response of
the PAM
solid line is for the PAM filter bank alone.
includes the preemphasis filter.
_
_Illlst-I.YILI.------
11
filter bank.
The
The dashed line
____I__
__111
Figure Captions for Chapter 2
111
FIGURE 2.9
The magnitude of the
Impulse response of PAM filter bank.
The peak response
impulse response of each filter is shown.
in
all
channels.
Each
to the impulse occurs simultaneously
The ringing in channel 19, and
horizontal tick is one msec.
are artifacts resulting
the anomalous width of channel 2,
from the abrupt high frequency truncation of the frequency
These curves include the effect
responses of those channels.
of windowing the impulse response with a Hamming window, but
do not include the effect of the preprocessing stage shown in
Figure 2.1.
FIGURE 2.10
The center frePAM filter bank response to two sine waves.
quencies of each channel in Hertz and Bark are listed Along
the sides. The vertical dimension is output signal magnitude.
The vertical distance between the zero lines of adjacent chanThe input signal,
nels corresponds to a magnitude of 45.
which consists of the sum of a 46 Hz and a 1674 Hz sine wave,
is shown at the bottom, delayed by 9.9 msecs.
This is the
group delay of the PAM filters, so that the input and output
signals are vertically aligned.
FIGURE 2.11
Spectral slices of the PAM filter bank
shows two views of the response of the
same test signal shown in Figure 2.10.
cal dimension is gain in linear units.
cal dimension is gain in dB (re 1.0).
pared with Figure 2.4.
response.
This figure
PAM filter bank to the
In panel A, the vertiIn panel B, the vertiPanel B should be com-
FIGURE 2.12
PAM transduction stage and its derivation.
Panel A shows the
This model synstructure of the PAM transduction stage.
thesizes the low frequency response of the idealized transducThe nonlinearity in Panel A is
tion model shown in Panel B.
the DC transfer function of the model shown in Panel B.
This
_
_I
_ID
P__
1
112
Figure Captions for Chapter 2
transfer function is generated using the measuring setup shown
in Panel C.
See the text for a detailed explanation.
FIGURE 2.13
Four
possible
transduction
functions.
four
The
functions
shown are a raised hyperbolic tangent, a raised half cosine, a
Notice that the transition region
ramp, and a step function.
of the first three functions are all approximately equal, but
that the step function has a zero width transition region.
FIGURE 2.14
DC
transfer
functions.
These
are
functions
the
result
of
using the the raised hyperbolic tangent transduction function
Each curve is the result of setting the
shown in Figure 2.13.
bias to a different value, which is shown next to each curve.
FIGURE 2.15
DC
transfer
functions
for
the
transduction
four
functions
Notice that the shape of the underlying
shown in Figure 2.13.
transduction functions has little effect on the shape of the
near the saturating
from
the
is
clear
resulting transfer functions, especially
It
functions.
amplitudes of transfer
characteristics of the transfer functions resulting from the
step function that the presence of a transition region in the
transfer function gives
zero quiescent values.
rise
to
transfer
functions
with
non-
FIGURE 2.16
sine waves.
PAM transduction stage response to two
The
input
The
signal, and picture format is the same as in Figure 2.10.
The vertvertical dimension is transduced channel amplitude.
the zero lines
ical distance between
corresponds to an amplitude of one.
of
adjacent
_-
-
·
-
-_
-__
channels
113
Figure Captions for Chapter 2
FIGURE 2.17
Comparison of the PAM filter bank and transduction stage
The input signal is the same as in Figure 2.16.
responses.
Panel A, which is duplicated from Panel B of Figure 2.11,
shows the output of the filter bank stage with channel ampliPanel B shows the output of the transduction
tude in dB.
In
stage: another view of the response shown in Figure 2.16.
In
1.0).
(re
in
dB
is
gain
dimension
vertical
the
panel A,
panel B, the vertical dimension is transduced amplitude in
linear units.
FIGURE 2.18
The input variable
Single time constant adaptation circuit.
is Vin, the source voltage, and the output variable is Iout ,
the current through the diode.
FIGURE 2.19
Panel A
Response of the STCAM to a step onset and offset.
Panel B shows Iout·
a function of time.
shows Vin as
Although the current through the diode is zero when the voltage across the diode is negative, the dotted line in Panel B
shows the current that would flow if the diode were to be
momentarily short circuited at any given instant.
FIGURE 2.20
Demonstration of the constant incremental gain characteristics
A
and Panel B shows Iout.
Panel A shows Vi,
of the STCAM.
Each signal
series of eight overlapping signals are shown.
begins at the quiescent value, increases stepwise to the first
plateau, and after a delay of 0 to 280 msecs, rises again to a
Note that the size of the incremental response
higher level.
to the second step is independent of the delay between the
first and second step, and independent of the degree of adapThe size of the
tation that has occurred during that delay.
steps were adjusted for equal amplitude AFTER the
input
transduction stage.
_II
ii
Figure Captions for Chapter 2
114
FIGURE 2.21
Origins of nonlinearfties in the PAM response.
Panel A is the
input to the transduction stage.
Note that the vertical axis
in Panel A is in dB.
Panel B is the output of the transduction
stage.
Note the quiescent response level, and the
saturation in response to higher and higher input levels.
Panel C is the output of the STCAM. Note the additional nonlinearites created by the diode in the adaptation circuit.
The transduction stage is responsible for nonlinearities in
the response to onsets, and the adaptation stage is responsible for nonlinearities in response to offsets.
FIGURE 2.22
Demonstration
of
forward
masking
in
the
responses
of
the
STCAM.
Panel A shows the input to the STCAM.
Panel B shows
the output of the STCAM.
The dotted line in Panel B is the
hypothetical baseline quiescent response.
The initial (masking) signal is intense enough and long enough to cause almost
complete adaptation.
This reduces or eliminates the response
to the following (probe) signals.
FIGURE 2.23
Another demonstration of masking phenomenom, and the effect of
Panel A is the input to
the intensity of the masking signal.
the STCAM, and Panel B is the output.
FIGURE 2.24
Multiple time constant adaptation circuit.
The input variable
is Vin, the source voltage, and the output variable is Iout'
the current through the diode.
FIGURE 2.25
Adaptation circuit with two time constants.
_
ICr
__
Figure Captions for Chapter 2
115
FIGURE 2.26
Responses of the circuit shown in Figure 2.20.
Panel A shows
Vin
Panel B shows
out (including its hypothetical negative
portion) and I s.
Panel C shows Vc 0 and 10, the response of
the 15 msec time constant elements. Panel D shows Vc 1 and I,
the responses of the 40 msec time constant elements.
FIGURE 2.27
Response of the MTCAM to signals with ramp onsets of various
Three overlapping signals are shown, with rise times
slope.
Panel A shows the input to the adapof 5, 20, and 60 msecs.
tation stage, Panel B the output of that stage, and Panel C
the output of the adaptation stage. The gaps between adjacent
signals are 5, 30, and 100 msecs respectively.
FIGURE 2.28
Effect of previous signal levels on the size of the response
of the MTCAM to onsets and offsets. Two different overlapping
Each signal contains transitions between a
signals are shown.
The timing of the transitions, and
LOW, MID, and HIGH level.
their exact size, is adjusted to assist in the interpretation
Panel A is the input to the transduction
of the figure.
stage, Panel B the output of the transduction stage, and Panel
C the output of the adaptation stage.
FIGURE 2.29
Demonstration of the dynamic range of the transient and steady
Three overlapping signals are
state response of the MTCAM.
Panel A shows the input to the transduction stage.
shown.
Each signal has a rise and fall time of 20 msecs Panel B shows
Note that the response
the output of the transduction stage.
to the signals has saturated, but that the response to the
highest signal is wider than the response to the lowest sigThe
nal.
Panel C show the output of the adaptation stage.
incremental response to the three signals are different, even
though the output of the transduction stage has saturated.
This is because the onset of the highest signal is more abrupt
ur
31
Is ·"·I
. --I-----
1.
- `.·1111·-·-------.-
Figure Captions
for Chapter 2
116
after it has been windowed though the transduction stage. The
greater abruptness results in a stronger transient response.
FIGURE 2.30
The input
PAM adaptation stage response to two sine waves.
signal, and picture format is the same as in Figure 2.10. The
vertical dimension is normalized channel amplitude. The vertical distance between the zero lines of adjacent channels
The arrows
corresponds to a normalized amplitude of three.
mark the times at which the spectral slices shown in Figure
2.31 occur.
FIGURE 2.31
Comparison of the transduction stage response, and adaptation
The input signal is the same
stage response at various times.
is duplicated from Panel B
which
A,
Panel
as in Figure 2.30.
of Figure 2.17, shows the output of the transduction stage at
Panel B shows the output of the adaptation stage at
80 msecs.
and 140
55 msecs (before the onset of the signal), 65, 70, 8,
msecs (during the signal interval), and 171, 210, 270, and 390
In both panels, the
msecs (after the offset of the signal).
vertical dimension is normalized amplitude.
FIGURE 2.32
Masking postprocessor response to two sine wave test signal.
The input signal, and picture format is the same as in Figure
2.30. Each channel output value is zero if the corresponding
value in Figure 2.30 is greater than zero, and one if the
The arrows mark
corresponding value in Figure 2.30 is zero.
the times at which the post-offset spectral slices shown in
Figure 2.31 occur.
-F1.
--
II-
---
·-----.
--
111---1..11------
-_ ____~~~~
_~~~~~~~_
~~~~
117
FIGURE 2.1
/I
W
o
.
=
=
a
0
0
0.0
co
%
e
C.L
-.
C=
Il
Z:
I-
=
Q
.
-
uC
0
=
W =c
w
XJ =
W
L-X cC =
-
--ol
0
It.I
II II
" C.
=
LO°=
CL
i
·
A_
C
ui
(V
w =~U
_l
i_.
C.
0C
U
w@
0m
J
0
Cn
W
C
L
b
W.
0.u
.00
-
.
cD
C:):
----
-·I
1·-------
FIGURE
2.2
118
Frequency in Bark
0
20
0
6KHz
1.0
C
L3
ILD
Frequency in Hertz
_
IC
1__1
ll
FIGURE 2.3
119
w
n
.w
w
=
u=
LA.
¢
X_
0
w
mm
S
cU
,S
_
=0
I
C I--
cmi
,I
:3
a=
E_
I-
-
C):
FP:
IIII
_
I£
tl
PWC=
Pz
C=
I
I
I I
........- ·
|
roll
91 -
I--·
I
I
I
I
I
I
391iS SIS AIUs
II I
I
-------------a
I
I
III
IIIII
II
I
I III
I
iiMII
I
I
1UH133dS
III
I
I
I-
C
Z
mlB
I
-
l
FIGURE 2.4
i
11
L
+
m
C
+
I
a)
U
(Y
D
0T
a)
L
Ii
kD
I
!51
Cr)
I
gP u!
_I
I__
___I__·__________VCILI/_R___TCYY-YCI__
U!Le
---.- -----
121
TABLE 2.1
TABLE OF PAM FILTER CETER FREVUENCIES
Center Frequency
Filter
Number
- - -·I
-
I
in Barks
in Hertz
1
2
3
4
5
1.015
2.029
3.044
4.058
5 .051
202.9
304.4
405.8
507.3
6
7
8
9
10
6.117
6.827
7.892
8.957
10.02
659.5
761.0
913.1
1065.
1218.
11
12
13
14
15
10.95
11.94
12.94
14.06
15.00
1421.
1674.
1979.
2384.
2790.
16
17
18
16.01
17.02
18.02
19
19.00
20
19.98
3298.
3906.
4617.
5428.
6392.
101.5
I
122
FIGURE 2.5
_o
Channel 2
Channe
4
_
,e
Io
C
t0
1'D
-
L---
Chnnr
1
l 18
*0
+50
Channel 20
20
a
Frequency
n Brk
_ __l___slll__1______ll___sl_______
_1_1__1_
_____111__
FIGURE
2.6
123
Yes
Chonne 1 2
0
-60
-__
--
Chnne 1 4
-60
9
Channe
aK
-
6
C
C
Qj
wq
tu
-60
LO
0
LL
Channel
-60
Chanrnl 101
-68
0
6KHz
Frequency
_
_b
_
0
6KHz
in Hertz
1_1_
I
_ _1
124
FIGURE 2.7
LD
N
L-
LU
CD
lLJ
[1 1
II
CN
I
I
I
8P
U!
I-
CD
to
I
LI
I
NIU5
III-
-
-
125
FIGURE 2.8
LO
I
I
I
N
I
I
U)
I
/L
Ill
LL
(
C4
---
CN
(
CO
Nt
Lfl
I
I
I
I
I
gp U! NIUS
I-~~~~--ax~~~~~
~~~~~~~~i~~~
· C1Dl~~~~~~r~~~--PiRI
U~~~~~~·
l~~-----
LO
I
FIGURE 2.9
J
126
,
I
1
=
-
X\@s-
I
,
I
T
'
__tra\:_
I 1 1 1 1'
L_
/
-
o
n
t_
|
7TS
_
E
I
-
_
I
I
I
I
g
T
T>-/7
I
=-l
Cu
, II
ca
-
.)Cc.
r,
|
|
o~.
i
,
,
,
,
,C7St
'
|
I
§
§
Tv
-
I
I
I
,
4
ff1
'
I
I - - ->h tS/ ti r \ \,
-T
7
I
I
|
§
-|
g
I
I
T-
, ,-. , .
I
r-
I
I
s
I
I
I
I
I
T--
§
I
\t
I
I
[ . .e
n
\1+'-
- 14
_
(0
g |
I
X <D
1,
§.ll --- TJi|
X
I
T
I
r
-~ ----
--
--r
I
, .
\l
I
I
·.
I
I
--
-1
'
'
'
'
-r
§
W
§
T
-
1
-1--
-r--
g
l
|
|
X
X
g
g
I
|
|
|
. . .
I
,
time in msecs
~-II-
T-
T
1
\4
..
·
-~I
I
I
t
I '---n
W-
20
--
r
l\k
s X
I
, .
-
---
·---
,eI
I
I
I
---i
20
ff11--
FIGURE 2.10
127
BEFiTF
BRRK
HERTZ
l82
1
2
j
·
I
I
I
223
I
3
384
4
4,6
5
587
6
68
7
761
8
913
9
aM='
17-7~~~~~~~~~~~~~
17
t ;t
-""
T" T~
1
T
1""'~
17
7
. 7-77l~",! i~f iI
i
ii7
11,l
-
r
I
i1
MS
1i65
¢~TI
1218
11
1421
12
1674
13
I
'
14
2384
15
2792
2 ·
I
I
;
I
I
I
a
"I--------------------
1979
t
MS
I iI
I ; 1 ;
II1UY/I1UYIII~~~~~~~~~~~~~~~~~~~~~~~~~~~~llrllllllllllili1
2
ggg
MS
3298
MS~~~~~~~~~~~~~~~~~~~~~~~~~~~~'llll
17
is
L7
3986
18
4617
19
5428
20
6392
FIGURE 2.11
128
A. Filter Bank Output
30
a)
-
E1
0
1
Frequency in Bark
20
B: Filter Bank Output in dB
40
m
Ci
0
d.
<E
-40
.
Frequency in Bark
1
_
I_
C-·--·---·-·CI--·
-_--
1111
'
20
129
FIGURE 2.12
A: PAM TRANSDUCTION STAGE
NONLINEARITY
OUTPUT
B: IDEALIZED TRANSDUCTION MODEL
IMPIrr _lr.lAi
OUTPUT SIGNAL
OFFSET BIAS = B
C: MEASURIN6 DC TRANSFER FUNCTION
1__
I
Ii
FIGURE 2.13
130
n
Q.
C)
C
.0"
L
__ __1____1__________1__I__
_
111_·_1__·_·____1_
131
FIGURE 2.14
co
C
C
03
-In
C
4-
tA
-I
wnw I XQW
1i1
FIGURE 2.15
132
i
step
ramp
o
100
C
0
ct
0
a.
1c 00
a
0.
S.
wn
S
w
t
S
44n
a
a
To
0
4-
a
zr
c
M
'I'x
E
r
K
C
.4
S
c
Stimulus Intensity In dB
Stimulus Intensity in dB
sin
tanh
100
a
100
U'
0
a
CL
a
0:0
Co
4I
a
us
I"
S
S
40
a
tA
a
.9
S
x4US
.4
%n
0
0
Stimulu
_
Iensity
Stimulus Intensity In dB
in dB
_ I
_1_1
______1____
____1__1___1______ll
133
FIGURE 2.16
HERTZ
BARK
1
I I I I I. I , , I .
I I I
. . .
. . . .
. I -.
I.
, , .
102
1 1 1
2e3
2
3
I
3W4
4
4i6
5
57
I
6
7
II I
I . . ,
, .III
~ . I ~,
I
~I ~I ~
I . . . . II. I I I
I
llll I
Iu
I
. . . .. . 1 1 1
I
lI l
I I I
iI
i,
III
iI ,
II , III I
.
I
I
t t
II
,I
I i,~T I
I
I Il
,,,
iiii
iiiiii1'
llrlllll
I
. I I I
iiii
i
, t
lll'
1421
l~litiljlillilii
1674
..
..
..
14
. ..
..
. . ..
..
..
..
~~I~~~~~~~
..
1979
...........
2384
III..1..
15
I
I
II
II
I
2798
1 I111
3298
16
[
17
18
~I:
i
i- i ~
II
Ii 4
I
ii
II
,
l
j
ii{iiTiI
~c*e
IIIIIII III
l
I
l
3916
i
[I
I
]
4617
19
29
1065
1218
12
13
913
I I '
ii
rF~r-i'~iliiiillIIIIgl
PR"""I
761
I . . . I' .I
I I I~~rI~lll
1 1 1
I~1
III~~lHI
660
1 1 1 1 1
.
I
II
18
11
,,I
'I,I
i
,( ~
~ ~ 1~~~~~~~~~~~~
t I
-P I i iI -c-~~~r~ald~~~
j~~
8
9
I II.
5428
I II. .
.
69 MS
588 Ms
6392
FIGURE 2.17
134
A: Filter Bank Output in dB
40
m
-o
Cc
0
Li
E
-40
Frequency in Bark
1
20
B: Transduction Stage Output
1
d
E
'0
c)
-o
C'0
C
0
_
lli
20
Frequency in Bark
1
q_ ___
-
1_·111111_1_
FIGURE 2.18
135
Ca
Single Time Constant Adaptation Model
---- ·----
FITUR[d
iTRTION
STRGE INPUT
136
I
Sa
s '
_
3
'UT
a
Ms
/
/
I/
_II
_ 11_·
_.i·I·----I
1E
i__
137
FIGURE 2.2N
R: RDRPTRTION STAGE INPUT
I -
__
_
C·
_I
_
LI
_
_I
_C
I11111 1 1
-- -----
a
0
I
.
.
I
_
_II
_
I
,I :-j
i
I__
,-.
-r --·---.
--·------ r -----I
-. -
5BB as
UT
B
---- ·l-,arra------1,_____1__1___
SBB as
138
FIGURE 2.21
30dB
R: TRRNSDUCTION
STRGE INPUT
..
III II I
- - I
-
-
-
.
.
-
-
.
-
I
-
-w
-
-
_
I
!
-30dB
S00 mI
II
_
_
i i i iii I Iii
iii
ii
Ii
B: RDRPTRTION
STRGE INPUT
i
II
-
0
JJ
I I
iiiii
i
i
........
--
UI
-
_a _I ___' -I
-
iii
_-_ __
._
· I
I
3
_ _t
_ _1
___
I
1
__
·
-
C: RDRPTRTION
STRGE OUTPUT
SOD ms
B
_
.
------------------ -----
~~~~~~~~~~~~-m
139
FIGURE 2.22
R: RDRPTRTION STRGE INPUT
1
rxl~I I I 1 I I I
4----
-
B
- -
*
--
r
----e~
I
-··I-
'1,
-
I
....
-·---
--
I1i
500 ms
0
3
'UT
0
ms
II
IIL
-
·-.
·- ·-- · lb
I11
14X
FIGURE 2.23
R: RDRPTRTION STRGE INPUT
1
B:
3 -
0-
RDRPTRTION
_Jj
.
- .
i.
--
..
.
i
- - -..
- . ..
f
I
_
.
I.
.
.
.
r
.
I
bm,.m,mq,~.q
I
. .
,
.. I
OUTPUT
I L
-
-
-
1000 Is
0
_ _
STRGE
B: RRPTRTION STRGE OUTPUT
_I
. ._
.
------
141
FIGURE 2.24
+
Rs
X
Cn
Iout
cl
Co
multiple Time Constant Rdaptation
.........
odal
- I
-··
FIGURE 2.25
142
Cl
Iout
Co
Dil ouble Time Constant Rdaptation iodel
~-------I-- - - -
-
I -
I
FIGURE 2.26
143
R: RADARPTRTION STRGE INPUT
I
0
0
S00 ms
3
Iou+
-5
0
0
ms
I out)
1
E
co
0
0
ms
Camw
1
SHORT TERM
RESPONSE
0
ms
FIGURE
2.27
144
R: TRRNSDUCTION 5TRGE INPUT
24dB
iAs
-24dB
S s
30 ms
lo ms
B: RDRPTRTION STRGE INPUT
a
800 ms
RDIRPTRTION
800 as
0
___
_I
UT
I
_____·_____________I
I
__
145
FIGURE 2.28
A: TRRNqSrlJT TnN qSTAGF TNPlJT
24dB
M
a
-24dB
R: AnFIPTATTInN STAF TNPIT
.l
0
600
r
AnFPTT
TI nN STAF.F nl TP! IT.
b
CII
s
600 ms
I
_ _C
FIGURE 2.29
146
A: TRANSIDU[T ION
80dB
Ms
-10dB
R
n:PTTTTnN
B
0
See as
0
3
C: RDRPTATION
STRGE OUTPUT
B
50B ms
a
-
-------
II
_-----------·---
-Tl
__
_I
147
FIGURE 2.30
BARK
HERTZ
BER TAKC
I ..... .
- --
. ,. . . .' I I I
. . I
--
.. . .
. .,
-
-L*
--r
i
I
I
i·-·
FT p
'' I
--
-
I
112
23
304
466
517
5
660
S
7
I
|-,Ir-
TX
I
I
761
-
8
~IFr- TI
9
~
Ir
1
1-7
~T
iI I . I I A I
l
1
1
I I I I I I I I I I I I I I I I -i
913
1065
1
1218
11
1421
12
1674
13
1979
14
2384
15
2798
3298
I
17
,
' ,
I
1
,
J
l, I I
I. , {I
'I 6 i .
T,
.
I. I. I
II,
,,
O 1
1
i Ii I IIt I.* .It . I
. I1I I I I
I ,K I- I
I
I,
~""'~""'
I
. , , I '
I
18
3986
4617
I
19
2
. I I I , I 1 1 i 1 1 .
IIIII
I
Is
-
-
------------------------
I
"
6392
I
I
Ms
11,
5428
a
I .rr I II I
H
-
I
·---------- ---rr------------
·
FIGURE 2.31
A:
148
Transduction Stage Output
1
E
a)
C)
C
0
1
Frequency in Bark
B:
20
Adaptation Stage Output
3
O
Q.
()
CC
0
Frequency in Bark
1
I
__
_
II
IFIC
___I
20
II
j
FIGURE 2.32
BARK
149
HERTZ
BERTRKN
4
1
2
fi
i
i -1
71
17
1
rrrr-r
1 T`r
3
(-
rr
II-TI
r
17
I-r
_
II·I
7
r-77-
102
1
irl
1
I
f
rllT"I
4
Irr
LL
---
-
4L6
,
I-----e
-
r
5
203
304
_
_
__
i
r-r7
507
6
668
7
761
8
913
9
1065
10
11
Ii 1
ri7-1
T I
rl
T I -f7-1
1 rl
T 1.1
rl
rfll
t rl
1 Tt7-17T7
77
717
17
Tr7
1
7
i
1 j
1218
71
1421
I
i
177'C-rl-I
yl
I -r
1 7-TI--TTT-TTJ-I
I 7--Cl+i-7Fr7--TT1
11r
ir
I
12
--
I
--
-
--r----
-
C
1674
-
13
1979
14
2384
15
"""""'"""" '
'
i'
'""'
2790
16
17
18
3298
I
17
I r-l
i 7
r
1 1
19
20
g
20
17
I
Tr-7
IT-r-
r 77
1
T-1
r
rT
77
1 r
1 f 77
1 1 1-77
-TT
1 i
7-Tr
7-17-r7Tr77r7
) -r77-1
T TTi-l
7
rl
1 17
i
1 71r
T71
I
I
ri
1
rl'
I
rl
3906
I
4617
5428
r
6392
r
0 1MS
50
I ,
Ir
Ms
150
CHAPTER 3
PAM RESPONSE PATTERNS AND PHONETIC DISTINCTIONS
3.1
INTRODUCTION: USING THE PAM TO STUDY PERCEPTUAL RESPONSES
TO SPEECH
This chapter examines
the
in detail the response patterns of
Peripheral Auditory Model to two
sets of speech
signals.
A simple property of the PAM response to each set is shown to
be
correlated
with
listeners'
perception
of a phonetic
trast exhibited between elements within the set.
contrasts
in question
are,
in
the
first
distinction between a word-initial /b/
distinction),
and
tion between an
distinction).
perceptible
in
Although
acoustically
in
each
two
acoustic
variations
phonetic
contrast
and
The phonetic
of stimuli,
/w/
or
and /p/
only
a
set,
the
more
ways.
such
has
/b/
as
been
single
The
these
the distinc-
(a VOICED/UNVOICED
phonetic
stimuli
may
interpreted
as
the
(a STOP/GLIDE
the second set of stimuli,
intervocalic
within
set
con-
in
contrast
the
fact
sets
that
produce
evidence
is
vary
multiple
a
single
that
the
relationship between acoustic properties and phonetic features
is
context
dependent.
Many
such
acoustic
relationships"--multiple acoustic properties,
which
can
be
varied
to
contrast--have been found.
vide
convincing
Be--1 11
evidence
'I
_~n
I·L__
produce
an
that no
_I___
II__
combinations of
identical
Their variety and
"trading
phonetic
frequency pro-
simple one-to-one relation-
II - --
151
PAM Response Patterns and Phonetic Distinctions
exists
ship
waveforms
as
and
etc.,
quency,
stop
formants,
the
gap durations,
distinctive
fre-
fundamental
which
features
speech
of
properties
acoustic
such
between
efficiently
characterize speech at the phonological and perceptual level.
If the correspondence
theory described
in
the
Introduc-
tion is correct, we might expect that in the peripheral auditory system some of these complexes of acoustic properties are
merged
into
a single
phonetic counterpart.
relationship to its
a simpler
auditory property that bears
This would consti-
tute a "decoding" of the speech signal by the peripheral auditory system.
Another
creation
the
form of decoding would be
of a measurable property in the PAS response that represents
For instance,
an abstract property of the acoustic response.
in both of the sets of stimuli examined in this chapter, temporal properties of the acoustic signal, such as closure duration and speaking rate,
phonetic
account
of
are important elements in an acoustic
listeners'
PAS
The
perceptions.
can
be
said to participate in the decoding of speech if its response
patterns contain a measurable auditory property, such as average
firing rate,
is
which reflects these abstract temporal proespecially so
perties.
This
auditory
transformation
appropriate, or apt:
linear
relationship
phonetic feature
of
if
the
for instance,
between
into a linear
ca. -C I
sra*r
an
it
can be
property
shown
is
that
the
particularly
if the PAS converts a nonacoustic
relationship
. ------i
property
between an
I--
I--
and
a
audi-
PAM Response Patterns and Phonetic Distinctions
152
tory property and the same phonetic feature.
In using the PAM to model peripheral auditory response to
speech,
and
using
various
response properties,
parameters.
These
values must be
parameters
teristics;
quiesent
term,
long
and
postprocessors
to
assigned
determine
measure
to a number of
filter
bank
(spontaneous) response rate;
term
adaptation
times;
the
PAM
charac-
rapid,
level
of
short
-firing
below which channels are considered to be in a "masked" state;
etc.
It is reasonable to wonder about the extent to which the
results presented in this chapter depend on the exact parameter
values
chosen.
No
formal
study was made
of
the
sensi-
tivity of the PAM response to various parameter values.
How-
ever, a few informal observations can be made.
First,
sider,
a
number
for example,
of
parameters
adaptation
threshold level
Con-
following the offset of stimulation.
This duration is dependent on the
the
interrelated.
the duration of time during which a chan-
nel is in a masked state
rate,
are
selected quiescent response
time constants chosen,
and
used, among other parameters.
the
masking
For a particu-
lar stimulus, there are an infinite number of possible sets of
parameter
tion.
do
not
values which will result in the same masking dura-
Therefore the validity of the results
depend
"correct":
just
on
all
that their
the auditory system.
·I1_
jD
of
_I_
the
chosen
to be presented
parameter
combination
is
values
being
representative
of
PAM Response Patterns and Phonetic
Distinctions
..
, . ~1 . I
many of
Secondly,
reasonable limits
the
values
153
established
can be
within
from psychoacoustic and physiological
data.
The quiescent response rate and adaptation time constants are
examples of parameters for which such data is available.
restricting our analysis goals to demon-
Thirdly, we are
strating that there is a correlation between PAM response pro-
than
This is a
and phonetic features.
perties
trying
to
directly
calculate perceptual
If a correlation is
PAM output signals.
less
found,
rigorous goal
responses
from
its existance
is likely to be relatively insensitive to the exact parameter
values chosen.
Fourthly, although no formal study was made of the effect
of varying
values
parameter
were
changes
in
used
PAM
in
values,
a number of different parameter
informal
response
pilot
properties
runs.
were
No
extraordinary
observed
over
the
range of parameter values used.
Nevertheless, a systematic investigation of the effect of
changing
PAM
parameter
values
would
be
useful,
as
would
further attempts to determine optimal values from psychoacoustic and physiological data.
_
_a
__
PAM Response Patterns and Phonetic Distinctions
3.2
154
STOP/GLIDE EXPERIMENT
This experiment involves a set of 55 synthesized CV syllables
perceived
tokens
were
(1983).
by
listeners
created
The
and
original
described
first,
stimuli.
A
and
simple
as
either
analyzed
by
experiment
then
the
"ba"
or
Landahl
and
its
response
"wa".
and
results
of
the
auditory analysis will be
Maxwell
will
be
to
the
PAM
proposed,
statistical comparison will be made of the traditional
tic
phonetic
analysis
and
the
alternative
auditory
The
and a
acous-
phonetic
analysis suggested by the PAM response patterns.
3.2.1
The original experiment
3.2.1.1
The
study
Background
Landahl
by
Miller
that overall
of
initial
tions
and
Maxwell
Liberman
syllable
formant
changed
(/wa/).
and
study
was
(1979),
a
in
replication
which
it
was
of
a
found
duration affected the boundary duration
transition
from hearing
a
across
stop
which
(/ba/)
listeners' percepto hearing
a
glide
To quote Miller and Liberman:
The combined data ... clearly indicate that, as syllable
duration increases, there is a perfectly regular change
in the way our subjects identified the
synthetic speech
stimuli] as
ba] and [wa]: the longer the duration of
the syllable, the longer the duration of transition
needed in order to hear [wa].
_l_______llsO·______________
___
___
__ _
IC
III__
155
PAM Response Patterns and Phonetic Distinctions
authors believed would make the tokens more
that the
stimuli
synthetic
used
replication
Maxwell
and
Landahl
The
realistic than those used by Miller and Liberman.
Experimental Procedure
3.2.1.2
3.1
Figure
synthesizer.
formant
cascade/parallel
(1980)
Klatt
the
using
created
were
stimuli
The
the
shows
way in which the important synthesis parameters were varied to
create a continuum of tokens between a distinct initial labial
Each token has
stop and a distinct initial labiovelar glide.
an
of
the
Hz,
and the
second
first and
formant,
final
tially and rises to
1230 Hz.
F3
the amplitude of
is 230
is set at 620 Hz ini-
F2
is 770 Hz.
value
and
initial value of Fl
For all tokens the
voicing, vary.
a
During the initial region both the fre-
steady state region.
quency
by
followed
region
transition
and
prevoicing
initial
is held constant at 2860 Hz
for all tokens, and the fundamental frequency is held constant
at 114 Hz for all tokens.
The amplitude of voicing rises dur-
interval
from a level 10 dB below its steady
ing
the
initial
state value,
and
falls
by
again
15
dB
during
the
final
20
msecs before the end of the syllable.
The
stimuli
stop/glide
continuum
in two ways.
First,
was
created
by
varying
the duration of the initial for-
mant transitions varies in 5 msec intervals between
for
most
the
_
/b/-like
tokens
the
and
65 msecs
----------------··*I
15 msecs
for the most /w/-
1.1
156
PAM Response Patterns and Phonetic Distinctions
and
but begins
time,
transitions.) Second,
of
syllable
the
Syllables
299
msecs
were
created
were
a
the
of
each of
For
each
the
11
87,
than the
formant
steadystate
portion
158,
123,
syllable
rise
rise
same
independently of the onset
duration
total
the
has
earlier
the duration of
created.
with
voicing
msecs
15
ends
was varied
having
of
amplitude
(The
tokens.
like
times.
duration,
tokens
in
resulting
times,
and
228,
a
total of 55 stimuli.
of
The bandwidths
and
cies,
the
formants,
synthesis
other
higher
parameters
formant
adjusted
were
appropriate for a male speaker saying the vowel /a/.
gest
difference
between
Liberman and Miller
the
Landahl
the
and
parameter which controls the
low-pass cut-off
voice source during the initial prevoicing
tial
value
of
this
parameter was
ments between 562 Hz for the most
for the
most
/w/-like tokens.
by
adjusting
the
frequency of the
interval.
The iniincre-
and 500
Hz
tokens, this parameter
rose to 5000 Hz after the end of the prevoicing.
Landahl
and
/b/-like tokens
all
be
The big-
varied in logarithmic
For
to
stimuli
Maxwell
stimuli was created
frequen-
According to
and Maxwell,
[This adjustment] interacts with amplitude of voicing
such that for the /w/ endpoint the effect was one of
strong prevoicing and a gradual increase in amplitude
over the formant transitions, while for the /b/ endpoint
the prevoicing was much weaker, as is appropriate for a
stop consonant, and amplitude of voicing increased very
abruptly, also appropriate for the release of a stop.
__ C
-------------
sls-j
-------
__C~as_
?_I__
157
PAM Response Patterns and Phonetic Distinctions
Figure 3.2 shows the waveforms of the tokens with the 15
msec
rise
syllable durations are
Each of the five
60 msec rise time.
and the
time
for both the
shown
fast and
slow rise
(The tokens with the 65 msec rise times have an
time tokens.
unexplained ripple in their amplitude envelope.
Perceptually,
they appear to be good exemplars of /wa/, and are included in
all
statistical
the
However,
calculations.
of
because
their
visual anomaly, tokens with 60 msec rise times will be used in
figures to represent the slow-onset endpoint.)
Three test tapes were constructed,
repetitions
each containing eight
for a total of 440 stimuli per
of each stimulus,
The tapes differed only in their random ordering of the
tape.
tokens.
Ten
paid
subjects
to
listened
and
students,
hearing
disorders.
subjects were told
none
reported
Each
subject
to
judge
Eight
of the
All of the subjects were col-
subjects were male, two female.
lege
the tapes.
each
any history
of
speech
listened to each tape.
stimulus
and
respond
or
The
with
either /b/ or /w/: no other responses were allowed.
3.2.1.3
Results and Analysis
for all
subjects,
shown in Table 3.1.
The same
The average percent of stop
for
each of
the
55
is
tokens,
responses
information is shown in graphical form in Figure 3.3.
I _
_
__
It can
PAM Response Patterns and Phonetic Distinctions
be
seen
equal to
as
that
for
(rise
transitions
time
20 msecs) syllables of any duration
with
beginning
times
abrupt
of
55
a
msecs
or- more
a glide.
beginning with
Similarly,
stop.
are
less
are
than or
interpreted
with
syllables
consistantly
However,
158
rise
interpreted
as
intermediate region
in the
of 25 to 50 msec rise times, the percentage of stop judgements
more
likely
the longer the syllable, the
syllable duration:
depend on the
it
is
that
a particular
rise time will
be
per-
ceived as stop-like.
The
rise
STOP
between
time
associated
and GLIDE,
a
as
with
function
the
phonetic
boundary
syllable duration,
of
can be calculated by converting the percent stop judgements to
normalized
those
and
z scores
scores.
conversion
(This
categorical decision model
stop
percentage
of
difference
between
a
fitting
a
assumes
is
assumed
value of an
line
to
psychophysical
in which the
(e.g., Durlach, 1968)
responses
the
regression
linear
to
depend
on
the
internal perceptual vari-
able, which is affected by Gaussian noise,
and
a categorical
criterion or boundary value for the same variable.) The interpolated rise time duration which results in a z score of zero
is
then taken to be the boundary duration.
unduly weighting
the
regression
line by
In order to avoid
results
far
removed
from the boundary duration, z scores of less than -2.0 or more
than 2.0
(percentages
above
97
or below
3 percent)
are
not
included in the linear regression calculation.
C------
-----·-.I_---i·---U-
_____·_11----_111_---
PAM Response Patterns and Phonetic Distinctions
The
resulting
rise
time boundaries
159
are listed
in
Table
3.1,
and plotted as a function of syllable duration in Figure
3.4.
The boundary rise time appears to be a nonlinear func-
tion of syllable duration, with a lower limit of 27.5 msecs
for 87 msecs syllables and
299 msecs syllables.
Liberman
show a
an upper limit of 37.6 msecs
for
The corresponding data from Miller and
greater
syllable
duration
effect:
80
msec
syllables had a boundary duration of 31.9 msecs, and 296 msec
syllables
had a boundary duration
experiments
duration
the
for
boundary
short
duration
syllables,
but
of
46.6
msecs.
is
sensitive
lengthening
In
both
to
syllable
the
syllable
beyond 200 msecs appears to have little effect.
Landahl and Maxwell's acoustic phonetic analysis of these
tokens
suggests
that
the
phonetic
feature
associated with an abrupt change in amplitude.
the
feature
tinuant],
-continuantj,
rise times which
and
other
the
is
Since /b/ has
feature
+con-
are fast enough to be considered
abrupt signal the stop consonant.
properties which cue
/w/ has
-continuant]
To the extent that acoustic
phonetic
differences
between
/b/
and /w/ are ambiguous, perceptual judgements appear to depend
on the auditory system's interpretation of the abruptness of
the onset.
Presumably the duration of the steady state por-
tion of the syllable serves to calibrate the onset time:
the
longer the syllabic nucleus, the slower the speaking rate, and
the more abrupt a particular onset time will appear.
_
_-_111----__----·1---·--ri(-
If the
ll
160
PAM Response Patterns and Phonetic Distinctions
50 msecs,
and
25
acoustic
This
Why
the
is
endpoint
"slow"
and
transitions
rather than some other values?
around 25 and 50 msecs
*
and
straightforward
leave some questions unanswered:
"quick"
the
are
is
analysis
phonetic
consistent, but it does
Why
perceived rate can influence the
the
judgement.
perceptual
*
is
normalization
such
no
But if the onset time is in the intermediate range
necessary.
between
slow,
very
or
quick
very
is
onset
boundary
between
relationship
duration
and
beyond
200
syllable duration a nonlinear one?
*
Why
duration
syllable
the
increasing
does
msecs have little effect?
The PAM Response
3.2.2
to
stop/glide
the
measurable
syllable
of
and
these properties and listeners'
rise
time
because
they
equivalent
higher
_
and
are
syllable
not
representation,
of
but
the
of
which
pattern
rise
the
are
time
correlation
considered
acoustic
rather
are
PAM
find
to
first
and
between
Again,
perceptual judgements.
duration
values
response
properties
study
then
the
response of
attempt
will
PAM
the
acoustic
abstract
duration,
We
stimuli.
properties
the
reflect
examine the
section we will
this
In
abstract
signal,
dependent
or
on
any
a
level waveform segmentation process.
_ms
1i
Isrl
PAM Response Patterns and Phonetic Distinctions
Much
trated
of our
attention
in
this
section
161
will
be
concen-
on the onset and offset characteristics of the PAM in
response to these
stimuli.
Let us define
the onset response
to be the response of the PAM in the first 20 msecs following
any significant increase above the
a
stimulus
begins.
This
length
spontaneous
of
time
is
firing rate as
long
enough
to
encompass the first two glottal pulses of our stimuli that are
strong
enough
model.
Let us define the offset response to be
to
be
well
within
the
dynamic
range
the
of
response
of the PAM to the last full glottal pulse before the 2
of a
stimulus.
practice
regions
it
Although
these
are
informal
the
offset
definitions,
in
is always easy to identify the location of these
in the
response pattern of the PAM.
The
strength
of
the onset or offset response (within a particular channel) is
defined
to
be
the
maximum
response
signal
value
obtained
within the onset or offset interval.
3.2.2.1
PAM Processing Procedure and Parameters
The stop/glide stimuli were digitized
per
second
from
audio
The
audio
signal
was
digitizing.
The
tape
(sample
low-pass
digital
period
filtered
signals
at
were
at
12,987
samples
=
77.000
usecs).
6,400
Hz
before
preemphasized
as
described in Chapter Two, and then processed by the spectral
analysis
stage
was
stage.
the
~~~"~~~~D~~~"----C-~~~~__
The
DC
nonlinearity
transfer
function
____
used
of
in
a
the
transduction
raised
hyperbolic
PAM Response Patterns and Phonetic Distinctions
162
tangent function, with a bias parameter of -1.1.
A three time
constant adaptation stage was used, with time constants of 15,
40,
and
320
response.
3.2.2.2
msecs,
contributing
equally
to
the
The adaptation ratio was 3.5.
PAM Response Patterns
Figure
3.5
shows
the
response
of
the
stop/glide stimulus with the fastest onset time
the
transient
longest
syllable
duration
(299
PAM
to
the
(15 msecs) and
msecs).
The
periodic
pulses in the PAM output are the model's response to the glottal pulses
in the
amplitude,
the PAM's
(channels
state
three
input.
onset
through
response
Because
in the
of the
response
eleven)
is
in
rapid
the
increase
Fl and F2 region
stronger than
same region.
This
its
steady
is due primarily to
adaptation, since the acoustic amplitude of the onset is
than
or
equal
three
through
tions
add
to
to
the
five,
this
amplitude of
and
eight
and
the vowel.
nine,
onset-peak effect:
the
For
formant
in the figure.
percent
stop
In
channels
transi-
frequen-
These formant transitions can be seen
summary,
responses),
less
within the onset region
the formants move through and then beyond the center
cies of these channels.
in
this
and
has
token
an
sounds
like
onset response
/ba/
that
(98
is
stronger than its offset response.
Figure
stop/glide
3.6
shows
the
response
of
the
PAM
to
the
stimulus with the same onset time as the previous
I·-·l--·---siL··l··---
_-------- -. --
PAM Response Patterns and Phonetic Distinctions
(15
figure
The
msecs).
but
msecs)
state
steady
three glottal pulses long.
of
portion
this
(87
duration
syllable
shortest
the
163
syllable is
only
still
fast
The onset, however, is
enough that the envelopes of the PAM channels in the Fl and F2
of
regions,
where
most
followed
by
steady
a
is,
energy
show
token
This
decline.
and has
stop responses),
(100 percent
/ba/
the
initial
peak
also sounds
like
an
an onset
response
looks
somewhat
that is stronger than its offset response.
shows a response pattern
Figure 3.7
different.
msec onset
the
The stimulus in this figure is the token with a 60
and a 299 msec
amplitude
onset
than
voicing
of
duration.
in
The gradual
stimulus
the
response that is smaller in almost
the
that
PAM
response
portion of the token.
at
the
increase
results
all of the
beginning of the
PAM
a
in
in
channels
state
steady
Even 200 msecs after the beginning
of
the steady state region, when most of the potential adaptation
has occurred, the firing rate is still higher in most channels
than during the onset.
This token sounds like /wa/ (1 percent
stop responses), and has an onset response that is weaker than
its offset response.
Figure 3.8
shows the PAM
response to
msec onset and an 87 msec duration.
onset
and
a
short
syllable
a token with
a 60
The combination of a slow
duration
results
in
an
offset
response that is much stronger than the onset response in all
channels
containing
b
significant
energy.
.---------
Once
again,
this
PAM Response Patterns and Phonetic Distinctions
perceived to begin with a clear glide, eliciting no
is
token
164
stop responses at all from the experimental subjects.
to have
little
figures, syllable duration appeared
four
In the previous
effect on
either perceptual
responses
relative sizes of onset and offset responses.
the
onsets
in
these
3.9
Figures
slow.
figures
and
3.10
were
either
the
This is because
very
the
represent
or
fast
PAM
or
very
responses
to
stimuli with intermediate transition and amplitude rise times
For these tokens, syllable duration has much more
(30 msecs).
Figure
of an effect.
3.9
shows
duration, which
with a 299 msec
the PAM response to a token
is
perceived to
be
strongly
stop-like (94 percent stop responses), while Figure 3.11 shows
a token
an
with
87 msec
generally
glide-like
region of
the lower
pulse
pulses
onset.
This
offset
and F2
response.
vowel is
percent
formants
of
than
later
tends
offset response
Fl
(17
stop
responses).
patterns,
for these
the
for
to place the
shortest
stimuli
with
In
be
the
the glottal
region is
the
most
rapid
peak response very close to
duration
syllable,
(the pulse near 120 msecs)
so
that
the
for channels in the
approximately the same size as the onset
In contrast, when the steady state portion of the
allowed to continue
tial adaptation occurs
for 200 msecs
or more,
substan-
and the relative size of the last full
pulse (near 330 msecs) is relatively weak.
_
to
to the largest response peak occurs one or two
leading
glottal
the
is perceived
duration, which
165
PAM Response Patterns and Phonetic Distinctions
A Possible Auditory Analysis
3.2.2.3
These informal observations regarding the relative sizes
and offset response peaks
of onset
suggest a possible measure
of "onset strength"
as a property of the peripheral
response to speech.
An abrupt onset of acoustic energy should
result in an auditory response that is
pared
relatively strong com-
response to the stimulus
with the
auditory
following the onset.
A simple postprocessor was designed to provide a quantitative
The
measure of onset strength for the current set of stimuli.
first
step
in
the
postprocessing was
single-channel
composite
waveform
to
produce
a
smoothed
PAM
from the multi-channel
This was done by using a 20 msec half hamming win-
response.
dow to smooth each channel 6f the PAM response.
of the hamming window was used, so that the
The left half
non-zero portion
of the smoothing window included the current response and the
response during the previous 2
msecs, with most of the weight
At each point in
given to responses within the last 10 msecs.
time,
the
channels,
smoothed
so
that
values were
the
whole-nerve firing rate,
output
averaged over all
value
represents
twenty PAM
the
expected
averaged across all frequencies,
and
smoothed with a 20 msec window.
The second step in the postprocessing was to select parglottal
ticular
The
response.
second
pulses
glottal
significant
-
-
to
pulses
glottal
I
represent
chosen
pulse
in
the
onset
were
and
as
the
response,
and
defined
the onset
__
offset
:
i_____L_____lil;l-_J;L-
PAM Response Patterns and Phonetic Distinctions
166
The loca-
the last full glottal pulse in the offset response.
tions of these pulses, which always occur at the same time in
value
largest
the
extract
to
written
was
program
puter
com-
A
are listed in Table 3.2.
same length,
stimuli of the
smoothed composite PAM response signal within
attained by the
In all cases a
the temporal limits established by Table 3.2.
glottal pulse was well within
limits,
the
and corresponded to
a subjective choice of the correct onset or offset pulse.
for
Thus
we
stimulus
each
a
derived
have
quantitative
measure of the strength of the onset and offset of the periNotice that we do not have to meas-
pheral auditory response.
ure time--either rise time or syllable duration--in any direct
that
itself
Instead the PAM
way.
of
size
the
is
behavior which
its
in
response
turn
is
dependent
time,
a measure of
provides
on
adaptation
in a
particular
dependent
on
time
in
way.
The
Figure 3.11 shows a set of five stop/glide signals.
acoustic waveform is
shown on the left, and the smoothed com-
posite PAM response on the right.
The top signal is perceived
to be strongly /ba/-like for all syllable durations, while the
bottom
signal
durations.
is perceived
The
less
to be
three middle
/ba/-like
or
Note
that the glottal pulses
composite signal.
for
signals
are
perceived as
on
the
syllable
depending
more
strongly /wa/-like
are
__
being
duration.
still clearly visible in the
The vertical lines indicate the position of
____ __1_1__(___1________II____·
_
all
_1
PAM Response Patterns and Phonetic Distinctions
the
pulses
response
onset
and
offset
The
durations.
syllable
for
responses,
each
of
five
the
possible
the
indicate
lines
horizontal
the
strength of
represent the
to
chosen
167
strength of the onset glottal pulse for each signal.
The percentages above
each response pattern are the per-
cent of stop responses to the shortest syllables (left value)
and
time.
This
of
response
stop or glide.
(top signal),
syllables
as beginning with a stop.
it
is
significantly
with
relative sizes
the
duration
a
glide.
the onset is
When
of any duration are perceived
than
is slow enough
any
of
the
of any duration
For
intermediate
are
offset
the
syllable,
and
the
that
pulses
perceived
onset
times,
of the onset and offset pulses will
of
fast
than any of the offset
When the onset
smaller
syllables
(bottom signal),
beginning
rise
offset pulses, and the per-
enough that it is significantly larger
pulses
indicated
figure clearly shows the relationship between the
relative heights of the onset and
ceptual
the
(right value) with
syllables
longest
perceptual
as
the
vary with
response
varies in the same way.
Statistical Analysis
3.2.2.4
In this section we will attempt to answer three questions.
*
How well do the acoustic properties of rise
syllable duration explain the perceptual data?
I
-^
I
_1_
11__
_I
time
and
____l_____·C____1__L____I____
PAM Response Patterns and Phonetic Distinctions
168
k*
How well do the auditory properties of onset
rate explain the perceptual data?
*
How linear is the relationship between offset firing rate
and the onset rate representing the phonetic boundary
between stop and glide perception?
and offset
The first two questions can be answered using a multiple
regression
lyses,
analysis.
In
the dependent
both the acoustic and
variable
is
auditory ana-
the percent of
stop
judge-
ments by the listeners, converted to a normalized z score.
In
the acoustic phonetic analysis, the two independent variables
are
rise
time
and
syllable
duration.
In
the
auditory
analysis, the two independent variables are onset firing rate
and offset firing rate.
The results of the multiple regression analysis are shown
in Table 3.3.
It can be
seen that both the acoustic analysis
and the auditory analysis provide a good explanation for
variability
in
the
data:
the
acoustic
analysis
the
explains
88
percent of the variability, and the auditory analysis explains
94 percent of variability.
The auditory measures of onset and
offset firing rate are more strongly linearly correlated with
the
perceptual
responses.
significant at the p=.0l
The
difference
is
statistically
level, as measured by a Student's
t
ratio of the size of the residuals in the z scores after the
respective linear regressions.
__
I
_
_
1I
169
PAM Response Patterns and Phonetic Distinctions
In both
for most of the variability in the
accounts
stimuli
the
of
From the discussion above,
data.
it should be clear that the
the
Since much of
equivocal.
results when the onset measures are
affect
only
duration
syllable
with
associated
measures
characteristics
onset
the
of
cases, the measure
the current data contains onset characteristics clearly india
of
cative
a
or
stop
glide,
have
measures
onset
the
the
larger partial correlation coefficient.
In order to analyze the role of syllable duration and its
and glide-like onset times.
between stop-like onset times
on
dence
tory
rates
firing
onset
the
analysis,
were
stimuli
stimulus
percent
whose
offset
stimulus
and
Each stimulus
offset
whose
rate of the
first
clusters
_I_
least
~
~
~
~
than
z score between +2.0
rate was within
three members,
~
with
two
around which a cluster had
with a
cluster member,
by more
differed
was
and
~
added
to
both
had
~
and
five percent of the
the cluster.
Of the clusters that were formed using this method,
them had at
firing
For the audi-
into
grouped
rate
firing
from a previous
been formed.
depen-
A cluster was formed around each
similar offset firing rates.
-2.0,
onset
glide-like
and
Figure 3.12 shows both of these graphs.
rates.
the
It
firing rate of the phonetic boundary between
offset
stop-like
showing
graph
a similar
construct
to
possible
which graphs
phonetic boundary
the
of
the dependence on syllable duration
is
3.4,
Figure
return to
can
we
auditory measure,
~
eight of
positive
s_ _
~~~~~~~-_---
and
PAM Response Patterns and Phonetic Distinctions
170
An interpolated boundary onset firing rate
negative z scores.
was calculated for each of these eight clusters using a linear
least-squares fit to the z scores.
It can be seen that the relationship between the auditory
Why
measures of onset and offset firing rate is quite linear.
is the auditory relationship a straight line while the acoustic
relationship
the
kind
of
is
a
"syllable
saturating
curve?
duration"
that
Presumably because
is
used
perceptually
grows in much the same way that firing rate falls off: quickly
at
first,
between
and
slower
then
perceptual
duration
and
So
slower.
(in this
case)
auditory response (offset firing rate)
system
appears
with a measurable
to
provide
and a property of
The peripheral audi-
central
the
relationship
is more direct than the
acoustic correlate of syllable duration.
tory
the
system
auditory
response property that corresponds linearly
to the way in which syllable duration is perceived.
3.3
VOICED/UNVOICED EXPERIMENT
this
In
"rabid" were
able
and
tokens
the
were
experiment,
natural
utterances
of
the
word
edited to adjust the duration of the first syllintervocalic
perceived
by
stop
gap.
listeners
The
be
to
Ii
-·l-·L-lll--
resulting
either
-
set
the
of
word
171
PAM Response Patterns and Phonetic Distinctions
the
"rabid"
or
tions.
The
described
first,
stimuli.
A
in
and
created
edited
the
analyzed
by
duraand
Price
The original experiment and its results will be
Simon (1984).
evident
jwere
tokens
on
depending
"rapid",
word
and
then
simple measure
the
auditory
the
of
of
response
the
degree of
response will
be
the
to
PAM
the
forward masking
proposed, and
the
correlation between that measure and the perception of voicing
will be explored.
The original experiment
3.3.1
3.3.1.1
Background
A number of studies have demonstrated that many acoustic
properties can cue the distinction between voiced and unvoiced
(Edwards,
stop consonants.
Lisker, 1978;
1981;
Klatt,
of these properties is
closure duration.
Although
weak
a
only
plays
Lisker,
Stevens and Klatt, 1974; Zue, 1976).
vocalic stops, one
tion
1975;
Edwards
in
role
found
voicing
the
1957;
For inter-
length of the
that closure duraidentification
in
natural speech, and does not exhibit a bimodal distribution of
values, he found that voice onset time and consonant duration
play very important roles.
closure
duration
increased,
the
overwhelming
perhaps
·__
has
variation
effect
because
in
on
these
In edited speech tokens, where the
been
in
substantially
closure
the
tokens
duration
decreased
can
have
or
an
voiced-unvoiced
distinction,
it elicits the
same response
PAM Response Patterns and Phonetic Distinctions
that
Another
speech.
or
VOT
changing
is
English
can
that
affect
voicing
preceding
the
of
duration
the
natural
in
does
duration
property
acoustic
in
identification
consonant
172
Edwards found that in normal speech this is the weak-
vowel.
investigated,
13 acoustic cues to voicing that he
est of the
but as we shall see it can significantly affect voicing perception in properly controlled circumstances.
Price
The
closure
which
of
category
and
levels
effects
of
and
age
to
Thus
were
edited
in
used
order
These
acuity.
"rabid"
to
syllables
were
of
way
in
the
the
equal
avoid
two
effect
linguistic
consonants.
stop
approximately
with
effects
four
signal
durations
on
level
presentation
intervocalic
in
the
examined
experiment
vowel
subjects
hearing
HL.
and
voicing
younger
listened
Simon
stimulus
and
age
listener
and
audiometric
confounding
of
groups
played
at
the
subjects
60 and 80 dB
duration,
closure
investigated:
Older
vowel duration, subject age, and presentation level.
Experimental Procedure
3.3.1.2
Four tokens of the word "rabid",
native
spoken by an adult male
speaker of English, were used as a source of stimuli.
Spectrograms
of
these tokens
tokens were low-pass
are
shown
in Figure
3.13.
The
The resulting sig-
filtered at 3.5 KHz.
nals had amplitudes at and above 4 KHz that were at least 70
dB
below
_
the
spectral
_
peak
in
I
----
_
the
middle
___
of
the
_·___--.---.------rl
stressed
----
-·-·I-D--
PAM Response Patterns and Phonetic Distinctions
initial vowel.
173
The authors state that
The words used ... had stop release bursts with predom-
inant energy at relatively low frequencies (i.e., below
4 KHz). Thus it is likely that little, if any, information was destroyed in the filtering process.
After
filtering, a waveform editor was used to
replace
the original 90 to 95 msec closure durations with one of four
intervals of silence: 35, 65, 95, or 125 msecs.
In a similar
manner, the original preclosure voicing, which had a duration
of 200 to 230 msecs, was adjusted by removing or duplicating
glottal
pulses
in
the
Figure
portion
"vowel" durations:
create one of four
msecs.
steady-state
3.14
the
shows
160,
waveforms
of the
180,
of
vowel to
200,
four
or 220
stimuli
representing the endpoints of both the closure durations and
vowel durations.
The
resulting
set of
64
stimili
(four original tokens
times four closure durations times four vowel durations)
recorded on tape.
were
Eleven randomized sequences of the complete
The first sequence was used as a practice
set were recorded.
In the other sequences, each stimulus was separated from
set.
the following stimulus by three seconds of silence.
Each sub-
ject listened to the tape twice, with a delay of several days
between the
sessions.
played at 60 dB HL.
played at 80 dB HL.
During the
first session the tape was
During the second session the tape was
During
the
second session only five of
the sequences were played followed the practice sequence.
-X
I.
"
81
C·-
---i----------·-------·-·-------
PAM Response Patterns and Phonetic Distinctions
experimental
Ten of the
and
26),
attempt
and
to
find
older
(between 16
were young
ten of the subjects were older
made
was
subjects
174
subjects
(over 55).
with
An
pure-tone
audiograms as close as possible to the younger subjects.
This
was done because the authors wanted to study age effects other
than
hearing
level.
audiograms
of
both
thresholds
of the
Figure
groups
of
older group
3.15
shows
subjects.
are
less
The
the
pure-tone
mean
pure-tone
than 1
dB below the
mean for the younger group at all frequencies measured below 4
KHz.
The
the
stimuli were presented
subjects
were
asked
to
other medial consonant.
to
respond
(At the
subjects
with
monaurally,
"b",
op",
or
and
some
shortest closure durations,
some subjects reported hearing "t" or "d", presumably reflecting the perception of an intervocalic flap.)
Analysis
3.3.1.3
Figure
3.16
from
subjects,
tion,
vowel
shows
as a
the
average
percent of
"p"
responses
function of age group, level of presenta-
duration,
strongest effect seen
and
closure
in this
duration on the percent of
"p"
duration.
By
far
the
figure is the effect of closure
responses.
Closure durations
of 35 msecs elicted predominantly "b" responses, with some "t"
or
"d" responses.
tions
-I'
-
of
125 msecs
-
No
"p"
responses
occurred.
elicted predominantly
---··--·---··PIIgpsl··1111119·(·
·
Il
Closure
"p" responses.
duraFor
175
PAM Response Patterns and Phonetic Distinctions
durations
tokens with closure
vowel
duration,
level
of
intermediate
presentation,
to
and
extrema,
these
subject
age
all
have an effect:
*
The longer the preceding vowel,
the
longer the
boundary
duration;
*
The older the subject, the longer the boundary duration;
r*
The
quieter
the
the
presentation,
longer
boundary
the
duration.
These effects are summarized in Figure 3.17, where the interpolated closure boundary durations are
shown as a function of
the other independent variables.
interpolation was
using
z
normalized
scores
The
a
in
manner
to
similar
done
that
described in the previous experiment.
of closure duration on
The effect
typical
preted
of
in
acoustic properties
a
categorical
stop consonant
as
voiced
voicing
is
perception
which are perceptually interThe
manner.
of
identification
the
or unvoiced changes abruptly across
some boundary value of closure duration.
The auditory system
acts as if some perceptual anchor exists at the boundary location,
no
although
such
anchor
is
visible
in
the
acoustic
waveform.
The
trading
- - --
effect
of
relation
vowel
duration
between multiple
a ---- P6
"-
is
another
acoustic
·
cues
example
to
a
··------------
of
a
single
PAM Response Patterns and Phonetic Distinctions
phonetic distinction.
of
vowel
duration
dependent"
duration
in
the
cate a voiced
One possible explanation for the effect
follows
"context
along
what constitutes
much
explanation
stop/glide
stop,
176
and
for
the
the
effect
experiment:
long
"short" and
closures
same
line
of
as
the
syllable
short closures indian
unvoiced stop,
but
"long" depends on speaking rate.
An intermediate closure duration would be considered longer if
it appeared in the context of a rapidly articulated sentence,
while
the
same
articulated
closure,
sentence,
occurring
would
in the
constitute
midst
a
shorter
Articulation rate, in turn, can be estimated
tion.
Therefore
a
longer
vowel
articulation rate, and results
of
duration
a
slowly
closure.
from vowel durasignals
a
slower
in more "short closure"
judge-
ments and hence more voiced responses.
Price and Simon present a tentative alternative explanation,
based
on assumptions
regarding the peripheral
response to their stimuli.
They assume that
sensitive
of the release burst
to the
amplitude
auditory
"the listener is
in deciding
whether a /p/ or /b/ was heard." They then argue that changing
vowel
duration,
affect the
This
level
level
of
explanation is
neither
Edwards
nor
of presentation,
auditory response
problematic
Zue
found
for
any
and
to
subject
the
release burst.
several reasons.
systematic
l(n----
shows
relatively small
--
differences
in
First,
differences
the burst amplitude of voiced and unvoiced stops.
model
age will
the
in
Second, our
size of
__-_l__l_-0^·__---·
the
PAM Response Patterns and Phonetic Distinctions
177
release burst as a function of the parameters mentioned above.
Hence we shall not pursue this proposal in detail.
Continuing their analysis,
Price and Simon cite a number
of studies which suggest that there are age-related effects in
the processing of temporal
Maki,
1976),
and
that
information in speech
temporal resolution
age
(Ludlow, Cudahy and Bassich,
the
age
related
differences
in
1982).
deteriorates with
They
boundary
(Beasley and
speculate
duration
that
that
they
found "are correlated with the neural response of the auditory
system
to the
correlation
release
may
be
burst of the
due
to
a
consonant,
lengthening
and that this
with
age
of
the
recovery time of neural fibers."
3.3.2
The PAM Response
In this section we shall
the
voiced/unvoiced
some
measurable
examine
stimuli.
property
of
the
We will
the
PAM
response of
first
the PAM
attempt
response
to
pattern
to
find
which
corresponds to the abstract acoustic phonetic property of closure boundary duration.
which
vowel
duration
Then we shall
and
neural
investigate the way in
recovery
time--as
hypothesized correlate of age--affect this property.
have
little
to
say about
a
We shall
the effect of presentation level on
the closure boundary, because the dynamic range of the PAM is
too limited to allow us to
test stimuli that differ in inten-
sity by 20 or even 10 dB.
_
_ I_ _
1_1
I__
______l_________s_____1__11_11_____1
-
PAM Response Patterns and Phonetic Distinctions
3.3.2.1
PAM Processing Procedure and Parameters
The stimuli were digitized at
from
audio
tape.
The
audio
phasized
as
described
spectral
analysis
transduction
hyperbolic
To model
in
the
The
Chapter Two,
the DC
transfer
response
of
filtered
at
then processed by
nonlinearity
used
function
with a bias
the younger
second
signals were preem-
and
The
function,
samples per
low-pass
digital
stage.
stage was
tangent
12,987
signal was
6,400 Hz before digitizing.
the
178
of a
parameter
subjects,
in
the
raised
of
-1.1.
a three time
constant adaptation stage was used, with time constants of 15,
40,
and
320
response.
msecs,
contributing
equally
The adaptation ratio was 3.5.
PAM operating parameters used
to
the
transient
(These are the same
for the stop/glide experiment.)
To model the response of the older group of subjects, only the
middle adaptation time constant was changed.
from 40 to 5
msecs,
in
line with Price and
It was increased
Simon's
sugges-
tion.
3.3.2.2
PAM Response Patterns
Figure 3.18 shows the response of the peripheral auditory
model
this
to
a
figure
durations
II
stimulus
corresponds
and
the
to
shortest
Figures
3.19
stimuli
corresponding
_
created
through
I_
3.21
to
from Token
the
of
C11
the
show
the
shortest
four
the
four
1.
of
The
the
stimulus
four
closure
other
possible
three
in
vowel
durations.
endpoint
combinations
of
PAM Response Patterns and Phonetic Distinctions
longest
vowel
and
closure
179
durations.
General
shortest
and
features
of interest that are visible in these response pat-
terns include:
firing before the begin-
*
the steady level of spontaneous
ning of the first syllable;
*
the rising first and second formant during the transition
into the steady state portion of the first syllable;
*
the clear definition of
that are not saturated;
*
the saturation of the
and second formants;
*
the significant amount of adaptation of response in the
formant channels during the course of the initial vowel;
*
the almost total lack of spontaneous firing across all
channels at the beginning of the intervocalic stop closure;
*
the recovery of spontaneous firing in some channels during the course of the longer closures;
*
the strong alveolar offset transitions
second formant at the end of the word;
*
the recovery of spontaneous firing following the end of
the word, with heavily stimulated channels around the
formants recovering later than the less stimulated channels at high frequencies and in the valleys between formants.
the
glottal
pulse
in
channels which include
channels
the
in the
first
first
and
As mentioned above, Price and Simon suggested that changing
duration
the
of
the
change the size of
-
--
r
-r"l
stop
the
·----
closure
and
the
auditory response to the
I
P--------
-----------
-----------.--------.-
vowel
might
stop burst.
PA4 Response Patterns and Phonetic Distinctions
Figure 3.22 shows this
tern.
It
can
be
response, displayed as a spectral pat-
sen
longer vowel durations,
response.
However,
that
Because
these
shorter
closure
durations,
and
do result in a somewhat smaller burst
these
changes
corresponding to changes in
dB.
180
are
relatively
small,
stimulus amplitude of less than 5
differences
are
so small,
burst
response
size did not appear to be a robust auditory property to try to
correlate with voicing perception.
If we look at the auditory response within the
calic
intervo-
stop gap, and ask ourselves what other property of the
response pattern might be associated with a categorical boundary
for
msecs,
voicing
an
firing
perception
obvious
following
possibility
recovery
should be noted that the
through
they
3.21,
were
within
is
from
the
the
the
range
return
of
of
initial
50
to
10
spontaneous
syllable.
It
small vertical scale of Figures 3.18
and the poor resolution of the device on which
plotted,
make
it
appear
that
spontaneous
firing
begins considerably later than it actually does.
The
the
recovery of
masking
3.23
firing is most easily
postprocessor
described
through 3.26 show the output of
in
examined by using
Chapter
this
2.
Figures
postprocessor when
applied to the PAM response patterns to the endpoint stimuli.
In these figures, a high value in a channel at a given instant
means
that the expected firing rate in that channel is zero.
We will describe such a channel as
_
I
_CIIII_
"masked".
Ii/
As can be seen,
PAM Response Patterns and Phonetic Distinctions
181
masking begins to appear in heavily stimulated channels during
the
first
for one
syllable,
When
sure.
only with onset of the
begins
duration
closure
the
the
during
latter
However, sustained masking across
half of each glottal pulse.
almost all channels
two msecs
or
is
only
clo-
stop
35 msecs, masking
continues up to the release of the stop, with spontaneous firing
and
lowest
the
in
only
reappearing
the highest
some of
channels, where there is little acoustic energy in the
syllable.
first
In contrast, when the closure duration is 125 msecs
there is more opportunity for recovery, and spontaneous firing
spreads
the most heavily
but
all
across
stimulated channels.
in
3.24, which shows the masking pattern
By comparing Figure
response to a stimulus with a "short" vowel duration, and Figure
3.26,
stimulus
vowel
the
which
with
a
the masking
shows
vowel
"long"
duration affects
the
pattern
duration,
it
in
response
be
can
recovery period.
This
seen
is
to
a
that
due
to
long term adaptation constant, which results in more adapin
tation, and therefore a longer recovery time,
response to
220 msec vowels than to 160 msec vowels.
3.3.2.3
A Possible Auditory Analysis
Examination
through
suggests
3.26
masking
the
of
that
amount of masking present
at
the
end
of the
·II
-
composite
in
Figures
measure
3.23
of
the
in the peripheral auditory response
closure
-·W
some
patterns
in
an
intervocalic
stop may
be
PAM Response Patterns and Phonetic Distinctions
182
related to the perception of the closure as "short" or "long".
In this section we will develop a quantitative measure of closure
length
with
the
idea,
on this
based
perceptual
responses
explore
and
of listeners
its
correlation
in this
experi-
ment.
A simple quantitative measure of degree of masking at a
particular moment can be produced by counting the fraction of
channels for which the output of the masking detector is high
The resulting measure will have the value one
(equal to one).
when all channels
will
have
have
value
the
an
expected
when
zero
rate greater than zero.
all
firing
rate of zero,
and
channels have an expected
(Although we are not
presently con-
cerned with constructing a measure of masking which is psychologically valid,
system
could
it should be noted that the
taneous
firing
To do
of
rate
zero, detecting the
nervous
a masking detector similar to the one
implement
we have proposed.
central
requires knowing
so
an
auditory
fact that
the
neuron
current
that
is
rate
spon-
the
than
greater
is
zero,
and
generating a neural response that signals those facts.)
This measure was
in this
the
stimuli
was
smoothed using
resulting
signal
to the
applied
experiment.
a
5
represents
As a final step, the measure
left
msec
a
response of the PAM to
half-hamming
smoothed
The
window.
composite measure
of
the degree of masking in the peripheral auditory nerve.
_
_1
I
_ICL_
_1__
I
__
I_
_
PAM Response Patterns and Phonetic Distinctions
183
Figure 3.27 demonstrates the behavior of this signal and
its relationship to the acoustic signal and the composite PAM
firing
in
used
signal
rate
is a stimulus produced from
on top,
shown
acoustic waveform,
The
experiment.
previous
the
Token 1 and has a 160 msec vowel duration and 220 msec closure
duration.
smoothed
with
features
in
ing,
middle
The
a
this
clear
the
acoustic
20
vowel
stressed
recovery
of
to
the
glottal
spontaneous
in
firing
spontaneous firin
the
firing rate during
the
burst
visible
the
visible),
pulses
response
Interesting
window.
onset
saturation
the
PAM
composite
include the initial
response
(no
the
hamming
half
msec
signal
waveform,
is
signal
during
the
stop
steady
closure,
and
The bottom signal
the glottal pulses during the second vowel.
is the smoothed composite masking signal generated by postprocessing the PAM response to the
nal
has
firing
an
at
initial
value
(nonzero)
their
top signal.
of zero, because
spontaneous
rate
The masking sigall
channels
before
the
are
word
begins.
As the acoustic energy increases, heavily stimulated
channels
begin
exhibit masking
to
(complete
during the second half of glottal cycles.
bottom
the
trace
first
closure
as
and
the
has
vowels.
of masking
At
an
for
both
of the
stop
visible
the beginning
shows
firing)
This appears in the
spikes,
pulse-like
of
abrupt
increase,
and
in a step-like fashion over the course of the
The staircase effect is due to the fact that the PAM
only 20
______
second
amount
then decreases
closure.
glottal
lack
channels, and only
, -- i--- ---
a minimum amount of temporal
------
-------· ~~~~~~~~~~~~~~~~----~~~--·"·--------~~~~~
I
PAM Response Patterns and Phonetic Distinctions
With a sufficiently large number of
smoothing has been done.
the
channels
decline
presumably
would
level
masking
184
in
a
ramp-like fashion.
for
all
the
four
four
of
combinations
closure durations.
stimuli
endpoint
the
stimulus
three
these
shows
3.28
Figure
representations
from Token 1:
produced
and
shortest and longest vowel
of the
Two things
First, short
should be noted.
closure durations result in high masking levels at the release
of the stop.
Second, a longer vowel duration causes the mask-
ing level to remain higher for a longer period of time.
All of the experimental stimuli can be divided into sets,
such
that
set were produced
of a
all members
original token, and have the
16 such sets:
same
duration.
same
There
are
four original tokens times four vowel durations.
Analysis of the
offset of masking in the
simplified by realizing that its
stop closure can be
time course is the same for
For a given set, the only thing that
all stimuli in each set.
differs
vowel
from the
from one member of the set to another is the duration
to which masking
falls before
of closure, and hence the
level
the release of the stop.
Therefore if we can characterize the
time course of the decline of masking
duration
in
each
set,
we
can
use
that
analyze all four members of the set.
_
for the longest closure
I____I_____1Cb__l__-
characterization
to
185
PAM Response Patterns and Phonetic Distinctions
The decline of masking is sufficiently linear that we can
characterize it as a ramp, and fit a straight line to it using
Table 3.4 shows the result of fitting a
a least-squares fit.
straight
the
of the postprocessed PAM responses
line to each
sixteen
with 320 msec
stimuli
closure
to
These
durations.
results are with the PAM operating using adaptation constants
of 5, 40 and 320 msecs, the values used in the previous experiment,
and
used
this
subjects.
the younger
values
in
experiment
to model
lines are
The
the response of
to masking
fitted
that meet all of the following critia:
level
located within
less than 90 percent of the maximum masking
the stop closure;
level value; more than 10 percent above the minimum value; and
(30 to 70 percent of the channels masked).
between 0.3 and 0.7
Typically,
criteria
more
and
than
are
50
sample values
in
included
The goodness-of-fit measure
tion of the variation
the
signal
least-squares
meet
these
calculation.
listed in Table 3.4 is the
frac-
values predicted
(fit-
in masking level
ted) by the straight line.
per
It can be seen that the decline of
masking can be represented well by a straight line.
To
review, we are
of
auditory measure
the
attempting
"length"
to construct
of a closure
a
peripheral
duration,
and
have decided to base this measure on the degree of recovery
from
This,
adaptation.
in
turn,
is
being
measured
by
the
fraction of channels which have recovered sufficiently to have
a nonzero spontaneous rate.
I.....IP
I*
·
We can arbitrarily pick a certain
PAM Response Patterns and Phonetic Distinctions
for exampl' 0.5,
fraction,
more
fraction
than this
masking before
The
end.
an
to
comes
closure
the
recovered
channels have
the
is long
that a closure
and say
of
186
if
from
rightmost
column in Table 3.4 shows the interpolated times at which this
for each of
is reached
masking boundary
sixteen
the
stimulus
sets.
relationship between our masking boundaries
What is the
perceptual boundaries?
and listeners'
Each of the four panels in
graphical answer to this question.
this
show
figure
Figure 3.29 provides a
relationship between
the
the
masking boun-
daries and the perceptual responses for one group of subjects
Each data point corresponds to
and one level of presentation.
the results
a single vowel
for
duration,
for
stimuli
from a
single original token.
In general,
as vowel duration increases, both perceptual
and masking boundaries
increase.
If the perceptual and maskeach other,
ing boundaries
increase linearly with respect to
the connecting
line between data points will be straight.
the
perceptual
amount,
and
straight
the
lines in the panels.
are equal,
masking
line
increase
boundaries
will
be
parallel
to the
the
If
same
diagonal
If the perceptual and masking boundaries
the corresponding
data point will fall directly on
the diagonal.
I
P1
_
__ls______m__l__l__1__1_1
__
___
_
Ij-
PAM Response Patterns and Phonetic Distinctions
It is
exists
clear
from Figure
between
listeners'
the
perception
3.29 that
interpolated
of
the
a
linear
masking
stimuli
as
187
relationship
boundaries
voiced
or
and
unvoiced.
For some tokens, groups of listeners, and presentation levels,
the match appears to be almost perfect
(c.f.,
Token
3, young
group, 60 dB; and Tokens 1 and 3, young group, 80 dB).
On the
other hand, it is also clear that the masking boundary is not
perfectly correlated with all the perceptual data.
of correlation differs between tokens.
The degree
Token 4, for instance,
shows a much greater change of perceptual boundary than masking boundary as vowel duration increases.
Clearly, listeners
are sensitive to some properties of these tokens that are not
being captured by our masking measure.
ing,
given
the
number
of
acoustic
This is hardly suprishave
been
found to affect voicing perception in intervocalic stops.
The
important point
the
is
that
a
features
that
simple measurable
property of
peripheral auditory response shows a strong linear correlation
with the phonetic feature of voicing, that the nature of that
property makes it reasonable to suppose that it could serve as
an anchor
auditory
for categorical perception, and that the peripheral
system
apparently
combines
unrelated
in
a
acoustic
single
response
properties
of
property
the
two
stimulus,
namely vowel duration and closure duration.
What of the other two controlled variables in this experiment?
As we mentioned before,
_
_ r
the dynamic
---- \·*( -·
range of the PAM
1.1
PAM Response Patterns and Phonetic Distinctions
is
too
limited to be able to
or even 10 dB.
However,
that the current model
change
the
188
intensity by 20
input
small changes in input intensity show
not
does
predict
the observed
results:
increasing the input intensity results in longer masking boundaries
(because
report
shorter
of
increased
perceptual
Figure 3.29 as a general
listeners
while
adaptation),
boundaries.
(This
shift downward
of the
is
visible
in
data points
in
the right panels compared with the left panels.)
In
general,
daries.
upward
the
older
This
of
can
subjects
be
seen
the data points
top panels.
by
an
auditory
neurons.
in
increase
In
cessed
with
the
time constant
changes
Figure
3.30
Simon
with
for
results
shows
the
suggest
age
PAM,
time
stimuli
increased
of
this
shift
compared with
this
recovery
To
might
be
time
of
into
an
test
this
experiment were repro-
this
short term adaptation
to
50 msecs.
Table
configuration of
No
other
shows
the
the model,
and
3.5
relationships between percep-
Whereas
are
ideal
1___/1__1__·_1___________1______11(···
boun-
general
translates
were made.
subjects
the diagonal lines marking an
that
constants.
in
the resulting
younger
a
as
the
this
from 40 msecs
values
for
perceptual
in the bottom panels
tual and masking boundaries.
points
3.29
PAM operating with the
in parameter
interpolated
longer
Figure
in
the
adaptation
hypothesis, all of the
reported
and
Price
explained
increase
the age of the subjects.
controlled variable is
The final
in Figure
roughly
3.29 the
centered
data
around
relationship, and the data
_
__I_
II
PAM Response Patterns and Phonetic Distinctions
points
in
for
Figure
clustered
189
older subjects are mostly above the diagonal,
the
the
3.30
data
about
symmetrically
for
points
the
the
older
data
the
and
diagonal,
are
subjects
points for the younger subjects tend to fall below the diagoThis suggests that Price and Simon's hypothesis would be
nal.
an adequate explanation of the differences in the responses of
the younger and older subjects.
3.4
CONCLUSION
of
port
the
correspondence
theory
of the
role of
sup-
in
evidence
In this chapter we have presented some
peri-
the
This theory pro-
pheral auditory system in speech perception.
poses that the transformations which speech signals undergo in
the peripheral
phonetic
auditory system help to
features
speech depends.
upon
the
which
What kinds of
decode the
underlying
content
linguistic
transformations
of
have we
the
seen,
and how might they constitute a decoding of the speech signal?
The evidence of this chapter suggests five types of peripheral
auditory transformations that may help decode speech.
From abstract to concrete: The peripheral auditory system
can transform an abstract
duration,
:
--
onset
"""
vowel
time,
----PL
acoustic property,
888
duration,
or
such as syllable
closure
duration,
190
PAM Response Patterns and Phonetic Distinctions
into
concrete, measurable
a
auditory neural
of the
Whether or not the central nervous system actually
response.
to that property is
responds
property
ures 3.12,
an unanswered question, but Fig-
3.29, and 3.30 demonstrate that the CNS responds to
something highly correlated with the properties we have identified.
From complex to singular:
The peripheral auditory system
can transform a group of acoustic properties that are known to
participate in a phonetic trading
relationship into
a
single
The clearest example of this is the way in
auditory property.
which closure duration and preceding vowel duration combine to
influence the level of masking in the auditory response.
From nonlinear to linear:
transform a nonlinear acoustic phonetic relationship into
can
a
The peripheral auditory system
auditory
linear
phonetic
This
relationship.
is
clearly
shown by Figure 3.12, where the nonlinear relationship between
syllable duration and onset boundary duration
into
a
linear
relationship
between
offset
is
transformed
rate
firing
and
onset boundary firing rate.
From
system
can
continuous
to
segmental:
The
undifferentiated
transform an
peripheral
acoustic
auditory
dimension,
such as time or amplitude, into a more sharply segmental auditory dimension that can form the basis for quantal or categorThis type of transformation is in some sense
ical perception.
_~~
~
~
~
~
~
~
-_
_ ___111_____11__1__1I-
PAM Response Patterns and Phon''etic Distinctions
the
reverse
of the
previous
type,
and
could be described as
from linear to nonlinear.
Auditory properties
taneous
and
steps
rate,
saturation,
191
masking
serve
such as
spon-
to warp uniform
in an acoustic scale into nonuniform steps in an
tory
scale.
Related
to
this
is
the development
audi-
of concrete
response patterns which could serve as an auditory anchor for
the categorical perception of a phonetic distinction.
the experiments discussed
Both of
in this chapter provide examples of
this pre-categorical transformation.
In the first experiment,
the offset firing rate appears to provide a normalizing measure
which
divided
allows
into
the
a
continuous
categories
range
of
of
"fast"
onset
and
an auditory
event that
allows
to
"slow".
second experiment, the recovery of spontaneous
stitute
rates
In
be
the
firing may con-
a continuous
range of
closure durations to be divided into the categories of "short"
and "long".
From
context
free
to
context
dependent:
The
peripheral
auditory system can transform an acoustic signal whose properties are uncorrelated with its properties 50 or 100 msecs earlier
into
an
auditory
time course of
200 msecs.
the
memory.
response
-iiOS·rP·l-----·-·l--
acoustic signal
properties
over
reflect the
the previous
100
or
In other words, adaptation of firing rate provides
peripheral
term
the
response whose
on
auditory
system
This
dependence
previous
context
with
a
of
the
is
simple
kind
peripheral
analogous
to
some
of
short
auditory
of
the
192
PAM Response Patterns and Phonetic Distinctions
aspects
examples
of
peripheral
able
contextual
of
the
perception of speech,
correlation
auditory memory
duration
on
the
between
saw clear
and we
perceptual
memory
and
in the effects of vowel and syll-
phonetic
boundaries
of
continuant-
noncontinuant and voiced-unvoiced speech.
_
I_
_
_
_·
_
_
1
_____1__11__9______1II
I_
___Cs__slsll_____^_____
193
FIGURE CAPTIONS FOR CHAPTER 3
FIGURE 3.1
Schematic diagram showing synthesis parameters of stimuli in
At the bottom is shown the onset and
STOP/GLIDE experiment.
The upper section shows the
offset of amplitude of voicing.
The vertical dashed lines indicate the
change in Fl and F2.
(From Landahl and Maxwell, 1983.)
five syllable durations.
FIGURE 3.2
On the
Waveforms of stimuli with rapid and slow onset times.
right
the
on
left are shown stimuli with 15 msec onsets,
The set of five syllable durastimuli with 60 msec onsets.
tions are shown for each of the onset times.
TABLE 3.1
The average percentage of STOP responses from all of the subjects are listed
for each token. At the bottom of the table the phonetic rise
These bountime boundary between STOP and GLIDE is listed.
daries were calculated by fitting a linear regression line to
the percentages converted to z-scores.
Percent
STOP judgements
for each
stimulus.
FIGURE 3.3
judgements as a function of onset time. - A
separate curve is shown for each of the five syllable duraThe longer the syllable the greater the interpolated
tions.
(From Landahl and Maxwell, 1983.)
50% rise time boundary.
Percent
stop
FIGURE 3.4
The
Boundary duration as a function of syllable duration.
vertical axis is the interpolated boundary transition time for
which subjects would respond with an equal number of "b" and
"w" judgements.
_
_
__
____I
Figure Captions for Chapter 3
194
FIGURE 3.5
PAM response to stimulus with 15 msec onset and 299 msec duration. The nominal center frequencies of each channel in Hertz
and Bark are listed along the sides.
The vertical dimension
The vertical distance between the
is expected firing rate.
zero lines of adjacent channels corresponds to an expected
firing rate 3.0 times the maximum steady state firing rate.
The stimulus waveform is shown at the bottom, delayed by 9.9
msecs, the group delay of the PAM filters, so that glottal
pulses in the waveform and in the PAM output channels are
aligned vertically.
FIGURE 3.6
PAM response to stimulus with 15 msec onset and 87 msec duraSee the legend for Figure 3.5 for further information.
tion.
FIGURE 3.7
PAM response to stimulus with 60 msec onset and 299 msec duraSee the legend for Figure 3.5 for further information.
tion.
FIGURE 3.8
PAM response to stimulus with 60 msec onset and 87 msec duraSee the legend for Figure 3.5 for further information.
tion.
FIGURE 3.9
PAM response to stimulus with 30 msec onset and 299 msec duraSee the legend for Figure 3.5 for further information.
tion.
FIGURE 3.10
PAM response to stimulus with 3 msec onset and 87 msec duraSee the legend for Figure 3.5 for further information.
tion.
I
_
_Ce
11--
---
Figure Captions for Chapter 3
195
TABLE 3.2
Location of onset
nd offset pulses in smoothed composite
response signal.
The size of the onset and offset response
was taken to be the largest value obtained by the smoothed
composite response signal within the temporal limits specified
by this table.
FIGURE 3.11
Smoothed composite PAM responses.
The
acoustic waveforms of
five stop/glide stimuli are shown on the left.
The
smoothed
composite PAM responses to those waveforms are shown on the
right.
The stimuli shown represent the two endpoints of rapid
and slow onset times at the top and bottom, and three intermediate times in the middle.
The longest duration syllables
are shown in each case.
The leftmost vertical line indicates
the location of the onset pulse, while the five vertical lines
to its right indicate the location of the offset pulses for
each of the five syllable durations.
The number above and to
the left of each response pattern is the percentage of stop
responses to the shortest syllable with the onset time shown.
The number above and to the right of each response pattern is
the percentage of stop responses to the longest syllable with
the onset time shown.
TABLE 3.3
Summary of multiple regression analysis of stop/glide data.
FIGURE 3.12
Phonetic boundary of the onset measure as a function of the
offset measure.
Panel A is a redrawing of Figure 3.4, using
the acoustic phonetic measures of transition time and syllable
duration.
Panel B is the corresponding graph for the auditory
phonetic measures of onset firing rate and offset firing rate.
I--arrmr
TTln-------
Il---·---l--cl---*
---
---
-------------------·I---
Figure Captions for Chapter 3
196
FIGURE 3.13
Spectrograms of the
four original
"rabid"
tokens.
These ori-
ginal utterances are referred to in the text as Token 1, Token
2, Token 3, and Token 4 (figure courtesy of Price and Simon).
FIGURE 3.14
Waveforms
of
stimuli
with
shortest
and
longest
closure
and
vowel durations.
These four waveforms represent the endpoints
of both the closure and vowel duration sequences.
All of the
The
waveforms shown are edited versions of original Token 1.
closure durations shown are 35 and 125 msecs, and the vowel
durations are 160 and 220 msecs.
FIGURE 3.15
Audiograms of the two groups of subjects.
The means and stan-
dard deviations of the pure-tone hearing thresholds are shown
for the younger subjects ("Y"s) and older subjects ("O"s).
(From Price and Simon,
1984).
FIGURE 3.16
Percent of "p"
responses.
The
results
for
each age group and
The label
level of presentation are shown in separate panels.
"YN" indicates the younger group of subjects, while "ON" indicates the older group.
The four curves in each panel indicate
the results for each of the four vowel duration (1 =
160
The
msecs, 2 = 180 msecs, 3 = 200 msecs, and 4 = 220 msecs).
dotted horizontal line indicates the 50 percent phonetic boundary
each
for voiced-voiceless
panel is percent of
Simon,
responses.
The
"p"
responses.
vertical axis in
(From Price and
1984).
FIGURE 3.17
Interpolated voiced/unvoiced phonetic boundaries.
The lines
marked with "Y" are for the younger group of subjects; "O"
indicates results
for the older group.
The
dotted
lines
_
rl
I_ _
L __
1'1
__1_11__^___
Figure Captions for Chapter 3
197
indicate the results for the 80 dB HL presentation level,
while the solid lines are for the 60 dB HL.
The independent
The arrows below
variable in this figure is vowel duration.
the horizontal axis indicate that the shorter the boundary
duration, the more "p" responses a subject would make, whereas
a longer boundary duration would result in more "b" responses.
(From Price and Simon, 1984).
FIGURE 3.18
PAM response to stimulus with 160 msec vowel and 35 msec closure.
This stimulus was created by editing the original Token
1. See the legend for Figure 3.5 for further information.
FIGURE 3.19
PAM response to stimulus with 160 msec vowel and 125 msec closure.
This stimulus was created by editing the original Token
1. See the legend for Figure 3.5 for further information.
FIGURE 3.20
PAM response to stimulus with 220 msec vowel and 35 msec closure. This stimulus was created by editing the original Token
1. See the legend for Figure 3.5 for further information.
FIGURE 3.21
PAM response to stimulus with 220 msec vowel and 125 msec closure.
This stimulus was created by editing the original Token
1. See the legend for Figure 3.5 for further information.
FIGURE 3.22
PAM response spectra of stop bursts.
response of the PAM to the release of
Panel A shows the response for stimuli
with 160 msec vowel durations, 35, 65,
Panel B shows the
sure durations.
I^
_
_
_
This figure shows the
the intervocalic stop.
produced from Token 1,
95, and 125 msec cloresponse for
stimuli
_
_I
__ __1____1_(
Figure Captions for Chapter 3
198
produced from Token 1, with 125 msec closure durations,
160, 180, 200, and 220 msec vowel durations.
and
FIGURE 3.23
Output of masking post-processor for stimulus with 160 msec
This stimulus was created by editvowel and 35 msec closure.
ing the original Token 1. The nominal center frequencies of
each channel in Hertz and Bark are listed along the sides. In
each channel, a low value indicates the PAM output is greater
than zero, while a high value indicates the PAM output is
zero.
The stimulus waveform is shown at the bottom, delayed
by 9.9 msecs, the group delay of the PAM filters, so that a
postprocessor output value and the most recent stimulus sample
that produced it are aligned vertically.
FIGURE 3.24
Output of masking post-processor for stimulus with 160 msec
This stimulus was created by
vowel and 125 msec closure.
See the legend for Figure 3.23
editing the original Token 1.
for further information.
FIGURE 3.25
Output of masking post-processor for stimulus with 220 msec
This stimulus was created by editvowel and 35 msec closure.
ing the original Token 1. See the legend for Figure 3.23 for
further information.
FIGURE 3.26
Output of masking post-processor for stimulus with 220 msec
This stimulus was created by
vowel and 125 msec closure.
See the legend for Figure 3.23
editing the original Token 1.
for further information.
__1_
_
_
_L-·XII-
_ls-mrc-----__-------··-CII.----.
__111_1___1_1__1_11__--
199
Figure Captions for Chapter 3
FIGURE 3.27
The
Three representations of a stimulus perceived as "rapid".
top signal is the acoustic waveform of a version of Token 1
with a 160 msec vowel duration and a 220 msec closure durasmoothed composite PAM
is
the
signal
The middle
tion.
response pattern, produced using the postprocessor constructed
The bottom signal is the comfor the stop/glide experiment.
posite masking detector output, smoothed using a 5 msec half
hamming window, as described in the text.
FIGURE 3.28
The signals shown
Three representations of endpoint stimuli.
contain the four possible combinations of shortest and longest
They were all constructed from
vowel and closure duration.
Token 1. The signals on the left are the acoustic waveforms,
the signals in the middle are the smoothed composite PAM
conusing the postprocessor
produced
response
patterns,
structed for the stop/glide experiment, and the signals on the
right are the composite masking detector outputs, smoothed
using a 5 msec half hamming window, as described in the text.
TABLE 3.4
Linear fit of decline of masking during stop closure: "Young"
These results are for the PAM operating with a
analysis.
short term adaptation time constant of 40 msec, the value used
Approximately 50 sample values are
to model young subjects.
included in the least squares fit for each line.
The initial
masking level, slope, and closure duration at which the masking level reaches 50 percent are interpolated values.
FIGURE 3.29
perceptual
and
masking
boundaries:
Relationship
between
These results are for the PAM operating
"Young" analysis.
with a short term adaptation time constant of 40 msec, the
The horizontal axes are
value used to model young subjects.
interpolated masking boundary durations, and the vertical axes
Data points
are interpolated perceptual boundary durations.
cms
--------------
-
-·a-----
-··------------rr·-
-
---·
-----
-
Figure Captions for Chapter 3
200
would
fall on the diagonal line if there were a
between the perceptual and masking boundaries.
perfect match
TABLE 3.5
Linear fit of decline of masking during stop closure: "Old"
analysis.
These results
are for the PAM operating with a
short term adaptation time constant of 50 msec, the value used
to model old subjects.
legend for Table 3.4.
For
more
information,
refer
to
the
FIGURE 3.30
Relationship between perceptual and
analysis.
These results are for
short term adaptation time constant
to model old subjects.
For more
legend for Figure 3.29.
1
31
11
_
_
masking boundaries:
"Old"
the PAM operating with a
of 50 msec, the value used
information,
-111_.__1-__
11_
_
_____lj
refer
to
_I
the
FIGURE 3.1
I-
-
-
-
I
_
_
_ _
-
-
-
201
m_
_ .
_
_
-
-
-
-
-
-
-
-
-
Pd
0
eV
_
_
O
-4
C
a)
E
a,
C)
4-
O
a,
- -
-
-
-
-
-
- - - - -
I
Iex""
V)
".
-co0
._C
/:
0
'-
O
a)
CM
'a
n
UE
eo
I
0
C4
E
r-
6"
_f_x_
m
0
a
0
Pd
o
49
4
-
I
0
0
1
0 0
P. Pd
P.y @
-
0
0
Pd
--i la
FIGURE
3.2
DURATION
in ms
ItIILk
I ll11 1.l1Ll,
.IlllLt
,i
'
.
I
SYLLABLE
ONSET TRANSITION: 60ms
ONSET TRANSITION: 15ms
rr""
202
...
....
L
w"
299
r- -- -I .........
! '11I .IlI
111
III~~~~~~
rrrr
I
s
Zr a s
I',
a, [
E
.
,E
f
lE
s
ls
T-, ,1 ,rI
228
158
W
_-. . .I 0-R
_I
It
I_
'1I ... .
Il1. .l
...1
..ll l
123
i
iI
...Ad,
LI_)I
-___
0
F
1
I_
¶
P
...
I- ........
II
I
II
I11r1iq
500ms
I`-------""I-~---`
---
87
TABLE 3 1
203
PERCENT STOP JUDGEMENTS
ONSET I
SYLLABLE DURATION in
TIME
in ms.1
299
228
158
ms.
123
87
-____+-____________________________________
15
98
100
100
100
100
20
100
99
100
100
99
25
99
99
98
93
801
30
94
89
80
58
17
35
58
46
23
12
5
40
25
23
8
3
1
45
10
8
2
2
1
4
3
1
1
0
1
2
1
1
1
1
1
0
1
0
0
0
0
50
55
60
65
I
I
I
I
I
I
I
STOP/GLIDE BOUNDARY RISE TIMES
37.6
."·""
D"sBI·m·°PIT
·a(e
aaraol--·--·---r.
51
36.1
33.0
1
in ms.
31.0
27.5
-----
II
FIGURE 3.3
204
Percent /b/ Responses by Syllable Duration
.AA
I VU
75
V,
(n
C
0a
0.
cn
so
9
X
A=
.0
O=
B
0=
3
D~ .
7
25
45
40
35
30
25
20
15
Onset Transition Time in msecs
_ _I
____________________
__·1____11
'1
·_XI
I
FIGURE 3.4
PHONETIC BOUNDARY--TRANSITION TIME
AS A FNCTION OF SYLLABLE DURATION
C
0o
e)
C
C4
(n
L0
O
ca)
E
'
30
aL_
m
100
150
200
250
Syllable duration in msecs
206
FIGURE 3.5
HERTZ
11DIRA
BARK
1
102
2
203
3
384
4
486
5
587
6
66
7
761
8
913
9
1065
I1
1218
11
1421
12
13
14
n l~ l1 I
.
i
.
t11.
I i
1i
I
I I
Illtalllllllltlllll!!1
, .......
.... LL[ L!L L fl !
,'J[[J[_J
iIJ. J J .L
15
i
i
i
11Mi11
I
1
X
1
01
i
1674
l
,
,
L,L,[ ! LLLI IL~__,
JlJ l
1979
2384
2798
3298
16
·
17
18
NS
]"
·
i
' .......
.. . . . .
i['''J tlJii~''ii[ ..
3906
0 Mil4617
5428
19
6392
20
S
-I
r'
I
----
-·-
--
--111
-----------"x
FIGURE 3.6
207
2
1D5 AR
BARK
HERTZ
1
1b2
2
I.
3
AA
-
~~~~I,
.....
'"
I I I
I
,,
. .
.'"i
.' '
.
203
.
334
I'I"'
I I I',",""
',
I ,
I''I"
I I '' .I
, " -
4
486
5
507
6
66
7
761
8
913
9
1065
13
1218
11
1421
12
1674
.11L
13
1979
i--I-li-.
14
I
2384
is
2790
16
3298
17
3906
18
4617
19
5428
20
6392
-
--
rReP--
ass
rr
----
-
208
FIGURE 3.7
HERTZ
O1DAR
BARK
1
1B2
2
203
ll Okll
1111111
,
.ll..
J -.
~,
3
4
,1
i
1 .ti
. t .1 ,
304
i1iij11111_llW
5
507
6
66e
7
761
8
913
9
1065
'
10
~l
lljl
"ll
l
~i
ll
iL..
1218
11
1421
12
1674
13
1979
14
I
T
2384
I
15
279
16
3298
17
3906
18
4617
19
ff
1
17,
~_1
X
1 1 1
I
T
28
-
iii
5428
1
6392
I
MS
..
5B
L P.
. ,.. I
-uuuI
IfumE
Ir
I I
I
1
406
it
lslll··-··1···---··--·I(·--(··.
I
.
'-
- -
MS
FIGURE 3.8
209
.
BARK
HERTZ
1
162
2
283
3
364
4
U86
5
587
6
I
i
7
761
8
913
9
18
1065
A 1......1......
AwTilI TTTTr TTTTT
1218
I1
1I21
12
1674
13
1979
14
2384
15
2790
1.6
I
3298
17
3986
18
4617
19
5U28
ILLL
28
--
666
6392
I
----------
hl
FIGURE 3.9
210
04D1AA
BARK
HERTZ
1
182
2
283
3
304
4
486
'-,^4,,,,, .,IIIULL
§,,,,,,,,
. .*
5
.
. . . . .
57
Ip...'**1111111111111111*
668
6
ll
7
I
11m6lltllilllJuI
761
..iIuuruIiAitliIJIailil
......
J"
8
913
9
1065
10
1218
11
* 1
I*I1.
j,,lll,__11111t111
1421
.
1674
12
13
ki liljl
ii
i
i
ill
l1
i,,,M,,,'
~~~i
l
i
1979
l
l
itil
'
{
1
'
i
'
i
2384
14
15
16
..........
- I
r
-
-
''1'
J
As
I
I ,'
18
I
-
.... '
.
1.
I -'1 .
JI
Lit
I
. I I
I
_
I I
.
.........
-
-
~ ~~iiiiiI
O
J
. . . . . .'I
17
19
2798
.iI
.
I
.
Iiq
-
J.
-a
[
..
[
',II
I
. .
- I
..
.
.
I..
.i. . . ' . . I.I. '-
I
I
I
I
I
I. . .
- - . .
- L I
I
_i
{l1
. I- - ,_.-------. . .
- .
-
]
.
I
4'
-
.._ 1
-
-
.
3298
3986
4617
5428
6392
2{
1..-1-1--_--.____.
..?-
.........
iI
i fI . [t't.
i iFI'
G~~~h....
. .
.
.....
'i I'....'I
ti l
t,.I , .,lI------,,I, I
-
i..
.
_..
-
--
..-.sII-I^--
-sl--0-
--
---
211
FIGURE 3.10
BRRK
1
2
04D05RA
1
-
i.
.
.
.
.
HERTZ
182
1C
---
.
I
i
.
,
i
l
I
It
I
li'
1 '
I
T
I'" I
,
II
I
i
i
i
I
I
I
I
VI
3
4
213
384.
.
I.
.
.
.
. ·
~.
[
i
1
I
I
I
I
I
I
.
''~
I
.
.
. .'
'1
"
.
I
. "
!
i.
. . Il"
t"
I
I' .
.' .
.
.
I
I
,
I'I Ii
416
5
507
6
668
7
761
8
913
9
1065
18
1218
11
1421
12
1674
13
14
,[l,
1979
4
2384
L5
2798
16
3298
17
3906
18
4617
19
28
------------
--I
-1-1.......
----I-- ----
.........
I~~ 5428
6392
TABLE 3.2
212
LOCATION OF COMPOSITE ONSET AND OFFSET RESPONSE PULSES
PULSE TYPE
onset pulse, all
APPROXIMATE POSITION
(in msec)
tokens
95 to 100
offset pulse, 87 msec. tokens
offset pulse,
123 msec. tokens
offset pulse, 158 msec. tokens
120 +-
3
I
155 +-
3
|
190 +-
3
260 +-
3
330 +-
3
offset pulse, 228 msec. tokens
offset pulse,
_
299 msec. tokens
I
_ 1_1_·____·______________/_____11_1
....
_ _ _·
_
FIGURE 3.11
ACOUSTIC WAVEFORM
213
COMPOSITE AUDITORY RESPONSE
00I
o
I
fto
-
ONSET
smhI -
TRANSITION
in ms
15
30
1
i.t
35
l
I
-25%
40
/
i
1
60
I
0
11 11
1'
I
so500ms
._
..
_
TABLE 3.3
214
RESULTS OF MULTIPLE REGRESSION ANALYSIS
OF STOP/GLIDE SYLLABLES
Sample Size:
Independent Variable (Y):
55
Z scores of % stop judgements
Acoustic Phonetic Analysis
Dependent Variable 1
(X1):
Onset time in msecs
Dependent Variable 2
(X2):
Syllable duration in msecs
Partial Correlation, Y with Xl:
-0.92
Partial Correlation, Y with X2:
0.15
r2 =
0.02
0.94
R2
0.88
r2 = 0.85
Multiple R Coefficient for
Regression
of Y with X1, X2:
=
Auditory Phonetic Analysis
Dependent Variable 1
Dependent Variable 2
(Xl):
(X2):
Partial Correlation, Y with XI:
Partial Correlation, Y with X2:
Multiple R Coefficient for
Regression of Y with X1, X2:
Onset firing rate
Offset firing rate
0.93
-0.18
r2 = 0.86
r2 = 0.03
0.97
R2 = 0.94
Student's t ratio of Y
residuals
from acoustic and
auditory regression:
2.68 with N-3 = 52 df
(Rejects alternative hypothesis--that acoustic residuals
are less than or equal to auditory residuals--with p < .01)
_____
__
_____
__
I__
___1_1_·_
______11_1_·1_·_1__.
F~IGURE
3.12
715
A: ACOUSTIC-PHONETIC BOUNDARY FUNCTION
C
O
._
._
u
5
c
3
0
m
Iu
in
c
-
'a
Syllable duration in msecs
B: AUDITORY-PHONETIC BOUNDARY FUNCTION
_
·
I
I
I
I
I
.75
80
.85
I
i
I
.70
.65
Cb
Ii
4.)
C0
.60
Cu
CA
.55
Offset rate in normalized units
__IICI
_
___
_11
_·II__
1
1
_
i
FIGURE 3.13
--
I
-
-
-
_.0mu ...
216
-
--Ip _A
-
-o
-
-mu--
*C
mmam.
__
-L
Cq
"it
.0
*-
4
- .l
i
I
-
I
I
~L ~
-
i
-- Nnmm.
a
-
l
mI
'
A=
}I
*
-
X
~ZC
-
*^*
I
43
M
-
Z-
=yEF
I
. I
. n
C
I
I
..GW.
-~
- mb ...
m
--
L
-_
-
.
.
.
o
-
~
.
L
,A
:LM
a
O
I
%:
yl
co
'·
·
---
-
I
V
-------------------- ---------
I
,in
------------
I
C
I
I
IT
-
--
----
-----------
I
,n
---------- ·---
I
cu
I
411uB
0:
Ix
FIGURE 3.14
217
VOWEL
CLOSURE
DURATION
DURATION
in msec
160
in msec
I
rr-n
-m
,1
'A
O
35
P1~-IIi1
I
Of~U·
50ms
.. I
160
-an
s z xs r
TI(((L1Ilr
220 r"m
0
220
A'9 r
L
'I
-
-L·-
IY
v.-
't
35
I Mm mmL%4-w"''v
..
rIr'
s
,
500ms
liAIe_
'1
vTr~rtr~rrr
125
125
_Y1
-
-
218
FIGURE 3.15
CU
cn
6C
SI
U9
U)
cr
C
- F- >- -- i
C:
0
i
I
z
U.
(,
->
rn
0
C>-i
o·
N
0
Iq
C
!- M>--H0-·Ill>
-
z
LUJ
z
0
I-
FI.->.
LJ
0
LL.
t.o
o
is
Q
CO
o::
.;
o
U,
-
(
0to
0
Cu
-->-
---
0
I
!
r
o
0
0
_
o
N
I
o
lI
o
In
(SP) S3A31 GIOHS383HI
____
_
I
9NIV3H
_..1------·
219
FIGURE 3.16
ON, 60dB HL
YN, 60dB HL
IIAA
IvU
100
m
!- -;;,
/
80
80
I
II)
60
60
40
40
/
I
I
20
20
0
0
I
JL
I
I
.
I
35 65 95 125
SILENT CLOSURE (msec)
35 65 95 125
SILENT CLOSURE (msec)
ON, 80dB HL
YN, 80dB HL
II
100
100
80
80
?
I
_
I
I
60
60
I
Il
40
I1
I/
,
40
I
II
20
20
0
1
35
65
0
It .
95
125
SILENT CLOSURE (msec)
.1
!
1I
35 65 95 125
SILENT CLOSURE (msec)
FIGURE 3.17
224
0
0
4
E
0m,
-
or)
0o
LlJ
0
N
00
()
cr
0
0
z
0
Xo
l
Do
CR
0
uJ
>T
0-
I%%
CD
14%.
0_J
I
I
0
Cam
0
0
(39sur)
NOIJVflnCi 13MOA
_X____PI^II1__/_··a_··--··I
-
--1-·I-
.-·
w
C
LU"
I
0o
0 or)
LU
0
(0
- -1__11_---___11_1__1__.__1---_1---·
221
FIGURE 3.18
BARK
HERTZ
RAP IARK
1
192
2
293
I',..,
3
394
496
A AA
"AA- Ai: AI
5
6
&'"" I..
t
' ......
7
587
,,[i
L1
..
686
761
:ai
8
913
__~~~~~~~~
,,
"
,
i·i~!ijiiii,,IiiijiiIJj]i'
1065
9
- 1218
11
1421
12
1674
13
1979
14
2384
15
2798
16
3298
17
3986
18
4617
19
5428
20
I
. I-
6392
II II
I
I I I "1ll1lI
-------
111
FIGURE 3.19
BARK
222
RAP 13AK
Ail _",,"-
1
I
i
i .
.i
.
j
-i
i
i
i
i ·
! ;
HERTZ
·
.
i
i
i
i
i
A-i._ .
i
i i
i
102
i
2
3
203
J
~
r
1
I
1
t
i
|
~sI
I .
..
.
.
I
_L
11~1&L
4
.
.
.
.
.
I
I
Ii I
I
I
1
Ai
i['[i
[[
i
I
I
4
j
k111
1106
5
507
6
665
LLj~u.
7
8
..
9
13
304
1
,.........
*
I
_LIWrW_
r, ,
.. r
. g'j$
..
.
7681
913
1I,,..
1065
_
'U..
l§4
|raf
B6
'rfiW
'1'w
B|&
1218
1421
12
1674
13
1979
14
2384
15
2790
16
3298
17
3906
18
r
4617
"I'
19
~~~I
.l . I
20
Ms
' 1''1ill
Il i
L I, 1
.
I
140
. I I r
. .. . .
I I
.
I
I
I 1IfiI 1I 1 -?-rj r-I
. . . .
I
, -1 --, - II II II I-I
?
.
PI
&,
I ,
g . . .
. . I
I
I
I
i
5428
6392
508 MS
it IHI- i .
I
-----------------·----------·---------------·--
--------------------------
--------·-----
FIGURE 3.20
BARK
223
RRP13ARK
HERTZ
192
2
3
293
444klf
4t44
4
rllr
_l~~r _44
I
384
4
496
5
6
i
--
,i'iu
--.
. ', .
,~fIA,.j
.
. .l .l.,
1 ,;
,i I!
597
_AA
_1I1
A
__
669
,,_,
'
','
7
761
8
9
913
~,
i
i
I .
,,,i.
t,
mI
~~~~~~~~~~~~~~~i
- _
l
~
;
j
I
i
i
'
1065
j,
19
1218
/iiiijiiy
L1
j
1421
12
1674
13
1979
[4
2384
15
2799
16
3298
17
3996
18
4617
19
II
. II
.
. . .
29
,
5428
6392
;;
i2 11
~4
i'
nxarssr-cL1·--·I·
,
Ir 11 lllllllr
I
FIGURE 3.21
BARR K
224
RRAP 133K
HERTZ
182
2
3
283
-rrrii
AIAAAi*~4rI
384
AAALrr4
4
406
5
507
i. ,,,.ULLv""'-u,
Jtt·r
6
660
7
761
8
913
9
1065
10
1218
11
1421
12
1674
13
1979
14
2384
15
279g
16
3298
17
3986
18
4617
19
5428
6392
20
S
I
I
I
--
~
"
- - - - --
1
225
FIGURE 3.22
PAM RESPONSE SPECTRA:
STOP
BURSTS
1,~,
:,4',L
rjsd
.-,-'B
3
A:
35 to 125 msec Closure Durations
125
c
0
(d
0
20
1
CO
0
a)
.
cU
',N
3
E
B: 160 to 220 msec Vowel Durations
0
z
160
0
20
1
Frequency in Bark
-
-r
--L------·lll
IllllP-----"----
226
FIGURE 3.23
RAP 10AKN
BARK
HERTZ
I111111 I II
1
102
2
3
4
203
. .. . . . . . . . I.
·
ii -1
_1
! . . ~. .,1
. !.!
! . . ! . . .,,,,
_iiI
1
1
314
1
5
50?
6
66W
7
761
8
913
9
1 65
1218
11
12
'""
'l .
1674
13
1979
14
2384
15
=r-r-r-lilans
II
16
- - - - -- -- -
2799
I"
- - - -
-
. . .
17
. .. I I.
18
.
19
. I I ..
II
.
26
.
I
-"-----"`---ll~x"~~""ll~-~~
.
. . I . . .
I I
I I I I
3298
. . . .
39W6
. ..
4617
.
''
III'
I.
I
I
r
I I
I-
I
I, Ij r I
,
.I I I
I
I
I
. I Il
I
i I I
. rI
iI
I
.
I
'`
---------`----I--"'
----'---I--------^-`------`----I----
I
.
56
,
'
. . . .
.111111
,,I
I II
. . .
Ms
--
1421
jjlll
i iiiiim
.. I I.............
i.
11 1 1
I
I
5428
I
I
MS
6392
227
FIGURE 3.24
i'58 , te.}.':"!.t
..zi ,,~ ,
RRP13RKM
BARK
HERTZ
1
102
2
203
3
3W4
4
406
5
507
6
660
7
761
8
913
9
1065
10
1218
1i
1421
12
1674
13
1979
14
2384
15
279W
16
3298
17
3906
18
I I
I
I
19
20
r
.
.
|
.
I
I . I
I
I I I I.....
I ' l l
I I
.
. Li
I I I
k1t
1 . . .
I
I
t
.f
.
.
. .
. . . . .
.
.
.
.
.
.
.
.
T. . . . . B
I
.
.
.
.
.
.
..
I I
.W . .§
.
.
.
I
.
.
.
.
. . . . .
.
.
.
.
.
.
- - - - - - - - - - -
I, I~fl . . I
.
.
4617
.
.
5428
_
6392
I
-
- IIC
as
--s
-----
··----------------
--
-
L------.-C-.---0·11·1_--·1_------
---------------_-----_II
-·
228
FIGURE 3.25
BARK
HERTZ
RRP138RKM
182
1
2
,~~~~
~ ,.
IIIII~s~
3
I11
,11
. . .,,,. . . . w . . . .
11111i11111
.
!
. . . . . . . . .
I
203
_...
II
w
384
486
4
1111
1
X1
T--T--,"1.... 1
1111
5
6
1
.1.........
1 1 1_111l1W
507
660
761
7
8
1I 1
a,,, ,,,1, 11 1X
1111
..
~
1111
II 11111 ,11
12
13
fI ll . III ..
§
14
II 11 I,.....
1 .
lllll|II
· Hill11111
913
'
l
~m!~Jm1818111111111llll I
, i1"'"illilall
1
111
|l_ l1lllllr,_
~"
11111·
I ijmmmlli"'
~IIIIIIIIIIIImIIII
9
11
1,1
1~11
#111
11
11
1111
1065
1218
1421
1 I 1 11
1674
lllll.ll........ll.U..
....
,11
11 1
I1 IIffII 1111101!1WI
1979
.
2384
15
2790
16
3298
3986
4617
18
19
.
I
-
-
- -
-
-
I
I I I I
. I
. I
I
ft171
I
I IrI' 1jllllljlj l I
-
-----~-
-
-
-
-
-
l
5428
. .
6392
FIGURE 3.26
229
HERTZ
RAP1.33AK
BARK
11II 11111111 1111
2
102
I
LII
J~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~I
IU
3
203
364
4
406
*T
5
507
6
660
7
r 1111
11 ll"
.,ll
I
111g
!!.1
761
8
913
9
1065
16
1218
11
1421
12
1674
13
1979
14
2384
c
15
~ii~
ii ,
I 1-
ti
i !
t 11Pi11111
i i I i i i · r Ti
.............
i
....l !
16
17
i I i ii
.
I
I
I
II!
2790
l
3298
I II
3906
i
I!
i
18
4617
19
l
26
I
I
5428
6392
S
I
-I
-
------ -----------------
FIGURE 3.27
.
11W rrp z
23I
.
l
00 msecs
0
fr'l I I
I..
I
I
I
I
I
500 msecs
0
a
I
Ill
l i lL
.I
-.
i,.
.IIJl
I - - - -- - - -- - - I- - - -- - -
l
500 msecs
0
_
II.
--
__
I
FIGURE 3.28
231
_
-J
W
-J
z_
0
<
t
t-
I
u
I
B
0
0
o0
.- e
O
a.
4
0n
w
E
X(
rn
E
w
E
0
u
[]
0
U.
w
it
r
f
0
oC
r
!
CO
O
0
0
c
k
E
9
I~~~~~)
o
00
CO
L
c=
E
U
O
O
0
z
1
V-
>
°D
13
D0w
O
o
c
'
_
__
232
TABLE 3.4
LINEAR FIT OF MASKING LEVEL DURING STOP CLOSURE:
RESPONSE OF PAM FOR YOUNGER SUBJECTS
Token
Number
Vowel
Duration
(msec)
Initial
Masking
Level
1
1
1
1
160
180
200
220
0.89
0.87
0.87
0.87
-6.10
2
2
2
2
160
180
200
220
3
Goodness
of
Fit
50% Masked
Boundary
(msec)
-4.84
-4.56
.92
.94
.93
.94
64.6
64.3
75.8
81.4
0.89
0.91
0.91
0.92
-6.31
-5.96
-5.96
-4.84
.82
.84
.84
.88
61.3
68.8
69.01
3
3
3
160
180
200
220
1.02
1.06
1.03
1.02
-9.67
-8.94
-7.61
-6.74
.94
.94
.92
.92
53.7
62.1
69.4
77.8
4
4
4
4
160
180
200
220
1.03
1.00
0.92
0.90
-10.9
-9.05
-7.23
-6.06
.98
.97
.97
.95
49.1
55.3
58.4
65.5
Il____q____________s____________
_II
Slope
(l/msec)
(x .001)
-5.80
I_
____I___I
87.0
FIGURE 3.29
YNG: 60 DB
z
YNG: 80 DB
ii
i ,, ,,,i, i,,,,
,,
Cn
z
233
cli
zon
I
L
z
10I
z
o
0
IC
r.
z
0
C
na:
r_
c
0
z
Ca
u
I-
L
0
z
a
z
o
I
I
I
I
I
I
I_
I I
I
a_
I
, ,I,
I I I
I I I
I I
10
URTIN1BOUN
IN M.
MASKING BOUNDARY DURATION IN MS.
MASKING BOUNDARY DURATION IN MS.
LEGEND:
x 'TDKE L
+ TDKEN
2
a TOKEN
3
* TOKlEN
OLD: 60 DB
OLD: 80 DB
,,
c,
zrl
z
z
·
X
z 101
0
Z
o
I-
I-
¢
Cc
0
zC
Ia:
C:
0z
0O
c
·
o
z
C
·
I-.
c
o
w
O
a.
_
0
II
I
I
I
I
I
I
I
I
RSKINC BOUNDR
MfSKING BOUNDR
.
I
.
I
I.
I.
I.
OURRT IO aIN
II
..
I I I I I
I
I I
I
I
S.
DATION IN MS.
MNSKING BOUNDRRY OURTIrON IN MS.
"
TABLE 3.5
234
LINEAR FIT OF MASKING LEVEL DURING STOP CLOSURE
RESPONSE OF PAM FOR OLDER SUBJECTS
Token
Vowel
Duration
(msec)
Initial
Masking
Level
1
1
1
1
160
180
200
220
2
2
2
2
(l1/msec)
(x .001)
Goodness
of
Fit
50% Masked
Boundary
(msec)
0.89
0.86
0.88
0.87
-5.14
-4.76
-4.32
-3.91
.92
.94
.93
.94
76.2
76.1
88.0
95.0
160
180
200
220
0.88
0.91
0.92
0.94
-5.21
-5.06
-5.17
-4.40
.82
.84
.84
.88
73.5
81.4
81.3
100.0
3
160
3
3
3
180
200
220
1.02
1.07
1.06
1.09
-8.14
-7.77
-6.83
-6.60
.94
.94
.92
.92
63.8
73.6
81.5
89.8
4
4
4
4
160
180
200
220
1.04
1.01
0.94
0.93
-9.01
-7.74
-6.25
-5.51
.98
.97
.97
.95
59.5
66.4
69.8
77.6
Number
I_
II_
I_
Slope
__
235
FIGURE 3.30
RAP N
DB
YNG: 60 ~~~~~~
...
C;
x
I
1--
z
n
z
z Iog
z laS
0
I-
a
YNG: 80 DB
0
r
-
cc
ac
Cl:
C
a0
z
0
a
a
m
z0
Li
u
I-
w
z
I
o-wa
I
II
I I
I
I
I
r.
I
_
MASKING BOUNDARY DURATION IN MS.10
_
I
I
.
I
.
I
m
I
.
m
I
I
.
I
IL
I
I 1
100
'
MASKING BOUNDARY DURATION IN MS.
NASKING BOUNDARY DURATION IN MS.
LEGEND:
x
+
a
*
tIOIN
TEN 2
DEN 3
TOKEN
&
OLD: 80 DB
OLD: 60 DB
I
x
(n
r
z
z
Zr0
Z
I
I
I
o
o
Cc
C-
C
0
a:
0
o
z
o
a
a0
1.
(a
>
aII. e
0
I
I.
I.
I.
I.
I.
I
I
I
..
I
I
I
0
L-
a
I
aRSKI
MISKING BOUNODARY DURATION IN MS.
I
I
I I
I
OUNDRY DU
I
I
I
I
I
I
10ATION
IN MS.
MASKING BOUNDARY DURRTION IN MS.
236
CHAPTER 4
EXTENSIONS AND FURTHER EXPERIMENTS
The previous two chapters describe a preliminary examinaauditory
system
simplifies the encoding of phonetic information in the
speech
tion of
signal.
hypothesis
the
The
indeed
might
order.
results
seem to indicate that such a hypothesis
be warranted,
chapter
this
In
that
and
be
will
of
improvement
further
outline
I will
extended.
which the current work could be
extensions
peripheral
the
that
One
described.
is
is
in
directions
in
research
some
Two major types of
extension
the
other
The
the Peripheral Auditory Model.
and
is
the continued re-examination of acoustic phonetic experiments,
particularly those involving
temporal
from an
effects,
audi-
tory perspective.
4.1
EXTENSIONS TO THE PERIPHERAL AUDITORY MODEL
The current PAM was
theory
of
peripheral
developed to test the correspondence
auditory
function.
intended to be a general purpose
nor
a
physiological
model,
Because
it was
not
functional model of the PAS,
only
aspects
of
the
peripheral
auditory response that were judged to be most relevant to consonant representation were modeled.
In the discussion below,
relevance to the representation of speech will
the
primary
criterion
_
for
determining
_pl)p·jiilli·P1-Ci
the
continue to be
nature
of
-_
---------
the
·
~ -_
1__C_1__
Extensions and Further Experiments
237
improvements to the current model that will be suggested.
4.1.1
Extending the Spectral Analysis Stage
The PAM should be extended to include the effect of the
outer and middle
ear on
invariant
filter
eliminate
the
spectral
need
data on
balance
the
to
approximation
for
them in the PAM,
using
acoustic waveforms.
to
and
the
PAM
characteristics
speech
result
would
response.
the
of
include Mehrgardt and Mellert
linear timeshould
structures
these
preprocessing
A
signals
realistic
in a more
Typical
external
and
(1977), Rabinowitz
before
sources
for
middle
ear
(1981), Shaw
(1974), and Zwislocki (1975).
The
current
PAM does
not
model
the
true
phase
teristics and group delay of the basilar membrane.
linear
delay
phase
across
characteristic
all
is
bank
filter
used,
with
channels.
a
charac-
Instead, a
constant
This
is
group
convenient
because it results in a peak response to a burst, for example,
that occurs at the same time at the output of each filter bank
channel.
respond
However,
in
this
the
peripheral
manner.
The
auditory
latency
of
system
does
auditory
not
neural
response to an impulse depends on the characteristic frequency
of
the
neuron,
very
with
low
CF
fibers
msecs later than high frequency fibers
·
IICI-·-
responding
(Kiang, 1965).
up
to
3
Extensions
It
and Further Experiments
would
be
appropriate
238
to
extend
the
PAM
spectral
analysis stage to include more accurate cochlear phase characteristics.
Pfeiffer
and
Kim
(1975)
phase response data from a Fourier
have
derived
analysis of the discharge
patterns of fibers as a function of their position
basilar
membrane.
Their
data
cochlear
suggest
that
a
along
linear
the
phase
response, with a slope that is dependent on distance along the
basilar membrane, is a good model.
Figure 4.1 shows the slope
of
phase
lines
seven
fitted by
locations
hand
along
to
the
their
basilar membrane.
more recent studies (c.f. Allen,
cated
phase
data
characteristics,
result in a group delay that
1983)
even
for
fibers
Although
at
other
show much more compli-
this
simple
model
would
is smaller for higher CF fibers.
This in turn might affect the shape of such whole-nerve composite responses as were used
in property extraction algorithms
in this thesis.
The current PAM spectral analysis
lar
membrane
parameter,
as
an
essentially
time-invariant
linear
stage treats the basi-
one-dimensional,
system,
with
its
lumpedposition-
dependent physical parameters acknowledged only in the use of
different parameters for the twenty filter bank channels.
of these
All
assumptions are either open to question or known to
be false, and may affect the validity of the spectral response
of
the
PAM
transmission
_
to
speech.
line
II-------
----
models
Possible
(e.g.,
-I·l/··I(LIII(IIB·-
improvements
include
Lyon,
Zweig
1982;
using
et al.,
___·_C__I_____I(I1_1__-1111_^----
Extensions and Further Experiments
239
1976) with a larger number of output taps than the 20 channels
of the current
use
of two-
cussed
by
PAM
(Lyon proposes
64 channels);
and three-dimensional models,
Allen
and
Sondhi
(1979),
or
such as
even the
those dis-
and
Steele
and
for
speech
processing
Taber
(1979a,b).
An
might
even more important
be
to model
response.
the
extension
nonlinear
nature
of basilar membrane
Nonlinearities have been observed
in the
ratio of
middle ear to basilar membrane response at characteristic
quencies
(Rhode,
1971),
indicating
that
mechanics might contribute to a nonlinear
response of fibers to spectral peaks
occur
at
their
CF.
Further,
basilar
membrane
"sharpening" of the
(such as formants) which
nonlinearities
interaction of spectral components
fre-
involving
the
in complex tones have been
observed in both physiological and psychophysical experiments.
These include the reduction in response to one tone caused by
the
presence
of
another
(two-tone suppression), and
the gen-
eration of a third tone caused by the presence of two primary
tones
(combination
have been
and
tones).
implicated
Robles,
1974;
Basilar
membrane
nonlinearities
in both of these phenomena
Kim
et
al,
1980).
It
is
(c.f. Rhode
reasonable
to
assume that these nonlinearities would have an effect on the
auditory
formant
representation
frequencies,
together.
A
number
of
speech
especially
of
if
nonlinear
sounds
the
containing
formants
cochlear
strong
were
models
have
·
close
been
Extensions and Further Experiments
(Allen
developed
1980) which account
1979;
Sondhi,
and
for
240
1977;
Hall,
et
Kim
al,
these phenomena with varying degrees
of success.
4.1.2
Extending the Transduction Stage
In this
the transduction stage
combination of the
viewed
as
the
Following
current
form the
stage
can be
(c.f. Rabiner and Schafer, 1978,
the
channel,
the
each
representation
transduction stage
of the
using
a
nonlinearity
common
use
into
modified
in
vocoders:
an
the
alternative
accordance with
is
speech
separate transduction model.
waveform
magnitude
the
This constitutes
then modified in
function
derived
This is an example of a
of
transformation
representation
some
of
with
In the PAM, the resulting
from a
of
input signal
is calculated and decimated.
the analysis stage of the vocoder.
the
its
convolution of
response of
complex output
parametric
In
to
A schematic diagram of such a system is shown in
4.2.
impulse
PAM.
filter bank and transduction
a channel vocoder
Sec. 6.7.3).
Figure
of the
possible extensions
some
section we describe
which
Whether
model.
a
or
speech
is
then
not the
speech waveform is then reconstructed (that is, whether or not
the
on
resynthesis
the
purpose
stage of the
of
the vocoding.
channel magnitudes are
is
vocoder
In
implemented)
our
used directly as
case,
input
to
depends
the modified
the
adapta-
tion stage, and no resynthesis is necessary.
_
I
_I_Cll__
1'
___1__1___
Extensions and Further Experiments
model
the
synchronization
of
of
that measures
various
spectral
neural
degree of
the
components
synchrony
(Young and Sachs,
to
spectral
critical
a
to
band
such
of
fibers
to
of vowel sounds result in a good
auditory representation of vowel
quency
discharges
within
be
would
A number of studies have shown
the CF of each fiber.
around
PAM
current
signal
acoustic
the
of
components
the
extension of
important
An
241
1979;
spectra and fundamental
fre-
Delgutte and Kiang, 1984a,b,c;
Seneff, 1985).
To model this synchrony, both magnitude and phase inforabout
mation
the output of each PAM filter bank channel must
be preserved and properly modified by the transduction stage.
This
is equivalent to converting the filter bank/transduction
stage system into a phase vocoder
1978,
Sec.
6.7.2).
In
a
phase
(c.f.
Rabiner
vocoder,
and
signals
speech
represented by both the magnitude and instantaneous
of
filter bank channel
each
are
frequency
signal can
The original
output.
Schafer,
be resynthesized by adding together sine waves that are amplitude
ters.
frequency modulated by the
and
Note that a channel vocoder is
with the
instantaneous
the
synthesize
resynthesis
the
proper
stage
parame-
simply a phase vocoder
frequency parameters set
frequency of each channel.
PAM,
channel
vocoder
to the center
In this proposed extension to the
of
haircell
the
vocoder
response.
could
be
Various
used
to
synchrony
measures could then be applied to the resulting fine-time sig-
Bs
.·
-
Extensions and Further Experiments
nal.
242
Alternatively, equivalent synchrony values could be cal-
culated directly from the vocoder parameters.
The phase vocoder organization of the transduction stage
has
another
easily
advantage:
controlled
it
forms
extension
in
the
modeling
characteristics of auditory haircells.
such a system is
basis
for
a
the
flexible,
transduction
A schematic diagram of
shown in Figure 4.3.
If we assume that the
phase vocoder parameters of speech signals vary slowly in comparison to the center frequencies of the filter bank channels,
we can treat the output of each channel, to a first approximation, and over a short period of time, as a tone burst.
have data,
derived
physiological
either
from a transduction model
or
If we
from
studies, of the transduction characteristics of
haircells in response to tones, we can use that data to modify
the
vocoder parameters
thesis we took the
the
DC
transfer
accordingly.
In
Section
2.3
of
this
first step in this approach by calculating
function
response to a tone,
of
a
simple
transduction
model
in
and using the resulting transfer function
to modify the channel vocoder parameters.
In a similar way, transfer functions specifying the magnitude and phase characteristics of the fundamental and higher
harmonic components of the response of a model,
or
an actual
haircell, to sine wave stimuli, can be used to modify both the
magnitude and phase parameters of a phase vocoder.
Weiss
II
and
their
___
_1_1
colleagues
_I
Holton and
(1983) have provided exactly this
*_I__IC___·_____l_____I
___
1___1_______1___
11
Extensions and Further Experiments
243
kind of information regarding the response of alligator lizard
haircells,
including transfer functions based both on experi-
in preparation).
Leong,
(Weiss and
transduction
as
stage
specification
of
a
relatively
transfer
easy
characteristics
function
signals, because
the transfer
with
the
fits
well
transfer
using
With such a system it would
investigate
to
the
nonlinearities
transduction
cochlea
lizard
The organization of
vocoder
phase
functions of harmonic components.
be
alligator
the
of
on a model
mental data, and
effect
the
response
on
specific
of
the
speech
to
functions are used directly to
modify the vocoder parameters.
Extending
vocoder
would
the
first
provide
a
two
of
stages
the
PAM
for
facility
convenient
extensions to the underlying transduction model.
a
to
phase
evaluating
As discussed
in Section 2.3, the current model consists of a simple memoryless
saturating
this model in response to
number
curves.
of
The
nonlinearity.
characteristics
However, many
DC
transfer
functions
of
sine waves with a DC bias exhibit a
in
common
questions
with
remain
neural
rate-level
unanswered,
and
some
differences do exist between the response of this transduction
model and the response of auditory haircells.
One question that deserves more
tionship between the
characteristics
I I
is
the
rela-
underlying transduction function and the
of the
preliminary analysis
attention
DC
shows
transfer
that
_
the
function.
dynamic
______
For
range
instance,
of
the DC
____l_··m___q_______I__ _^IIIII___II__*CCII_*_1-·--
244
Extensions and Further Experiments
function
transfer
expressed
be
can
analytically,
is
and
independent of the exact transduction function under a variety
But this preliminary analysis also
of reasonable assumptions.
indicates
that
it might be possible to specify transduction
functions with an
range.
increased dynamic
This possibility
is of interest because studies by Schalk and Sachs' (1980) and
and Palmer
Evans
cells
hair
low
with
can have dynamic ranges that are at
firing rates
spontaneous
that
indicate
(1980)
least 10 to 20 dB larger than our
simple model, which accu-
rately models the dynamic range of most high spontaneous rate
fibers.
A second difference between the current tranduction model
and physiological
amplitude
stimuli.
potential
waveforms
Holton
in
response to high
shape of the
the
is
data
and
Weiss
response
to
(1983)
show
receptor
high-amplitude,
low-
frequency tone bursts that fail to square off even 20 dB above
the
amplitude
input
which
at
the
size
the
of
response
Under similar conditions of high input amplitude,
saturates.
our transduction model generates a square wave response.
This
difference in behavior is irrelevant in the current model, but
be
would
an
important
model neural synchrony.
problem
if
the
PAM were
extended
to
The question, then, is which elements
of the transduction model must be changed to prevent this?
A
third
problem with
the
transduction
model
is
that
fibers with a range of spontaneous rates apparently terminate
11
___11_·11111--------
111
---.-------·
--- I---------
245
Extensions and Further Experiments
same haircell
on the
1982a,b).
(Liberman,
Although
is
there
nothing explicit in the transduction model that conflicts with
this, there is a general
a
function
of
implication that spontaneous rate is
characteristic
some
of the
haircell,
not
of
Suggesting a plausible relationship between
auditory neurons.
the DC bias in the transduction model and some aspect of the
complex
haircell-neuron
enhance
would
the
validity
the
of
transduction model.
A
PAM
the
final
extension to the
would
be
to
current transduction
existence
the
model
stage
of
neurons with
of
a
The current model generates a
variety of relative thresholds.
variety of thresholds, but only fibers with a single threshold
are actually used.
chosen
thresholds
of the PAM to 80
Modeling as few as four different properly
could
dB or more.
effective dynamic
the
increase
Since relative
range
threshold
and
spontaneous rate are inversely related, modeling fibers with a
variety of thresholds would
biting both high and
that
spontaneous
result in response patterns exhi-
low spontaneous
firing--or
the
rates.
lack
Since it appears
thereof--may
play
an
important role in the decoding of speech signals, modeling the
full variety of spontaneous rates observable in the peripheral
auditory system might be an important step.
^
Ilt
I-·-- -"D--------
Extensions and Further Experiments
4.1.3
246
Extending the Adaptation Stage
The
stage
adaptation
in the
remain.
stage
is
the
most
carefully
developed
current PAM, but a number of important questions
Some of these questions relate to the nature of rapid
adaptation.
Is
it a linear phenomenon, or is it, as Delgutte
suggests (1980), more prominent at higher stimulus levels?
Is
it
it
qualitatively the
same as short-term adaptation, or
related to the refractory period of neurons?
relate
to
the
nature
of
long-term
is
Other questions
adaptation.
Are there
a
series of long time constants, or is a single-constant process
of
some sort a better model?
tatively the
questions
same as
could
transformation
long-term
result
that
Is short-term adaptation quali-
in
appears
a
to
adaptation?
better
be
Answering
model
very
of
an
important
these
auditory
to
speech
perception.
4.2
FURTHER AUDITORY PHONETIC EXPERIMENTS
In
Chapter
relationships
Three
using
we
the
reexamined
PAM.
some
The results
peripheral auditory system may play a role
speech,
other
and
acoustic
spective.
dates,
suggest
and
The
indicate that
---
--D---·---·-
the
in the decoding of
following
the
relationships
from
sections
present
sort
auditory
of
an
-I ·-----
auditory
per-
some likely candiproperties
might prove to be related to phonetic content.
lli--·-i
_
phonetic
that it might be worthwhile to reexamine
phonetic
indicate
acoustic
which
Extensions and Further Experiments
4.2.1
247
Further Temporal Effects
The
"rabid-rapid"
experiment
described
in Chapter
Three
is only one example of a rich collection of phonetic contrasts
that can be produced by varying the duration of a silence gap
within an utterance.
died include
1980;
"slit-split"
"say-stay"
(Dorman
Other minimal pairs
et al.,
that have been stu-
(Dorman et al.,
1979;
(Repp, 1981;
Best et al.,
1979;
1981);
Repp,
and
Fitch et al.,
1981);
"shop-chop"
"goat-coat"
(Repp,
1981).
4.2.1.1
The "Slit-Split" Contrast
In the case of
"slit-split",
introducing
a variable
IEs
following glide.
and
ment
the
Dorman
et al.
the contrast is produced by
amount of silence between the initial
report
In one version of
that
their
subjects
this
experi-
heard
"slit"
when the duration of the silence was less than 60 msecs.
durations
between
60
and
reported hearing "split".
250
msecs,
a majority of
For
subjects
As the silence was lengthened still
further, the stimulus was more and more likely to be heard as
Es] followed by "lit".
Figure
thesized
in
4.4
shows
the
same way
the
PAM
that
response
to
a
syllable
stimuli were prepared
syn-
for the
Dorman experiment: a male speaker produced the fricative noise
Cs]
IslPII*llll*ICl`li·
and
the
syllable
"lit".
Digital
recordings
of
these
_
Extensions and Further Experiments
248
tokens were spliced together with 200 msecs of silence between
the pieces were
In one,
sounded
token
"slit",
like
the
Consequently these
to
look
auditory
for
20,
stimuli
were
that
in
only
the
and
smoothed
in
reported
the
16
same way that
Three,
Chapter
in
channels
frequency
channels
PAM
of
of a
three
the
of
waveform
response
high
presence
cue the
that
the
experiments
the
frequency channels of the
original
the
postprocessed
through
except
properties
and
tokens,
synthesized
initial
of the
energy
channels might be appropriate places
shows
4.5
Figure
stop.
like
token
msec
200
s] followed by "lit".
of the
most
shows,
sibilant occurs in the five highest
PAM.
in
Informal listening revealed that the
"split", and the 500 msec token like
As Figure 4.4
silence,
of
0 msecs
separated by
the other by 500 msecs.
0 msec
from the same pieces.
other stimuli were produced
Two
them.
included.
are
The left panel of Figure 4.5 shows the waveforms of the three
stimuli.
The middle panel shows the composite response of the
high frequency PAM channels.
The right panel shows the compo-
site masking level of the same channels.
At
the bottom of the middle and
responses
listener
judgements
data has
as
been
a
from
Dorman
et al.:
function of silence
scaled
and
right
positioned
percent
duration.
so that
are
panel
of
The
shown
"split"
response
for any silence
duration of interest, perceptual data and PAM response values
are
I
vertically
aligned.
______I__C*_l_____
Throughout
the
following discussion
Extensions and Further Experiments
it
should be
remembered
that
249
the Dorman
perceptual
response
data are not based on the stimuli shown.
An
investigation of
the
relationship between
perceptual
responses and PAM responses to these stimuli might begin with
the following observations:
1.
For silence durations less than 60 msecs,
jects
reported
no
stop
perceptions.
Dorman's
sub-
this
same
During
period, the onset response to the glide is substantially
smaller than it is for a longer silence duration, because
of adaptation to the preceding
sibilant.
The
degree of
masking during this period is correspondingly high.
2.
For silence durations between 60 and
250 msecs,
the per-
centage of stop judgements monotonically increases.
Dur-
ing approximately the same period, the degree of masking
in the high frequency channels falls to zero.
3.
For silence durations greater than 250 msecs, the percentage of stop judgements decreases steadily, reaching zero
at 650 msecs.
taneous
firing
During this
rate of
same period of time the spon-
the high
frequency PAM channels
are approaching their quiescent value.
and Further Experiments
Extensions
250
The "Shop-Chop" Contrast
4.2.1.2
The contrast between an initial fricative or affricate in
the words "shop" and "chop" can be affected by the duration of
silence
the
preceding
word
it
is
spoken
" (Dorman et al.,
"Say
phrase such as
when
a
in
carrier
1979; Repp, 1981).
Silences less than 40 msecs tend to cue the fricative variant.
the
cue
silences
Longer
A
affricate.
cue
second
this
to
(Cutting
manner distinction is the rise time of the frication
A faster rise time
and Rosner, 1974; Howell and Rosen, 1983).
tends to signal the affricate manner, while a longer rise time
a
signals
time
as
Cutting and Rosner show a 40 msec rise
fricative.
the
boundary
phonetic
Howell
and
dent.
Finally,
found
Rosen
that lengthening
Repp
that
and
boundary was
the
his
these
between
frication duration
context depen1981)
(1978,
colleages
manners;
two
report
results in more fricative
responses.
of
All
correlates
acoustic
these
in com-
affricate/fricative manner distinction have one thing
mon:
adaptation,
through
response
auditory
peripheral
affect
they
to
shows how silence duration, rise
interact
auditory response.
these
cues
possible
the
affect
to
is
that
_____·l__lr__ll_____^__-_l____l-il-l_
timing
time,
and
Figure
stimulus
magnitude
of
the
of
amplitude
stimulus.
the
and
the
the
to
2.27
duration
peripheral
The acoustic account of the interaction of
confused
the
_1--1-_11-
and highly
relationship
context
between
dependent.
the
It
is
corresponding
Extensions and Further Experiments
251
auditory properties and the perceptual response will prove to
be more straightforward.
Speaking Rate, Auditory Response, and
4.2.1.3
ception
Per-
Phonetic
The experiments reported in Chapter Three concentrate on
the
and
auditory
short
of
effects
perceptual
term
temporal
changes: variations in the duration and rise times of the target
phoneme
or
itself,
its
immediately
surrounding
acoustic
context.
A number of experiments have shown that longer range
temporal
factors
can
also
phonetic
affect
and Miller, Aibel, and
instance, Miller and Grosjean (1981),
report
(1983)
Green
that
For
judgements.
the perception of "ba-wa" syllables
similar to the ones discussed in Section 3.2 can be affected
by
temporal
changes
duration of more
in
changes
remote
duration
In general,
phrase.
to
seems
the
to
syllables,
adjacent
syllables
in
carrier
inserted
pauses
of
a
in
the
phrase,
and
changes
in
the
the more remote the change, the
affect phonetic
perception.
carrier
less
it
(1977,
Similarly, Port
1979) showed that the speaking rate of a carrier phrase could
voiced/unvoiced
affect
the
rapid"
stimuli.
was
Once
relatively small
stop
again,
the
closure
size of
---
·-·i
--
in
"rabid-
the boundary
shift
(7 to 10 msecs) compared with the effect
of changing preceding vowel duration.
"
boundary
-·
---
-~1-·---
~-----·--C-
Extensions and Further Experiments
252
It would be interesting to determine the extent
these
results
are
predicted
influences these and
medium
of
by
the
PAM.
similar phonetic
long-term
adaptation,
it
If
speaking
judgements
might
to which
rate
through
be
possible
the
to
explain many rate-related effects solely through the response
patterns of
the peripheral
auditory system, without recourse
to higher-order auditory processes.
4.2.2 Further Investigations of Auditory Correlates to Voicing in Stops
The acoustic properties
tion of
voicing
which cue
the phonetic
distinc-
in stop consonants have probably received as
much attention as any other area of acoustic phonetics.
Klatt
(1975)
voice
onset
identifies
time,
release,
six
amount
burst
of
major
cues
for
low-frequency
amplitude,
fundamental
initial
energy
stops:
following
frequency,
the
prevoicing
during the closure, and the duration of the preceding segment.
Edwards
(1981) identifies ten acoustic properties that can cue
intervocalic voicing mode.
From an acoustic phonetic point of
view, a
stop
unified
theory of
consonant voicing
is hard to
imagine.
A peripheral auditory account of voicing perception might
be considerably simpler than the acoustic account.
pheral
response
to
stimuli
which
cue
voicing
The peri-
distinctions
through a variety of acoustic properties could be modeled via
1 1~~~ ______
__~~~~~~s_11~~~~
_~~~~_
I~~-----· Pi.~~~~11
_~~~~~~~I_____IBUI__WLI-·I__-~~~~~~~~~~~~~~~~~
Extensions and Further Experiments
253
the PAM, and the PAM response patterns examined with an eye to
finding simple properties that are related to voicing perception.
A
energy
ratio
is
of
low
a possible
frequency
energy
characteristic
to
of
non-low
an
auditory phonetic
account of stop consonant voicing perception.
quency
first
energy
More
low fre-
(such as prevoicing, and an early onset of the
formant following voicing onset) seems to cue
stop.
frequency
More high
frequency energy
higher fundamental
frequency),
a voiced
(such as a strong burst, or
or
less
low
frequency energy,
tends to cue a voiceless stop.
In using any metric based on auditory measures of spectral energy, an important question arises regarding the treatment of masked fibers--fibers that are not
fact
that
they have non-zero spontaneous
firing despite the
rates.
It would be
interesting to investigate the consequences of treating these
fibers
as
firing at
if they were strongly stimulated.
its
spontaneous
energy is present within
rate
its
is
indicating
frequency band.
A
fiber that is
that
not
A fiber that is
firing near its maximum rate is indicating the presence
large
amount
of
energy.
It
is
unclear
much
how much
of a
energy
is
present within the frequency band of a fiber that is not firing
at
rate.
all
the
fact
that
There could be very little
considerable
__
despite
amount:
the
fiber
__ ·_________________ILI__Q_
it has
a high
spontaneous
energy, or there could be a
response
is
ambiguous.
In
Extensions and Further Experiments
254
fact, the only certain conclusion is that there was a significant amount of energy present in the last 100 msecs or so.
One could, of course, interpret masking as the absence of
energy,
but an equally plausible alternative is to interpret
masking as being equivalent to the presence of energy.
Under
such a scheme, the output of the PAM masking detector postproused
cessor
could
in
contribute
energy.
It
is
3.3 would
Section
to
the
possible
ratio
that
be
of
as
treated
energy,
low and non-low
a
such
and
frequency
generalization
of
the
notion of auditory spectral balance could greatly simplify an
auditory explanation of stop consonant voicing cues.
Auditory Correlates to Place of Articulation in Stops
4.2.3
The final experiment that we shall
peripheral
auditory
stop consonants.
to
specify
the
correlates
propose is a study of
place
to
of
articulation
in
A considerable amount of work has been done
of
nature
place of articulation
invariant
acoustic
correlates
in word-initial stop consonants.
of
Blum-
stein and Stevens (1979; Stevens and Blumstein, 1981) proposed
that the spectral shape of the
ably identify the place
Kewley-Port
(1983a,b)
stop release burst could reliAs an alternative,
of articulation.
has
proposed
substituting
a
set
of
dynamic properties for Blumstein and Stevens's static spectral
shapes.
Kewley-Port's proposed properties remain invariant in
the sense that they are independent of the vowel which follows
I
__
___
1___1_1____
I
11_
_111__
)
Extensions and Further Experiments
255
the stop, but are dynamic in that they reflect the changes in
the
spectrum
during
the
first
release.
In the domain of
son,
Rayment
and
(1979)
40
msecs
following
auditory modeling,
showed
that
stop
the
Searle,
stop
Jacob-
consonants
in
syllables could be correctly identified with reasonable
CV
accu-
racy using vowel-independent features extracted from a filter
bank whose design was based on auditory characteristics.
Most recently, Lahiri,
Gewirth, and Blumstein (1984) have
proposed an acoustically invariant metric for place of articulation
in
diffuse
that measures
stops
(labial
the difference
versus
alveolar
between the
and
dental)
spectral balance of
the release burst and the spectral balance following the onset
of voicing.
Simply stated,
Lahiri et al.
suggest that dental
and alveolar stops are characterized by a spectral shape that
contains
a substantially
energy than the
frequency
vowel,
or
Labial
following
energy
the
larger
declines
amount
consonants,
constant proportion
on
of
the
proportion
vowel:
either
during
low
the
the
of low and high
are
frequency
amount
transition
frequency
other hand,
of high
energy
of high
into
the
increases.
characterized by a
frequency energy
in the
burst and the vowel, or an increase in the proportion of high
frequency energy during the transition into the vowel.
unlike
the
earlier
Blumstein
Thus,
and Stevens spectral shape pro-
perty, it is not the absolute spectral shape of the burst that
matters, but its
_
_I
PI
shape relative
I
to the shape of the following
I(
_IlnL
Extensions and Further Experiments
256
vowel--still regardless of the identity of that vowel.
The top panels of Figure 4.6 exemplify the type of metric
that Lahiri
the
PAM
speech:
et al. propose.
filter
/ba/
bank
and
in
/da/
These
response
as
panels
to
uttered
two
by
a
show the output of
tokens
male
of
natural
speaker.
(The
filter bank output has been smoothed with a 10 msec left half
hamming window.)
Two spectra are
shown in each panel.
taken from the peak of the release burst.
the
second
glottal
Lines ab in
each
pulse
figure
touch the
peaks of the burst spectra.
second and
vertical
fourth
lines
formant
are
following
the
second
channels
peaks
positioned
closest
The other is
onset
and
of
from
voicing.
fourth
formant
Lines cd in each figure touch the
of
the
vowel
at the center
ninth and sixteenth filter bank channel
the
One is
spectra.
The
frequency of the
locations.
These are
to the arbitrary frequencies
of 1500 Hz
and 3500 Hz which Lahiri et al. choose to use in measuring the
slopes.
Thus the slopes of ab and cd are determined by the height
and
location of
the
second and fourth
by the relative lengths of ca and db.
formants,
and measured
Lahiri et al. chose the
ratio
r = (d-b)/(c-a)
to
measure
burst
111_
and
the
the
change
vowel.
in
They
the
spectral
found
balance
between
that values of r >
1:
___ ____
__
the
.5, or
I
257
Extensions and Further Experiments
from a
negative values resulting
a
cated
labial
For
consonant.
our
indi-
denominator,
negative
stimulus,
/ba/
r equals
1.8.
Positive values of r < 0.5, or negative values resulting
from
a
numerator,
negative
our
For
consonant.
or
alveolar
0.55:
equals
r
stimulus,
/da/
an
indicated
dental
slightly
above the criterion value.
from
metrics
their
construct
et al.
Lahiri
LPC
spectra
produced from speech windowed by 10 msec Hamming windows.
The
Hz.
As
frequency
dimension
of
is
spectra
their
linear
in
described above, our spectra are generated by the PAM spectral
analysis
stage,
and
respects,
the
spectra
is
frequency
top
the
in
in
linear
panels
In
Bark.
Figure
of
other
4.3
are
similar to those of Lahiri and her colleagues.
The question now arises, what would be the
specified
ratio
slope
rest of the
exactly that:
these
were
they are spectra generated
from
new
spectra.
0.55 to -0.9:
For
the
to 2.3:
1.8
/da/
r
was
stimulus,
recalculated
r decreases
for
from
a value
For the /ba/ stimulus, r increases
that
is
farther
point in the labial direction.
a
Figure
a value that is farther from the criterion point
in the alveolar direction.
from
of
in the same way that the top
metric
The
smoothed.
through the
pro-
the output of the PAM, smoothed
spectra
stimuli
the
4.3 were
The bottom panels
PAM?
duced by doing
of putting
r
effect on the
-----
....
..--i-I
from the criterion
258
Extensions and Further Experiments
These examples suggest that it would be worthwhile to do
an experiment to discover whether the peripheral auditory system emphasizes
following
vowels,
spectra.
If so,
is
a
in spectral balance between bursts
shifts
relative
and
system may
the
shifts
seen
if the metric proposed by
reliable metric,
auditory
to
the
serve
transformations
to make
labial
of
and
in
and
acoustic
Lahiri
et al.
the peripheral
alveolar
stops
more distinct.
_·I
_II_
_l____1_n___l_________ll
259
FIGURE CAPTIONS FOR CHAPTER 4
FIGURE 4.1
Slope of phase response of auditory fibers.
The data points
show the slope of straight lines fitted by hand to phase
response data published by Pfeiffer and Kim (1975).
The
straight
line
is a linear
fit
to the
six data points
representing fibers with characteristic frequencies between 3
and 17 Bark. Note the data point at .5 Bark, with a slope of
zero.
FIGURE 4.2
Schematic diagram of one channel of a channel vocoder system.
hi(n) is the impulse response of the i'th channel. After the
analysis stage, the channel signal could be modified in some
way
(for example by the PAM transduction and adaptation
stages) and then, if desired, resynthesized.
FIGURE 4.3
Schematic diagram of a single channel of the proposed PAM
phase vocoder-based transduction stage.
The magnitude of the
channel band pass filter output is used as the input to the
signal modification stage.
In this stage, magnitude and phase
transfer functions for the harmonic response of an underlying
transduction model are indexed by the channel magnitude value.
The harmonic magnitude and phase information is used to reconstruct the modified signal using a harmonic synthesizer.
FIGURE 4.4
PAM response to stimulus perceived as "split".
The stimulus
The nominal
was created by concatenating Es] and "lit".
center frequencies of each channel in Hertz and Bark are
listed along the sides.
The vertical dimension is expected
firing rate. The vertical distance between the zero lines of
adjacent channels corresponds to an expected firing rate 3.0
ii(·ssllllii---··---r·IIIP··IIL··
I
260
Figure Captions for Chapter 4
The stimulus
times the maximum steady state firing rate.
waveform is shown at the bottom, delayed by 9.9 msecs, the
group delay of the PAM filters, so that glottal pulses in the
waveform and in the PAM output channels are aligned vertically.
FIGURE 4.5
Three representations of "slit/split/s-lit" stimuli. The signals shown were created by concatanating s] and "lit" with 0,
200, and 500 msecs of silence separating the sibilant from the
The signals on the left are the acoustic waveforms.
glide.
the signals in the middle are the smoothed composite PAM
response patterns of the five highest frequency channels,
The signals on
smoothed using a 20 msec half hamming window.
the right are the composite masking detector outputs of the
five highest frequency channels, smoothed using a 5 msec half
The two panels at the bottom of the the pichamming window.
ture in the middle and on the right summarize data reported by
Dorman et al. (1979) regarding subjects' responses to stimuli
The panels show the percent of
similar to those shown here.
"split" responses as a function of gap duration.
FIGURE 4.6
representations of the spectral shape of the burst and
second glottal pulse following voice onset for /ba/ and /da/.
The top panels show the magnitude of the output of the PAM
The bottom panels show the output of
spectral analysis stage.
the PAM adaptation stage. Lines ab touch the F2 and F4 peaks
Lines cd touch the F2 and F4 peaks of
of the burst spectrum.
the second glottal pulse following the onset of voicing.
Two
___I_I
I-
_
I___I_________
l____
FIGURE 4.1
261
SLOPE OF PHASE OF FIBER FREQUENCY RESPONSE
-
I
I
i
a
.6
m
O
09
C
.4
.
el
x
.2
I
! -----10
!
15
Fiber CF in Bark
IV I- -, 11 .
~~~~~~~~~~~~~~~~~~~~~~~~~~~~",
-1-- ...
I~~~
FIGURE 4.2
262
(a
__
0
o.-
a
CL
c
ML
en
fn
a)
C
U
X
C
E
CD)
'a
0
U.
C
a,
0,
L
U
I*
O
to
0
Q;
--j
_..
u0
:3
a
cm
<[
0
>-
E
O
C
C
0
a
-E
-0.
1
--
.9c
I
_
-
=n
i
I*_
0-1
C
I
i
0
I
i
zI
/T'*.tn
4-,
7j
a
a
L
&-
C
O01
-a
F
C.
>D
.,-(~1
_I___
-·----_-------I-___._
_·I-----·
Is--
11_1_1_----1
-
(U)
*n
'*
E
-C
Lr
1p
263
FIGURE 4.3
5
on
n0
-I
En
0
-
U
Li
O
wX
0d
z
w~~~~~~~~~~~-
CC
03
En
z
o
a
gi
-
. %'[3
w
U
0Li
0
O
i
C~~C03
g
on~~
wa
.C:
PM
SCK
cc:
.
~3
~~<
4
DX ~
Z
~
~~~~~~CO
C:
0.3
a3
E:~~~~~~~~~PtLl
n
I II
· ^-- I*-a(llll-*-
IJ(--···l
FIGURE 4.4
264
HERTZ
NS2WAK
BARK
..
~ i ~..~.l
..
,,
_
,,,
102
2
203
3
384
4
406
5
507
6
669
7
_---
8
761
_,4uw
..
913
9
1065
10
1218
11
1421
12
1674
13
1979
14
2384
15
2790
16
3298
17
3906
18
4617
19
5428
6392
2Q
S
4hm
-
-
e
---
-
CII------·
-·-i-.
·r
I
I--___-_X.(I--·_l1_11-111-__
^1_1_
FIGURE 4.5
265
[
-!
w
0
0
0
Vo
o
O
-i
(0
z
co
0
l-
0
a
o
c
w
C,
Z
0
a.
C,
At
-fW
r
r
O
0
O
~~~~C
co
0
a
U)
o
c
*_5
_
V-
5
1Q
K
E
X
V-
u-
I
a.
Id
w
0
!
Vo
I-
o
0
I-
0
Q..
o
sasuodsaj
0
0
!lds %
I`
U.
0
-1
4P"
k-
4
0
..
4
z
0
0
c
0
D
a
(a
C-1
C
o
0
0
CY
0
0
U)
o
Or
E
1:1
FIGURE 4.6
O
266
0
0
co
CU
co
0
la
Cr
O
U
u-
O
cn
O-
T-
0o
CO
0
o.2
o
CO
,)
aD
()
C
p
a
L-
y
n
4-'
:~
a
IO
.
Q.
cn
C
0
ao
0
0
0
a
CO
04
m
o
C
co
C
C
0am
.0
UO
Q
L
U_
T-
0
C,
0
Sp u apnllldwV~
___111___·_1_11____1__--^--·--·1_·11
0
CO
1
asuodsaeU paezlewJoN
Ll
-----
--_____11·__^_1__11__11__
267
BIBLIOGRAPHY
"AP responses in forward(1981),
relationship to responses of
masking paradigms and their
auditory-nerve fibers," JASA 69(2), 492-499.
Abbas,
P.
and
Gorga
M.
Allen, J. B. (1983), "A Hair Cell Model of Neural Response,"
in Mechanics of Hearing, E. de Boer and M. Viergever, eds.,
Delft University Press (Martinus Nijhoff Publishers).
Allen, J. and M. Sondhi (1979), "Cochlear macromechanics: Time
domain solutions," JASA 66(1), 123-132.
Allen, J. (1983), "Magnitude and phase--frequency response to
single tones in the auditory nerve," JASA 73(6), 2071-2192.
Art, J. and R. Fettiplace (1984), "Efferent Desensitization of
Auditory Nerve Fibre Responses in the Cochlea of the Turtle
Pseudemys Scripta Elegans," J. Physiol. 356, 57-523.
Beasley, D. and J. Maki (1976), "Time- and frequency-altered
speech," in Contemporary Issues in Experimental Phonetics, ed.
by N. Lass, Academic Press, New York, 419-458.
Best, C., B. Morraongiello, and R. Robson (1981), "Perceptual
equivalence of acoustic cues in speech and nonspeech perception," Perception & Psychophysics, 29(3), 191-211.
Blomberg, M., R. Carlson, K. Elenius, and B. Granstrom (1982),
"Experiments with Auditory Models in Speech Recognition," in
The Representation of Speech in the Peripheral Auditory System, R. Carlson and B. Granstrom, eds., Elsevier Biomedical
Press, 197-201.
Blomberg, M., R. Carlson, K. Elenius, and B. Granstrom (1984),
"Auditory Models in Isolated Word Recognition," Proceedings of
the IEEE Conference on Acoustics, Speech, and Signal Processing, San Diego, 17.9.1-4.
_
-·-·
r.
----------I--a"l-----
Bibliography
268
Blumstein, S. and K. Stevens (1979), "Acoustic invariance in
speech production: Evidence from measurements of the spectral
characteristics of stop consonants," JASA 66(4), 1001-1017.
Carlson,
R,
and B. Granstrom (1982),
"Towards an Auditory
Spectrogram," in The Representation of Speech in the Peripheral Auditory System, R. Carlson and B. Granstrom, eds.,
Elsevier Biomedical Press, 109-114.
Chistovich, L. A., M. P. Granstrem, V. A. Kozhevnikov, L. W.
Lesogor, V. S. Shupljakov, P. A. Taljasin, and W. A. Tjulkov
(1974), Acoustica 31, 349-353.
Chistovich, L. A., V. V. Lublinskaya, T. G. Malinnikova, E. A.
Ogorodnikova, E. I. Stoljarova, and S. Ja. Zhukov (1982),
Patterns
of
Auditory
of
Peripheral
"Temporal
Processing
Speech," in The Representation of Speech in the Peripheral
Auditory System, R. Carlson and B. Granstrom, eds., Elsevier
Biomedical Press, 165-180.
Cutting, J. and B. Rosner (1974), "Categories and boundaries
in speech and music," Perception & Psychophysics 16(3), 564570.
"Representation of speech-like sounds in
patterns of auditory-nerve fibers," JASA 68(3),
Delgutte, B. (1980),
the discharge
843-857.
Delgutte, B.
the Discharge
toral Thesis.
Speech-like Sounds in
Patterns of Auditory-nerve Fibers, M.I.T. Doc-
(1981),
Representation
of
Delgutte, B. (1982), "Some correlates of phonetic distinctions
at the level of the auditory nerve," The Representation of
Speech in the Peripheral Auditory System, ed. R. Carlson & B.
Granstrom, Elsevier Biomedical Press, 131-149.
Delgutte, B. and N. Kiang (1984a), "Speech coding in the auditory nerve: I. Vowel-like sounds," JASA 75(3), 866-878.
_1_·__1_______·11__1I_
-_
__------s-·-·-----I___11_1--..
.___.···C---·IY/Iilii·IICI(··-·
-__IIIII(P-·l---·------·I-···----
II
Bibliography
269
Delgutte, B. and N. Kiang (1984b), "Speech coding in the auditory nerve: II. Processing schemes for vowel-like sounds,"
JASA 75(3), 879-886.
Delgutte, B. and N. Kiang (1984c), "Speech coding in the auditory nerve: V. Vowels in background noise," JASA 75(3), 908918.
Dorman, M., L. Raphael, and A. Liberman
ments on the sound of
65(6), 1518-1532.
silence
(1979),
in phonetic
"Some experi-
perception,"
JASA
Durlach, N. (1968), A Decision Model For Psychophysics, Unpublished notes, Department of Electrical Engineering, MIT.
Edwards, T. (1981), "Multiple features analysis
calic English plosives," JASA 69(2), 535-547.
of
intervo-
Evans, E. and A. Palmer (1980), "Relationship Between the
Dynamic Range of Cochlear Nerve Fibres and Their Spontaneous
Activity," Exp Brain Res 40, 115-118.
Fant, G. (1960), Acoustic Theory of Speech Production,
& Co., 's-Gravenhage, The Netherlands.
Mouton
Fitch, H., T. Halwes, D. Erickson, and A. Liberman (1980),
for stoptwo
acoustic
cues
equivalence
of
"Perceptual
consonant manner," Perception & Psychophysics, 27(4), 343-350.
Hall, J. (1977), "Two-tone suppression in a nonlinear model of
the basilar membrane," JASA 61(3), 802-810.
Harris, D. and P. Dallos
(1979),
"Forward Masking of Auditory
Nerve Fiber Responses," J of Neurophysiology 42(4), 1083-1107.
Holton, T. and T. Weiss (1983), "Receptor Potentials of Lizard
Cochlear Hair Cells with Free-Standing Stereocilia in Response
to Tones," J. Physiol. 345, 205-240.
*9(
srril
---~-----
Bibliography
Howell, P. and S. Rosen (1983), "Production and perception of
rise time in the voiceless affricate/fricative distinction,"
JASA 73(3), 976-984.
Kewley-Port, D.
of place
322-335.
of
(1983a),
"Time-varying features as correlates
articulation in stop consonants," JASA 73(1),
Kewley-Port, D. (1983b), "Perception of static and dynamic
acoustic cues to place of articulation in initial stop consonants," JASA 73(5), 1779-1793.
E. Thomas, and L. Clark (1965),
Kiang, N., T. Watanabe,
Discharge Patterns of Single Fibers in the Cat's Auditory
Nerve, Research Monograph no. 35, The M.I.T. Press, Cambridge.
Kiang, N. (1980), "Processing of speech by
vous system," JASA 68(3), 830-835.
Kim, D., C. Molnar, and J. Matthews
(1980),
the auditory
ner-
"Cochlear mechan-
ics: Nonlinear behavior in two-tone responses as reflected in
cochlear-nerve-fiber responses and in ear-canal sound pressure," JASA 67(5), 1704-1721.
Khanna, S. and D. Leonard, (1982), "Basilar Membrane Tuning in
the Cat Cochlea," Science 215, 305,306.
Klatt, D. (1975), "Voice Onset Time, Frication, and Aspiration
in Word-Initial Consonant Clusters," J of Speech and Hearing
Research 18(4), 686-706.
Klatt,
D.
(1979),
"Speech
perception:
phonetic analysis and lexical access,"
312.
Klatt, D. (1980), "Software for a
synthesizer," JASA 67(3), 971-995.
__I__LVr_B__·_l__l__s__
a model
of
acoustic-
J of Phonetics 7, 279-
cascade/parallel
-I----1__---------
formant
^__1_
_
__
271
Bibliography
Klatt, D. (1982), "Speech processing strategies based on auditory models," The Representation of Speech in the Peripheral
Auditory System, ed. R. Carlson & B. Granstrom, Elsevier
Biomedical Press, 181-196.
Koenig, W., H. Dunn, and L. Lacy
graph," JASA 17(1), 19-24.
(1946),
"The Sound Spectro-
L. Gewirth, and S. Blumstein (1984), "A reconsideration of acoustic invariance for place of articulation in
from
a cross-language
Evidence
consonants:
stop
diffuse
study," JASA 76(2), 405-410.
Lahiri,
A.,
Landahl,
K. and E. Maxwell (1983), "The /b/-/w/ contrast and
the
from
Papers
duration,"
syllable
of
considerations
and
Nineteeth Regional Meeting, ed. by Chukerman, Marks,
234-243.
Richardson, Chicago Linguistic Society,
Liberman, M. C. (1978), "Auditory-nerve response
raised in a low-noise chamber," JASA 63(2), 442.
Lindsey, P. and D. Norman
ing, Academic Press.
Lisker,
L.
(1957),
(1972),
"Closure
Voiced-Voiceless Distinction
49.
Human
from
cats
Information Process-
and the Intervocalic
in English," Language 33(1), 42Duration
Lisker, L. (1978), "Rapid vs. Rabid: A Catalogue of Acoustic
Features That May Cue the Distinction," Haskins Laboratories:
Status Report on Speech Research SR-54, 127-132.
Ludlow, C., E. Cudahy, and C. Bassich, (1982), "Developmental,
age, and sex effects on gap detection and temporal order,"
JASA 71 Suppl. 1, S47.
"A Computational Model of Filtering, Detection, and Compression in the Cochlea," Proceedings of the IEEE
Conference on Acoustics, Speech, and Signal Processing, Paris,
1282-1285.
Lyon,
Lar
R. (1982),
IC1- --
Bibliography
272
Lyon, R. (1983), "Computational Models of Neural Auditory Processing," Proceedings of the IEEE Conference on Acoustics,
Speech, and Signal Processing, San Diego, 36.1.1-4.
Makhoul, J. and J. Wolfe (1972), Linear Prediction and the
Spectral Analysis of Speech, BBN Report No. 2304, Bolt Beranek
and Newman, Cambridge, MA.
Mehrgardt, S. and V.
Mellert
(1977),
"Transformation
charac-
teristics of the external human ear," JASA 61(6), 1567-1576.
Miller, J. and A. Liberman (1979), "Some effects of lateroccurring information on the perception of stop consonant and
semivowel," Perception and Psychophysics 25, 457-465.
Miller, J. and F. Grosjean (1981), "How the Components of
Speaking Rate Influence Perception of Phonetic Segments,"
Journal of Experimental Psychology: Human Perception and Performance 7(1), 208-215.
Miller, J., I. Aibel, and K. Green (1983), "On the Nature of
the Listener's Adjustment for Speaking Rate during Phonetic
Perception," JASA, Suppl. 1 73, S67.
Moore, B. (1982), An Introduction to the Psychology
ing, 2nd edition, Academic Press.
of Hear-
Ohde, R. and K. Stevens (1982), "Effect of burst amplitude on
the perception of stop consonant place of articulation," JASA
74(3), 706-714.
Patterson,
R.
(1976),
"Auditory
noise stimuli," JASA 59(3),
filter
shapes
derived
with
640-654.
Pickles, J. (1982), An Introduction to the Physiology of Hearing, Academic Press.
Port, R. (1977), The Influence of Speaking Rate on the Duration of Stressed Vowel and Medial Stop in English Trochee
Words, U. of Conn doctoral disseration, published by IU
Linguistics Club, Indiana University.
_1__1
1_1
1_·l_____r____lII_____·
___ _I
I
Bibliography
273
Port, R. (1979), "The influence of tempo on stop closure duration as a cue for voicing and place," Journal of Phonetics 7,
45-56.
Price, P. and H. Simon (1984), "Perception of temporal differences in speech by 'normal-hearing' adults: Effects of age and
intensity," JASA 76(2), 405-410.
nerve
fiber
"Coclear
Kim
(1975),
and D.
R.
Pfeiffer,
responses: Distribution along the coclear partition," JASA
58(4), 867-869.
Rabiner, L. and R. Schafer (1978),
Speech Signals, Prentice Hall.
Processing
Digital
"Measurement of
Rabinowitz, W. (1981),
immittance of the human ear," JASA 70(4),
the acoustic
1025-1435.
of
input
Repp, B., A. Liberman, T. Eccardt, and D. Pesetsky (1978),
"Perceptual Integration of Acoustic Cues for Stop, Fricative,
and Affricate Manner," Journal of Experimental Psychology:
Human Perception and Performance 4(4), 621-637.
"Phonetic and Auditory Trading Relations
Preliminary
Perception:
in
Speech
Cues
Acoustic
between
Report
on
Speech
Haskins
Laboratories:
Status
Results,"
Research SR-67/68, 165-189.
Repp,
B.
(1981),
Rhode, W. (1971), "Observations of the Vibration of the Basilar Membrane in Squirrel Monkeys using the Mossbauer Technique," JASA 49(4), 1218-1231.
"Evidence
(1974),
Rhode, W. and L. Robles
for nonlinear vibration in the
experiments
55(3), 588-596.
Sachs,
M.
and
E.
Young
(1979),
"Encoding
from Mossbauer
cochlea," JASA
of
vowels in the auditory nerve: Representation
discharge rate," JASA 66(2), 470-479.
steady-state
in
terms
of
-----
I,-
Bibliography
274
Schalk, T. and M. Sachs (1980), "Nonlinearities in auditorynerve fiber responses to bandlimited noise," JASA 67(3), 903913.
Schroeder, M., B. Atal, and J. Hall (1979), "Objective Measure
of Certain Speech Signal Degradations Based on Masking Properties of Human Auditory Perception," in Frontiers of Speech
Communication Research, edited by B. Lindblom and S. Ohman,
Academic Press, 217-229.
Searle,
C.,
J. Jacobson,
sonant discrimination
799-809.
and
based
S.
Rayment
on human
(1979),
audition,"
"Stop
con-
JASA 65(3),
Seneff, S. (1983), "Pitch and Spectral Estimation of Speech
Based on Auditory Synchrony Model," Proceedings of the IEEE
Conference on Acoustics, Speech, and Signal Processing, San
Diego, 36.2.1-4.
Seneff, S. (1985), Pitch and Spectral Analysis of Speech based
on an Auditory Synchrony Model, MIT Doctoral Thesis.
Shaw, E. (1974),
"Transformation of sound pressure level
the free field to the eardrum
56(6), 1848-1861.
in the horizontal
plane,"
from
JASA
Smith, R. (1979), "Adaptation, saturation, and physiological
masking in single auditory-nerve fibers," JASA 65(1), 166-178.
Smith, R. and M. Brachman (1980), "Operating range and maximum
response of single auditory nerve fibers," Brain Research 184,
499-505.
Smith, R. and J. Zwislocki
(1975),
"Short-Term Adaptation and
Incremental Responses of Single Auditory-Nerve Fibers,"
Cybernetics 17, 169.
Biol.
Steele, C. and L. Taber (1979a), "Comparison of WKB caclulations and experimental results for a two-dimensional cochlear
model," JASA 65(4), 1001-1006.
1·1_1__1_---1':
._.------------
Bibliography
275
Steele, C. and L. Taber (1979b), "Comparison of WKB caclulations and experimental results for three-dimensional cochlear
models," JASA 65(4), 1007-1018.
Stevens, K. and A. House (1961), "An Acoustic Theory of Vowel
Production and Some of its Implications," JASA 4(4), 303-320.
Stevens, K. and D. Klatt
in the voiced-voiceless
653-659.
(1974), "Role of formant transitions
distinction for stops," JASA 55(3),
Stevens, K. (1980), "Acoustic correlates
categories," JASA 68(3), 836-842.
of
some
phonetic
Stevens, K. and S. Blumstein (1981), "The Search for Invariant
Acoustic Correlates of Phonetic Features," Perspectives on the
Study of Speech, ed. P. Eimas & J. Miller, Lawrence Erlbaum
Associates, 1-38.
Weiss, T. and R. Leong (in preparation), "A Model for Signal
Transmission in an Ear Having Hair Cells with Free-Standing
Stereocillia: IV. Mechanoelectric Transduction Stage".
Young, E. and M. Sachs (1973), "Recovery from sound
in auditory-nerve fibers," JASA 54(6), 1535-1543.
exposure
Young, E. and M. Sachs (1979), "Representation of steady-state
vowels in the temporal aspects of the discharge patterns of
populations of auditory-nerve fibers," JASA 66(5), 1381-1403.
Zhukov, S. Ya, M. G. Zhukova, and L. A. Chistovich (1974),
"Some new concepts in the auditory analysis of acoustic flow,"
Soy. Phys. Acoust. 20(3), 237-240.
Zue, V. (1976), Acoustic Characteristics of Stop Consonants: A
Controlled Study, MIT Doctoral Thesis.
Pierce
and J.
Lipes,
R.
Zweig, G.,
compromise," JASA 59(4), 975-982.
(1976),
"The
cochlear
276
Bibliography
Zwicker, E. (1961), "Subdivision of the Audible Frequency
Range into Critical Bands (Frequenzgruppen)," JASA 33(2), 248.
Zwicker, E. (1970), "Masking and Psychological Excitation as
Consequences
of the Ear's
Frequency Analysis,"
Frequency
Analysis and Periodicity Detection in Hearing, ed. R. Plomp &
G. Smoorenburg, A.W. Sijthoff, Leiden.
Zwicker, E., E. Terhardt, and E. Paulus (1979), "Automatic
speech recognition using psychoacoustic models," JASA 65(2),
487-498.
Zwislocki, J. (1975), "The Role of the External and Middle Ear
in Sound Transmission," in The Nervous System, Vol 3: Human
Communication and Its Disorders, D. Tower (ed.), Raven Press,
New York.
_I
~~~_CIII_I- - -
- --_I -···--~----.1)[~~~~-111--.--
lI__ P