PSYCH 155/LING 155
syn
lab
UCI
COGNITIVE
SCIENCES
Lecture 3:
Psychology of Language
Prof. Jon Sprouse
Sound and Speech Sounds
1
The representations and processes of
Language
starting
representation
sensory
lexical
acoustic
phonetic
syntactic
S
dʒɑn
conceptual
VP
bɑt
NP
ə
kɑr
final
representation
2
The representation of speech sounds
The first step in language comprehension is to transform the physical signal
(e.g., speech sounds) into a mental representation.
sound waves enter
through the
hearing system
This is the job of the hearing system (e.g., the ears and auditory cortex) - the
hearing system converts external physical sound stimuli into mental
representations.
So we can start our discussion with two questions:
1. What are the (external) physical properties of sound
2. Which of those properties are critical for language?
3
Some physical properties of sound
(i.e., properties of the external
signal)
4
What is sound?
What is sound?
Distortions in air pressure
Distortions travel in waves:
ripples on a pond,
waves in the ocean
Sound waves are distortion
in air pressure
The eardrum can detect these
distortions, and your brain interprets
these distortions as sound
5
What is a sound wave, really?
Sound travels through air, and air is a gas.
Gas molecules fill whatever size space they are given.
space 1
twice the volume, half the density
You can push certain air molecules closer together for a time (increase the air
pressure, called compression):
compression
6
Compression and Rarefaction
When you push air molecules closer together (compression), you also create a
space behind the compression that is less dense called rarefaction.
compression
Compression and rarefaction are two opposing forces - compression causes
rarefaction behind it.
And because gas molecules want to equalize their density in a given space,
compression of one set of molecules will cause a “wave” of compression to occur
throughout the space as the compressed molecules try to “get away” from each
other.
7
A spatial visualization of the
propagation of a sound wave
The best way to visualize the way that a compression wave travels through space
is with a slinky:
If you push one end of a stretched slinky, you can see the first set of coils
compress, and watch the compression wave spread across the slinky as the coils
try to equalize their density.
8
A temporal visualization of sound
waves...
The way that sound waves propagate through space is not very important for
our purposes (we’ll leave that to physicists).
Instead, we are interested in the way that they propagate through time.
This is because speech sound is rarely just a single act of compression (one
wave). Instead, sound sources are usually a series of compressions that occur
with a specific frequency.
To visualize the temporal properties of sound, we focus on a single point in
space (i.e., there is no spatial information at all), and draw the compressions
and rarefactions that occur over time at that single point in space.
crest = higher air pressure,
coincides with compression
normal pressure
time
trough = lower air pressure
coincides with rarefaction
9
Describing Sound Waves
Once you have the temporal representation of sound waves, you can
describe their properties:
amplitude: the size of the distortion, measured in distance the molecules
move during compression and rarefaction.
This is how much energy the wave has. The more energy, the larger the
amplitude, the more the air molecules are moved.
We perceive increases in amplitude as increases in loudness.
10
Describing Sound Waves
You can describe the properties of sound waves:
frequency: the number of cycles of the wave per second. A complete
cycle is compression, rarefaction, return to normal.
3 cycles per second
12 cycles per second
We perceive increases in frequency as increases in pitch.
11
Describing Sound Waves
You can describe the properties of sound waves:
frequency: the number of cycles of the wave per second. A complete
cycle is compression, rarefaction, return to normal.
Hertz (Hz): the number of cycles per second
1Hz = 1 cycle per second
10Hz = 10 cycles per second
10,000Hz = 10,000 cycles per second
Human hearing can detect frequencies in the range of 10Hz-20,000Hz
However, most people lose the ability to detect high frequencies as they
age... this was the science behind those teenager only ringtones!
http://www.noiseaddicts.com/2009/03/can-you-hear-this-hearing-test/
12
Fundamental frequency
Guitar
Guitars have strings. Each string is a different thickness. This makes each
string vibrate at a different frequency, which leads to different tones
(which in music we call notes):
E
329.6 Hz
B
246.9 Hz
G
196
D
146.8 Hz
A
110
E
82.4 Hz
Hz
Hz
fundamental frequency (F0): every object has it’s own fundamental
frequency, it is the frequency at which that object vibrates.
13
Fundamental frequency
Human Voice
Your voice has a fundamental frequency (F0) too.
It is created by your vocal cords (which are really two flaps or membranes
that vibrate):
For males it is around 130hz (C)
for females it is around 220hz (A - almost an octave difference!)
BTW: Middle C is 261.6Hz
14
Harmonics
Harmonics: multiples of the fundamental frequency
When an object vibrates at its fundamental frequency, harmonics are also
activated:
F0
100 Hz
F0
200 Hz
F0
400 Hz
1st
200 Hz
1st
400 Hz
1st
800 Hz
2nd
300 Hz
2nd
600 Hz
2nd
1200 Hz
3rd
400 Hz
3rd
800 Hz
3rd
1600 Hz
4th
500 Hz
4th
1000 Hz
4th
2000 Hz
5th
600 Hz
5th
1200 Hz
5th
1400 Hz
15
Resonance
If you ask any teenager in a garage band what the difference is between an
acoustic guitar and an electric guitar, they will say that one has a hollow body
and one has a solid body.
And what does this difference mean for the instrument?
One difference is that an acoustic
guitar is louder than the electric.
That’s why electric guitars need
amplifiers.
The difference in loudness
(amplitude) between the two is
due to resonance.
But don’t get fooled into thinking that resonance is just about the loudness of
an instrument... we will see shortly that resonance changes more than that!
16
Resonance and hollow bodies
Resonance is a property of all objects. It is the fact that objects vibrate at a
certain frequency.
Resonance is what gives rise to
the idea that you can shatter a
glass by signing: the glass has a
frequency at which it vibrates,
and if you make it vibrate
strongly enough, it might break.
We are actually interested in the resonance that occurs inside of hollow
bodies like the body of a guitar: sound waves reflect off of the walls!
wall
wall
reflected wave 2
reflected wave 1
This reflection causes some
interesting things to happen to
the sound waves through a
process called interference.
original wave
17
Resonance and hollow bodies
There are two types of interference that can be created through resonance
within hollow bodies:
Constructive interference is when the peaks of the reflected waves line up.
This doubles the amplitude of those waves!
The two reflected
waves are in phase,
which means that
air molecules are
being compressed
by two forces at
once.
This doubles the
compression, and
thus doubles the
amplitude of the
wave.
18
Resonance and hollow bodies
There are two types of interference that can be created through resonance
within hollow bodies:
Destructive interference is when the peaks of one wave line up with the
trough of a second wave. This cancels out the wave!
The two reflected waves
are out of phase, which
means that air molecules
are being compressed and
rarefacted at the same
time.
These opposing forces
cancel each other out,
such that there is no
change in air pressure (no
wave).
19
Resonance bands
It is rare for waves to line up perfectly, or to mis-align perfectly.
Instead, you find ranges of frequencies that are amplified, and ranges of
frequencies that are de-amplified. We call these ranges resonance bands.
20
Harmonics and Resonance Bands
Harmonics also create resonance bands
So, when you play an A on a guitar at 110Hz, the body might also
resonate in a band around the harmonic at 220Hz, perhaps 200-240Hz.
It might also resonate in a band around the harmonic at 440Hz, perhaps
400-480Hz.
frequency
H4
H3
Band 5
Band 4
H2
Band 3
H1
Band 2
F0
Band 1
The resonance bands are what make an instrument sound “richer” than a
note played on a string that has no body behind it.
21
Harmonics and Resonance Bands
The Fundamental frequency determines the frequency of the harmonics
(because they are multiples)
In an open space, the 1st harmonic will have the highest amplitude (be
loudest), the 2nd will have the next highest, etc.
frequency
H4
H3
Band 5
Band 4
H2
Band 3
H1
Band 2
F0
Band 1
22
Harmonics and Resonance Bands
The Fundamental frequency determines the frequency of the harmonics
(because they are multiples)
In an open space, the 1st harmonic will have the highest amplitude (be
loudest), the 2nd will have the next highest, etc.
H4
frequency
Band 5
H3
Band 4
H2
Band 3
H1
F0
Band 2
Band 1
But when resonance inside an enclosed space happens, the shape of the
space determines the prominence of the harmonics.
Some will be amplified more than others because they occur within a
resonance band of the shape of the body
23
Harmonics and Resonance Bands
Different instruments have different bodies, and therefore have different
resonance bands
This means they emphasize of different harmonics.
Band 5
Band 5
Band 4
Band 4
Band 3
Band 3
Band 2
Band 2
Band 1
Band 1
This is why instruments “sound different” even though they are playing
the same note. (In music, they call the resonance bands overtones.)
24
The properties of sound that are
critical for speech
25
Which properties are critical for
speech?
Sound is a disturbance of air molecules that travels in a wave (compression,
rarefaction, return to normal), and like all waves it has the following
properties:
1. Amplitude (which we perceive as loudness)
This just changes loudness
2. Frequency (which we perceive as pitch)
a. Fundamental frequency
This just changes pitch
b. Harmonics
This can’t be changed alone
c. Resonance Bands
How do you change this???
How would we test each of these to determine if they are critical for speech
sounds?
General process:
1. Say a speech sound (e.g., ‘a’ or ‘ahh’)
2. Vary the property in question
3. See if the speech sound changes to a different speech sound (e.g., ‘eee’)
26
Speech Sounds and Formants
In fact, you can change the resonance in your vocal tract!
Step 1: Your vocal folds create a fundamental frequency (perhaps 200Hz)
that also has some harmonics (say, 400, 600, 800, etc).
H4
H3
H2
H1
F0
27
Speech Sounds and Formants
In fact, you can change the resonance in your vocal tract!
Step 2: These harmonics resonate in the “body of your instrument”. In the
case of speech, we have two “bodies”: the trachea and the oral cavity
Band 5
Band 4
Band 3
Band 2
Band 1
Band 5
Band 4
Band 3
Band 2
Band 1
H4
H3
H2
H1
F0
28
Speech Sounds and Formants
In fact, you can change the resonance in your vocal tract!
Step 3: We call the most prominent harmonic band for each body (that is,
the most prominent band NOT created by the F0) a FORMANT.
Band 5
= F2 (the second formant)
Band 4
Band 3
Band 2
Band 1
Band 5
Band 4
= F1 (the first formant)
Band 3
Band 2
Band 1
H4
H3
H2
H1
F0
We call these FORMANTS because
they are the frequency bands that
are used to “form” the speech sound
29
Speech Sounds and Formants
In fact, you can change the resonance in your vocal tract!
We can change which harmonic (band) is made most prominent by changing
the shape of the trachea and mouth by moving the tongue!
F2: the most prominent band in the mouth
The exact frequency of F2 will change
based on the shape of the mouth
F1: the most prominent band in the trachea
The exact frequency of F1 will change
based on the shape of the trachea
F0: the frequency created by the vocal folds
30
A demonstration of the effect of
trachea and mouth shape
ah
ee
F2: oral cavity
F1: trachea
duck call
eh
oh
http://www.exploratorium.edu/
exhibits/vocal_vowels/
vocal_vowels.html
31
Which properties are critical for
speech?
Sound is a disturbance of air molecules that travels in a wave (compression,
rarefaction, return to normal), and like all waves it has the following
properties:
1. Amplitude (which we perceive as loudness)
This just changes loudness
2. Frequency (which we perceive as pitch)
a. Fundamental frequency
This just changes pitch
b. Harmonics
This can’t be changed alone
c. Resonance Bands
We call these formants
The reason that you didn’t know how to change the resonance of your voice is
not because you didn’t know how to -- you DO know how to -- the reason is
that you don’t think of it as resonance, you think of it as talking!
(Similarly The reason that you knew how to change the loudness or pitch of
your voice is that you don’t think of those actions as talking, but as altering
your voice.)
32
The human voice versus other
instruments
As we mentioned previously, resonance is important for musical instruments.
The reason that a tuba and violin sound different is that they have different
resonances.
Instruments can’t change their resonances because they have rigid bodies. If
you want different resonances, you have to use different instruments.
Band 5
Band 4
Band 3
F2
F1
Band 2
Band 1
The critical difference between the human voice and instruments is that we can
change the shape of our instrument’s body, and thus change the resonance.
This is why we can make speech sounds, and instruments can’t (and why it
never sounds like an instrument is “talking”).
33
Measuring formants
We can visualize the formants of speech with a type of graph called a
spectrogram:
And indicates increases
in the energy (what
we’ve been calling
‘prominence’) with dark
shading.
The dark shaded lines
represent frequencies
that have lots of energy.
ee
oo
ah
6000 Hz
4000 Hz
2000 Hz
frequency
A spectrogram plots
frequency on the y-axis
0 Hz
Notice that each dark band occurs at a range of frequencies -- this is why we
call them frequency bands.
The two lowest frequency bands are F1 and F2, which we can highlight in red.
34
The end for today
Next time we will look at the details of the acoustic representation of speech
sounds, and how this representation is mapped to a more abstract
(articulatory) representation...
35