Digital Media
Dr. Jim Rowan
ITEC 2110
Audio
What is audio?
First, some demos
• Can you hear this?
– http://s3.amazonaws.com/listverse/audioill
usions/highfreq1.mp3
– “mosquito ring tone”
• Audio illusion “Creep”
– http://www.youtube.com/watch?v=ugriWS
mRxcM
The nature of sound
• There are two special types of audio
• Functionally and uniquely different than other
sounds
– Music
• Cultural status
• Can be represented as non-sound: MIDI
– Speech
• Linquistic content
• Lends itself to special compression
And it’s complicated
• Converting energy to vibrations and back
• Transported through some medium
– Either air or some other compressible medium
• Consider speech
– Starts as an electrical signal (brain & nerves)
– Ends as an electrical signal (brain & nerves)
– But…
And it’s complicated (cont)
http://en.wikipedia.org/wiki/Ear
– Starts as an electrical signal (brain & nerves) ==>
– Muscle movement (vocal chords)
• Vibrates a column of air sending out a series of
compression waves in the air
– Compression waves cause ear membrane to
vibrate ==>
– Moves 3 tiny bones ==>
– Causes waves in the liquid in the inner ear ==>
– Bends tiny hair cells immersed in the liquid ==>
– When bent they fire ==>
– Sends electrical signals to the cerebral cortex
– Processed by the temporal cortex
Audio Illusions
• Play a 200 Hz tone
– Softly at first
– Gradually increase the volume
– Most listeners will report that the tone
drops in pitch as the volume increases
• Complex tones are reported to have
lower pitch than pure tone of the same
frequency
Why do you think…
• You can’t tell where some sounds come from
(some alarms for instance)
• You only need one sub woofer when you
need at least two for everything else
• You can’t tell where sound is coming from
underwater
• Two things running at the same speed make
a “beating” sound
Why do you think… (cont)
• With your eyes closed you can’t tell whether a
sound is in front of you or behind you
• With tinnitis you hear sound that isn’t there
• Phantom sounds
– Heard… but not there
• Masking sounds
– Not simply drowning them out
– Can mask a sound that occurs before the masking
sound is played
Why do you think… (cont)
• You can hear your name in a noisy
room
– Cocktail party effect
– http://en.wikipedia.org/wiki/Cocktail_party_
effect
– Still very much a subject of research
Why?
It’s complicated!
• Sound is physical phenomenon
– Wavelength affects stereo hearing
– Speed of sound affects stereo hearing
– You can tell where a sound comes from if
• the wavelength is long enough and
• the speed that sound travels is slow enough
• to allow the waves arrive at your ears at different times
– Speed of sound affects stereo hearing
• It’s sensory and perceptual experience
• http://en.wikipedia.org/wiki/Psychoacoustics
Processing Audio
Processing audio
• How can we look at sound?
• What do you want to see?
• Waveform displays
– Summed amplitude & time
– Amplitude & frequency components at one
point in time
– Amplitude & frequency & time
Summed amplitude & time
Amplitude & frequency components at
one point in time
Amplitude & frequency & time
Summed amplitude & time
joe took father’s shoe
bench out
Amplitude & frequency & time
Digitized audio
• As we have seen earlier this semester
– Sample rate & quantization level
– Reduction in sample rate is less noticeable than
reducing the quantization level
• Jitter is a problem
– Slight changes in timing causes problems
• 20k+ frequencies?
– Though they can’t be heard they manifest
themselves as aliases when reconstructed
Audio Dithering
• Book description on page 284
– Figure 9.9
• Add random noise to the original signal
• This noise causes rapid transitioning
between the few quantized levels
• Makes audio with few quantization
levels seem more acceptable
Audio processing
terms to know
• Clipping
• Noise gate
– has threshold
• Notch filter
– 60 cycle hum
•
•
•
•
•
Low pass filter
High pass filter
Time stretching
Pitch alteration
Envelope shaping
One thing about humans…
• We can actively “filter out” what we don’t want
to hear
– remember the cocktail party effect?
• Over time we don’t hear the pops and snaps
of a vinyl record
– Have you ever recorded something that you
thought would be good only to play it back and
hear the air conditioner and the traffic?
• A piece of software can’t do this
Compressing sound
• Opposite approach than images
– With images you can toss out the high
frequencies
– With audio you can’t… high frequency
changes are highly significant
Compressing sound (cont)
• Voice?
– Remove silence
• Similar to RLE
– (how many seconds of silence?)
– Non-linear quantization
• “companding”
– Quiet sounds are represented in greater detail than loud
ones
• Mu-law
• A-law
– Allows a dynamic range that would require 12 bits into 8
bits
– 4096 (2**12) ==> 256 (2**8)
Compressing sound (cont)
• Differential Pulse Code Modulation (DPCM)
– Related to inter-frame compression
• It predicts what the next sample will be
• It sends that difference rather than the absolute value
• Not as effective for sound as it is for images
• Adaptive DCPM
– Dynamically varies the step size
• Large differences were encoded using large steps
• Small differences were encoded using small steps
Compressing sound (cont)
that are perceptually based
• Remove what doesn’t matter
• Based on the psycho-acoustic model
– Threshold of hearing
– High and low frequencies not as important (for
voice)
• They require much more power to be heard
– Loud tones can mask quiet ones
– It can mask sounds before and after they occur
Record & Playback
• There are two ways to “record” and then
“playback” the audio
– Play the instruments
• Record the performance
• Play the recording back
– Write the music down
• Send the written-down music
• Perform the written-down music
MIDI
• There is another way to “write down” the
music for performance later.
• Instead of writing it down on sheet
music…
• Write it down as machine instructions
• The music can be recorded loops
• …or it can be generated by the machine
MIDI
• You can use software to create or capture
MIDI music
• You can use software to play back the MIDI
stream
• P 307, Tables 9.1 and 9.2
– Voice numbers
– Drum kit numbers
• Why are there 128 voice numbers?
– Why not 129 or 127?
– How many bits?
• Why are there 32 drum kit numbers?
– Why not 33 or 31?
– How many bits
Questions?