Multimedia Authoring
Introduction to Sound
• History A Brief History of Computer Music
• MIDI
• Analogue sound
• Digital audio
Multimedia Authoring
1
MIDI Audio
MIDI (Musical Instrument Digital
Interface)
Creating and playing MIDI music requires
• Sequencer software - to record and edit MIDI data
• Sound synthesizer (e.g. sound card) – to generate music
• MIDI keyboard (not necessary for playback only)
MIDI is device dependent
Multimedia Authoring
2
MIDI Audio
General MIDI
Included with each note to be played in a MIDI file is a patch
number. This number defines which instrument is to play the
sound.
A GM – compatible patch map is most commonly used. 128
instruments and sound effects are included.
http://people.virginia.edu/~pdr4h/gmpatch.html
Multimedia Authoring
3
MIDI Audio
Strengths and Weaknesses of MIDI Files
Compact size
• Why are MIDI files so compact?
• Sequence of commands containing instructions about
how to generate music (some similarities with sheet
music)
Simple to change the instrument playing the sound
Cannot be used to generate arbitrary sounds
• Instrumental music only
Use MIDI if
• Digital audio not suitable due to lack of memory/bandwidth
• Have high quality sound source
• Have control over playback
• Don’t need spoken dialog
Multimedia Authoring
4
Analogue Sound
What is a Sound Wave?
Animation courtesy of Dr. Dan Russell, Kettering University
http://paws.kettering.edu/~drussell/Demos/waves-intro/wavesintro.html
Multimedia Authoring
5
Analogue Sound
Multimedia Authoring
6
Analogue Sound
Multimedia Authoring
7
Analogue Sound
Characteristics of Sound Waves
Wavelength; this is the distance from the crest of one wave to the crest
of the next.
Frequency; this is the number of waves that pass a point in each second.
Amplitude; this is the measure of the amount of energy in a sound wave.
Multimedia Authoring
8
Analogue Sound
Frequency
Rapid vibrations of air molecules create a high-pitched sound (treble);
a slower rate of vibration creates a low-pitched sound (bass)
(Beggs, Josh and Thede, Dylan, 2001).
Multimedia Authoring
9
Analogue Sound
Frequency
The frequency of a sound tells us how fast the air is moving, and is
measured in Hertz (Hz).
Most people can hear frequencies in the range 100 Hz-15 000 Hz.
60 Hz | 100 Hz | 440Hz | 500 Hz | 1 000 Hz |
8 000 Hz | 10 000 Hz | 12 000 Hz | 15 000 Hz
Human ears are most sensitive in a band from 2 000 Hz to 5 000 Hz,
and being able to hear in this range is important to being able to
understand speech.
2 000 Hz | 4 000 Hz | 5 000 Hz
A few people can hear very high frequencies above 19 000 Hz.
20 000 Hz
Multimedia Authoring
10
Analogue Sound
Amplitude
The size of a wave (how much it is "piled up" at the high
points) is its amplitude. For sound waves, the bigger the
amplitude, the louder the sound.
Schmidt-Jones, C. Amplitude and Dynamics, Connexions
Web site. http://cnx.org/content/m12372/1.2/, Jan 5, 2006.
Multimedia Authoring
11
Analogue Sound
Level
The level of sound, known as volume, is measured in decibels (dB).
4 000 Hz : maximum level 0.0 dB
4 000 Hz : half power -6.0 dB
4 000 Hz : very quiet -18.0 dB
4 000 Hz : getting too loud (starting to distort) +6.0 dB
Use the VU (volume) meter in GarageBand so that the loudest part of your
pieces approach, or just get into, the red range.
Multimedia Authoring
12
Analogue Sound
Analogue Recording and Playback
A direct representation of sound waves are stored. For example, Thomas
Edison's phonograph recorded sound on tinfoil wrapped around a cardboard
cylinder. A horn focused the sound waves onto a thin membrane attached to a
needle. As the cylinder rotated, the needle cut a continuous groove into the
tinfoil. The air pressure variations of the sound waves caused the diaphragm
and needle to move up and down, varying the depth of the groove to create a
recording. Vinyl records use basically the same principle, except the needle
moves from side to side.
To playback, the needle was placed at the start of the groove and the cylinder
rotated. As the needle traced the path of the groove around the cylinder, the
varying depth of the groove moved it up and down. The needle caused the
diaphragm to move the same amount as it did when the recording was made,
reproducing the original sound waves.
Multimedia Authoring
13
Analogue Sound
Analogue Audio Signals
Analogue audio signals are the electronic version of a groove on a record.
Variations in voltages on a wire are like the variations in the depth (or width)
of a groove. They both represent the air pressure variations of sound.
Microphones convert air pressure variations of sound waves into a varying
voltage sound signal. Magnetic tape records sound by representing variations
in sound waves as variations in the amount of magnetism stored in a metallic
coating on a thin strip of plastic tape.
Speakers have a coil of wire into which the sound signal is fed. The coil is
surrounded by a magnet. Variations in the signal cause the coil to move and a
diaphragm attached to the coil creates air pressure variations to form sound
waves.
Multimedia Authoring
14
Digital Sound
Digitising Sound
We can digitise sound from any source - natural or prerecorded.
In digital audio, sound is represented by a sequence of numbers that
correspond to the signal level at a predetermined interval. The signal is made
up of bits, where 0 represents low (off) and 1 high (on - close to maximum
voltage).
To hear the original sound, digital signals are converted back to analog as
speakers recreate sound waves using a digital to analog (D/A) converter e.g.
there is a D/A in CD players and sound cards.
Multimedia Authoring
15
Digital Sound
Sound Samples
If we digitise sound, we are sampling sound.
The process converts an analog signal to a digital by using an analog to digital
(A/D) converter.
The quality of digital recordings depend on the sampling rate and the bitdepth.
Every nth of a second, a sample of sound is taken and stored as digital
information in bits and bytes.
Sampling Rate: frequency measured in kilohertz
Bitdepth: how many numbers are used to represent the value of each sample also known as sample size, resolution, or dynamic range.
The more often you take a sample and the more data you store about that
sample, the finer the resolution and quality when played back.
Multimedia Authoring
16
Digital Sound
How Sampling Works
Figure 5-2, Vaughan (2008)
NB. The original waveform cannot be reconstructed if the sampling
frequency is too low!
Multimedia Authoring
17
Digital Sound
Sound Sampling Examples
4000 Hz
4 000 Hz sample rate = 8 000 Hz
4 000 Hz sample rate = 11 025 Hz
4 000 Hz sample rate = 16 000 Hz
4 000 Hz sample rate = 22 050 Hz
4 000 Hz sample rate = 32 000 Hz
4 000 Hz sample rate = 44 100 Hz
Multimedia Authoring
18
Digital Sound
Sound Samples
Professional recording studios use 96 000 Hz sampling frequency and mix it
down to CD quality. What is the sampling rate that gives CD quality?
To record a certain frequency we need at least (but preferably more than) two
samples for each period to accurately record the peaks and troughs. So we need
a rate at least twice as high as the highest frequency to be recorded.
•What is the highest frequency people are likely to hear?
•What sample rate is just greater than twice that frequency?
•If we use a sample rate of 96 000 Hz, will we be able to hear 48 000 Hz?
Using this sample rate, we will be able to record more than four samples
for each 20 000 Hz period. Therefore, the chance of losing high
frequencies dramatically reduces.
The three sampling frequencies most often used in multimedia are:
1. 11 025 Hz
2.
Hz
3.
Hz
Some equipment and software allows up to 192 000 Hz and this will probably
soon become the professional standard.
The advantage of the higher sampling rates are much better sound quality.
Suggest a disadvantage.
Multimedia Authoring
19
Digital Sound
Resolution
The resolution of a digital signal is the number of distinct integer values
available to represent the voltage level of an analog signal. The exact voltage of
a sample is rounded off to the nearest integer. The more integers the higher the
resolution and so the more accurately the voltage can be represented.
CD audio uses 16 bits per sample (16-bit resolution).
How many possible integer values?
Professional recording studios use systems with 24-bit resolution.
High sampling rates have little effect if the resolution is too low.
Multimedia Authoring
20
Digital Sound
Multimedia Authoring
21
Digital Sound
Quantization
Because computers process integers much more efficiently than floating point
(decimal) numbers, the value of each sample is rounded to the nearest
integer.
Because the voltage of an analog signal varies continuously, the values
measured for most samples will not be whole numbers. So the A/D converter
rounds the value of each sample to the nearest whole number.
The range of possible values is determined by the resolution of the signal.
Multimedia Authoring
22
Digital Sound
Problems with Quantization
Clipping:
If the amplitude is greater than the intervals available, clipping of the top and
bottom of the wave occurs. This can greatly distort it.
The maximum level is 0 dB. Other levels are negative. If the average level of a
signal goes above zero it is clipped. It is usual to set average signal levels a bit
below maximum to allow for unexpected peaks.
Unwanted background hissing: can be produced by quantization.
Quantization noise - noise generated by (bad) quantization.
The first digital pianos did not sell well - not much memory so not very
dynamic. Also, there was a rough edge at the end of samples.
The conversion of a piano sample (that slowly fades away) with 65536
levels to 8 bits (how many levels) gave a very poor result.
Remedy - dithering
While sampling the piano, the soundcard adds a little noise to the signal
(3-6 dB). This helps the signal become a little louder. We do not hear the
noise, which is very soft and does not change as much as the recorded
signal - ears overlook it.
Multimedia Authoring
23
Digital Sound
Quantization
Figure 4-2, Vaughan (2011)
Multimedia Authoring
24
Digital Sound
Bit Rate
The bit-rate is the number of bits per second used to represent a signal.
It has a direct correlation to file size and quality.
e.g. Bit rate of uncompressed audio:
bit rate = sampling rate * resolution * channels
e.g. 44,100 * 16 * 2 = 1,411,200 bps
Since the quality of your audio is based on the quality of your recording and not
the quality of the device that your end user will play the audio, digital audio is
said to be device independent.
Multimedia Authoring
25
Digital Sound
Bit Rate
The bit-rate is the number of bits per second used to represent a signal.
It has a direct correlation to file size and quality.
e.g. Bit rate of uncompressed audio:
bit rate = sampling rate * resolution * channels
e.g. 44,100 * 16 * 2 = 1,411,200 bps
Since the quality of your audio is based on the quality of your recording and not
the quality of the device that your end user will play the audio, digital audio is
said to be device independent.
Multimedia Authoring
26
Digital Sound
Space required for stereo digital audio
(uncompressed)
Bit
Depth
Sample
Rate
(Hz)
Bit Rate
File Size
(Mbit/sec) of one
stereo
minute
(MB)
File size
of a three
minute
song
(MB)
16
44,100
1,35
10.1
30.3
16
48,000
1.46
11.0
33
24
96,000
4.39
33.0
99
Multimedia Authoring
27
Digital Sound
Digitizing Process
1. When we press the record button, the soundcard starts a very accurate
stopwatch (sample rate).
2. low pass, anti-aliasing filter: very high frequencies that sound card cannot
handle cut are off (maximum frequency that can be recorded at a certain
sample rate is half that rate - called the Nyquist frequency)
•reduces the sound quality
•without it, the sound would be more seriously damaged (become
unrecognizable)
•low pass - lets the low frequencies pass through
•anti-aliasing - smoothes, blurs
•filter - takes away some part and leaves the rest
3. Every time stopwatch finishes a cycle, the analog to digital converter
(ADC) in the soundcard looks at the filtered input signal and calculates how
loud it as at that exact moment and transforms the loudness level to the
nearest digital number, which is stored in memory or secondary memory.
Multimedia Authoring
28
Multimedia Authoring
References
A brief history of computer music
http://adagio.calarts.edu/~eric/cm.html
Sound Waves
http://www.fi.edu/fellows/fellow2/apr99/soundvib.html
Digital Audio
http://emeld.org/workshop/2003/Bartek-demo.html
How analogue and digital recording works
http://electronics.howstuffworks.com/analog-digital1.htm
Vaughan, T. (2011) Multimedia: Making It Work, 8th Ed. ,
New York: McGraw-Hill
Multimedia Authoring
29