Introduction to Processing Digital
Sounds
adapted from:
Barb Ericson
Georgia Institute of Technology
Sept 2005
Georgia Institute of Technology
How does Hearing Work?
• The outer ear “catches” sounds
• The eardrum vibrates
• The inner ear translates the vibrations to
nerve impulses for the brain to interpret
Georgia Institute of Technology
Acoustics, the physics of sound
• Sounds are waves of air
pressure
– Sound comes in cycles
– The frequency of a wave is
the number of cycles per
second (cps), or Hertz
• (Complex sounds have
more than one frequency
in them.)
– The amplitude is the
maximum height of the
wave
Georgia Institute of Technology
Volume and Pitch
• Our perception of volume is related (logarithmically) to
changes in amplitude
– If the amplitude doubles, it’s about a 3 decibel (dB) change.
– A decibel is a ratio between two intensities: 10 * log10(I1/I2)
– As an absolute measure, it’s in comparison to threshold of
audibility
• 0 dB can’t be heard.
• Normal speech is 60 dB.
• A shout is about 80 dB
• Our perception of pitch is related (logarithmically) to
changes in frequency
– Higher frequencies are perceived as higher pitches
– We can hear between 5 Hz and 20,000 Hz (20 kHz)
– A above middle C is 440 Hz
Georgia Institute of Technology
Digitizing Sound
• In calculus you learn to
estimate a curve by
creating rectangles
• We can do the same to
estimate the sound curve
– Analog-to-digital
conversion (ADC) will give
us the amplitude at an
instant as a number: a
sample
– How many samples do we
need?
Georgia Institute of Technology
What is digital?
• What’s the difference between
– Phonograph and CD?
– VCR tape and DVD?
• Why digitize sound?
• How is ripping to a mp3
different from recording to a tape?
– Reproduction: immediate and future
– Distribution
– Modification
• Why do digital media present new challenges
from analog media?
Georgia Institute of Technology
– Is copyright infringement
new?
Nyquist Theorem
• We need twice as many samples as the
maximum frequency in order to represent (and
recreate, later) the original sound.
• The number of samples recorded per second is
the sampling rate
– If we capture 8000 samples per second, the highest
frequency we can capture is 4000 Hz
• That’s how phones work
– If we capture more than 44,000 samples per second,
we capture everything that we can hear (max 22,000
Hz)
• CD quality is 44,100 samples per second
Georgia Institute of Technology
Try It
• Call a friend on a phone and play some
music over the phone
– How does it sound?
– Phones only transmit 8,000 samples per
second
– The highest frequency you can transmit by
phone is 4000 Hz
– This is fine for voice but what does it do to
music?
Georgia Institute of Technology
Playing a Sound
• We can create a Sound object just as we
created a Picture object
– Get a file name and save a reference to it
• String fileName = FileChooser.pickAFile();
– Pick a file that ends in .wav
– Create the sound object by asking the class to
create a new Sound object and initialize it by
reading data from the given file name
• Sound sound1 = new Sound(fileName);
– Play the Sound
• sound1.play();
Georgia Institute of Technology
Play Sound Exercise
• Try creating a Sound
object and playing it
by
– Specifying it in steps
– Specifying it all at
once
• How would you play
the same sound
twice?
Georgia Institute of Technology
Digitizing Sound in the Computer
• Each sample is stored as a number (two bytes)
• What’s the range of available combinations?
– 16 bits, 216 = 65,536
– But we want both positive and negative values
• To indicate compressions and rarefactions.
– What if we use one bit to indicate positive (0) or
negative (1)?
– That leaves us with 15 bits
– 15 bits, 215 = 32,768
– One of those combinations will stand for zero
• We’ll use a “positive” one, so that’s one less pattern for
positives so the range is from -32,768 to 32,767
Georgia Institute of Technology
Sound Basics
• new Sound(fileName)
– Will create a new Sound object from the data in the file with the
passed file name
• soundObj.play()
– Will start the sound playing
• soundObj.explore();
– Will open a sound explorer on the object
• soundObj.blockingPlay()
– Will play the sound and wait to return until the sound is finished
• soundObj.write(String fileName)
– Will write out the sound to the file
• Sound.convert(String mp3File, String wavFile)
– Will convert the sound in filename mp3File to a file called
wavFile so that it can be used by the Sound class
Georgia Institute of Technology
The Sound Explorer
• Not all of the sound is
shown when you
explore a sound
– Skips values to fit in
the window
• You can zoom in
– To see all sample
values
• You can zoom out
– To fit the sound in the
window again
Georgia Institute of Technology
Getting the Sound Sample Values
• A Sound has many values in it
– Numbers that represent the sound at that time
in the sample
• You can get an array of SoundSample
objects
– SoundSample[] sampleArray =
sound1.getSamples();
Georgia Institute of Technology
Explore the Sound Sample Values
• Zoom in to see all the sound values
Click here to pick an index
See the value
Type in an index
Click here to go to the next index
Georgia Institute of Technology
Print the Sound Sample Value
• You can get the SoundSample object from
the array at an index
– SoundSample sample = sampleArray[0];
• And then get the value from that
– System.out.println(sample.getValue());
• What are the first 10 values of the Sound
created from the file croak.wav?
Georgia Institute of Technology
Changing the Value of a Sound Sample
• You can set the value of a SoundSample
– sample.setValue(value);
– This will change the value in the Sound object as well
• So how would you change the value to the
original value * 2?
SoundSample sample = sampleArray[0];
sample.setValue(sample.getValue() * 2);
• How could you double all of the samples?
– By hand?
• How many samples are there?
– Using a loop?
Georgia Institute of Technology
For-Each Loop (Java 5.0)
• For each of the elements in a collection of
objects do the body of the loop
– Each time through the loop the variableName
will refer to a different object in the collection
for (type variableName : collection)
{
// statement to repeat
}
Georgia Institute of Technology
For-Each Loop to Process Sound Samples
SoundSample[] sampleArray = this.getSamples();
for (SoundSample sample : sampleArray)
{
int value = sample.getValue(); // get the value
sample.setValue(value * 2); // set the value
}
Georgia Institute of Technology
Increase Volume with For-Each Loop
public void increaseVolume()
{
SoundSample[] sampleArray = this.getSamples();
int value = 0;
// value at sample
// loop through SoundSample objects
for (SoundSample sample : sampleArray)
{
value = sample.getValue(); // get the value
sample.setValue(value * 2); // set the value
}
}
Georgia Institute of Technology
Testing increaseVolume
• String file =
FileChooser.getMediaPath(“gettysburg10.wav“);
•
•
•
•
•
•
Sound soundObj = new Sound(file);
soundObj.play();
soundObj.explore();
soundObj.increaseVolume();
soundObj.play();
soundObj.explore();
Georgia Institute of Technology
Decrease Volume Exercise
• Write a method to decrease the volume of the
sound
– decreaseVolume()
– Divide each value by 2
• What parts need to change from the last
method?
– Only the calculation of the new value
• Try it:
Sound s = new Sound(
FileChooser.getMediaPath(“ gettysburg10.wav”));
s.explore();
s.decreaseVolulme();
s.explore();
Georgia Institute of Technology
What Does For-Each Do?
• It needs to use each element in the array
one and only one time
– So it needs to keep track of the current index
– It needs to check that the current index is less
than the length of the array
• And stop when they are equal
– It needs to increment the index after the loop
body has executed
• This is explicit in a while loop
Georgia Institute of Technology
While Loop
• Loops while a boolean (true or false) test is true
– When the test is false execution continues with the
first statement after the loop
while(test)
{
// statements to be done while the test is true
}
– To use a while loop you may need to declare
variables before the loop and change them in the loop
Georgia Institute of Technology
While Loop to Process Sound Samples
int index = 0;
// starting index
while (index < sampleArray.length)
{
SoudnSample sample = sampleArray[index]; //
get current obj
int value = sample.getValue(); // get the value
sample.setValue(value * 2); // set the value
index++;
// increment index
}
Georgia Institute of Technology
Increase Volume with While Loop
public void increaseVolume()
{
SoundSample[] sampleArray = this.getSamples(); // get array
int index = 0;
// starting index
// loop through SoundSample objects
while (index < sampleArray.length)
{
SoundSample sample = sampleArray[index]; // get current obj
int value = sample.getValue(); // get the value
sample.setValue(value * 2); // set the value
index++;
// increment index
}
}
Georgia Institute of Technology
Tracing Execution
• The index is set to 0
• The value is set to the
value in the array at that
index (59)
• The sample value at the
current index is set to 2 *
value
• The index changes to the
next index (1)
• We check if the index is
less than the length of the
array and
– If so do the loop again
– Else jump to the first
statement after the loop
Georgia Institute of Technology
Memory versus Disk
• When we read from a file we read from
disk into memory
– Computers only do calculations on memory
• We change the values in memory
• The file on the disk hasn’t changed
• To save our new sound we need to write a
file to the disk
– soundObj.write(fileName);
Georgia Institute of Technology
While Loop versus For Loop
• It is easy to make mistakes when you use
a while loop for looping a set number of
times
– Forget to declare variables before the loop
– Forget to increment the variables in the loop
before the next test
• Programmers use a For loop when the
number of times to loop is known
– And a while loop when you don’t know
Georgia Institute of Technology
For Loop
• A for loop allows you to declare and
initialize variables, specify the test, and
specify the way the variables change
– All in one place
– But, they still happen in the usual place
for(int index = 0;
index < sampleArray.length;
index++)
{
}
Georgia Institute of Technology
Increase Volume with a For Loop
public void increaseVolume()
{
SoundSample[] sampleArray = this.getSamples();
// loop through all the samples in the array
for (int index = 0; index < sampleArray.length; index++)
{
SoundSample sample = sampleArray[index];
int value = sample.getValue();
sample.setValue(value * 2);
}
}
Georgia Institute of Technology
Methods calling Methods
• One method can call another
– Change decreaseVolume and
increaseVolume to call changeVolume
• You can use
– this.changeVolume
• To invoke the method on the current object
• Or you can use
– changeVolume
• The this is implicit and will be added by the
compiler
Georgia Institute of Technology
Copyright
• US Constitution (Article I, Section 8, Clause 8):
“To promote the Progress of Science and useful Arts”
– What can you copyright?
•
Fixed, tangible medium of expression with a modicum of originality
– How do you copyright?
•
Don’t need anything. Registration necessary for copyright infringement suits
– Authors given limited monopoly so they will disclose to public
– Concessions
1. Fair use
2. First sale
3. Limited Time
– Evolving Bargain: Copyright holder may profit from works and public
has access and can build upon them
– What would happen if information could only be shared if the owner
provided permission?
Georgia Institute of Technology
Fair use
•
•
•
Use copyrighted works without permission if the use does not
unduly interfere with the copyright owner’s market for a work
Include personal, noncommercial uses
4 prong test
1. Purpose and character of use (commercial vs. non-profit or
educational)
2. Nature of copyrighted work
3. Amount and substantiality of the portion used
4. Effect of the copying upon market
•
•
Example: using a VCR to time-shift a broadcast program
Reverse engineering
–
–
OK when extracting unprotected elements
Connectix Virtual PlayStation
Georgia Institute of Technology
Digital rights management
• Idea: copying is hard to control, so make the copying
process itself difficult
– Restrict the use of digital files in order to protect interest of
copyright holders
– Control file access
– Implemented in operating system, program software, or in the
actual hardware of a device
• Digital watermarking
– Make information so that unauthorized copying can be detected
• Serial Copy Management System (Audio Home
Recording Act 92)
• Dystopian and utopian results?
• Privacy issues?
Georgia Institute of Technology
Test Case
Artist Def Jeff creates a new song that includes a
four-bar percussion sample from a Rolling
Stones song. He uses the sample without
receiving permission from the copyright holder,
His DJ overlays the track containing the sample
with 40 tracks of original music and puts the
song on his latest CD.
• Were Def Jeff’s actions within the limits of fair
use?
• Would he lose a lawsuit?
• Were his actions ethical?
Georgia Institute of Technology