Introduction to Computer Science
and Media Computation
Barb Ericson
Georgia Institute of Technology
Feb 2010
Intro CS and Media Computation
1
Learning Goals
•
•
•
•
What is Computer Science?
What are programming languages?
What do computers understand?
What is Media Computation?
06-Intro-Object-Oriented-Prog
2
What is Computer Science?
• The study of process, how to specify what
is to be done and define the stuff being
processed.
• You can say that is the study of
computational recipes
– Ones that can be executed on a computer
– A recipe that runs on a computer is called a
program
06-Intro-Object-Oriented-Prog
3
What do Computer Scientists do?
• Study if there are better ways to write recipes
– Algorithms – textual description of how to solve a problem
• Study how to structure the data in the recipes
– Data structures and databases
• Determine if there are recipes that can't be written? That
make machines intelligent?
– Theory, artificial intelligence
• Study how to make computers easier for people to use
– Human-computer interface
• Study how computers communicate with each other
– Networking
• Study how to create 3D models and simulations
– Graphics and scientific computing
06-Intro-Object-Oriented-Prog
4
Multimedia CS1 in Python
•
Focus: Learning programming and CS concepts
within the context of media manipulation and
creation
– Converting images to grayscale and
negatives, splicing and reversing sounds,
writing programs to generate HTML, creating
movies out of Web-accessed content.
– Computing for communications, not
calculation
06-Intro-Object-Oriented-Prog
5
Python as the programming language
• Used in commercial
contexts
– IL&M, Google, Nextel, etc.
• Minimal syntax
• Looks like other
programming languages
– Potential for transfer
• Actually using Jython
which is Python with
Java underneath
06-Intro-Object-Oriented-Prog
6
Some Computer Science Topics
inter-mixed
• algorithms across media
– Sampling a picture (to scale it) is
the same algorithm as sampling a
sound (to shift frequency)
– Blending two pictures (fading one
into the other) and two sounds is
the same algorithm.
• representations and mappings
(Goedel)
– From samples to numbers (and
into Excel), through a mapping to
pixel colors
06-Intro-Object-Oriented-Prog
7
We will program in JES
• JES: Jython
Environment for
Students
• A simple editor (for
entering in our
programs or
recipes): the
program area
• A command area
for entering in
commands for
Python to execute.
06-Intro-Object-Oriented-Prog
Program Area
Command Area
8
Using JES – Try the following
>>> print 3 + 5
>>> print 23.2 / 3
>>> print 1 / 3
>>> print 1.0 / 3
>>> print 10 % 3
>>> print "Hello"
>>> print "Hello" + "Barb"
>>> print 10 > 3
>>> print 3 > 10
Print will print out
the result from the
following expression
06-Intro-Object-Oriented-Prog
9
Did any answer surprise you?
• Integer division results in an integer answer
1/3=0
– The values after the decimal point are thrown away
– If you want a floating point result using a floating point
value in the expression (1.0 / 3)
• You can append strings one after the other, but
this doesn't add any spaces
>>> print "Hello" + "Barb"
HelloBarb
• Python uses 0 for false and 1 for true
>>> print 10 > 3
1
>>> print 3 > 10
0
06-Intro-Object-Oriented-Prog
10
Command Area Editing
• Up/down arrows walk through command
history
• You can edit the line at the bottom
– and then hit Return/Enter
– that makes that last line execute
06-Intro-Object-Oriented-Prog
11
Demonstrating JES for files and sounds
>>> print pickAFile()
c:/ip-book/mediasources/barbara.jpg
>>> print makePicture(pickAFile())
Picture, filename c:/ip-bookmediasources/barbara.jpg height
294 width 222
>>> show(makePicture(pickAFile()))
None
>>> print pickAFile()
C:/ip-book/mediasources/hello.wav
>>> print makeSound(pickAFile())
Sound of length 54757
>>> print play(makeSound(pickAFile()))
None
06-Intro-Object-Oriented-Prog
12
Naming parts – declaring variables
• You can name the result from a function
call
– And then use the name as input to other
functions
myFile = pickAFile() # name the picked file
myPict = makePicture(myFile) # name the
picture
show(myPict)
• The value associated with that name is
06-Intro-Object-Oriented-Prog
13
Quick Calculation
• What if an item is 30% off and you also
have a coupon for an additional 20% off
the sale price?
– If the original cost was $45.00, how much is
the price after the 30% and then how much do
you pay with the additional 20% off?
– Use the python command area to figure it out
• Name the result of each calculation
06-Intro-Object-Oriented-Prog
14
Making our own functions
• To make a function, use
the command def
• Then, the name of the
function, and the names
of the input values
between parentheses
(“(input1)”)
• End the line with a colon
(“:”)
• The body of the recipe is
indented (Hint: Use three
spaces)
– That’s called a block
06-Intro-Object-Oriented-Prog
15
Making functions the easy way
• Get something working by typing
commands in the command window
(bottom half of JES)
• Enter the def command in the editing
window (top part of JES)
• Copy-paste the right commands up into
the recipe
06-Intro-Object-Oriented-Prog
16
A recipe for playing picked sound files
def pickAndPlay():
myFile = pickAFile()
mySound = makeSound(myFile)
play(mySound)
Note: myFile and mySound, inside pickAndPlay(), are
completely different from the same names in the command
area. We say that they are in a different scope.
06-Intro-Object-Oriented-Prog
17
Blocking is indicated for you in JES
• Statements that are
indented the same,
are in the same block.
• Statements in the
same block as the
cursor are enclosed in
a blue box.
• What happens when
you type
pickAndShow is that
all statements in the
06-Intro-Object-Oriented-Prog
block will be
18
Image Processing
• Goals:
– Give you the basic understanding of image
processing, including psychophysics of sight,
– Identify some interesting examples to use
06-Intro-Object-Oriented-Prog
19
We perceive light different from how it
actually is
• Color is continuous
– Visible light is in the wavelengths between
370 and 730 nanometers
• That’s 0.00000037 and 0.00000073 meters
• But we perceive light with color sensors
that peak around 425 nm (blue), 550 nm
(green), and 560 nm (red).
• Our brain figures out which color is which by
figuring out how much of each kind of sensor is
responding
• One implication: We perceive two kinds of “orange”
— one that’s spectral and one that’s red+yellow
(hits our color06-Intro-Object-Oriented-Prog
sensors just right)
20
• Dogs and other simpler animals have only two
Luminance vs. Color
• We perceive borders of
things, motion, depth
via luminance
– Luminance is not the
amount of light, but our
perception of the
amount of light.
– We see blue as “darker”
than red, even if same
amount of light.
Luminance is actually
color blind. Completely
different part of the brain.
• Much of our luminance
perception is based on
comparison to 06-Intro-Object-Oriented-Prog
21
Digitizing pictures
• We digitize pictures into lots of little dots
• Enough dots and it looks like a
continuous whole to our eye
– Our eye has limited resolution
– Our background/depth acuity is particularly
low
• Each picture element is referred to as a
pixel
• Pixels are picture
elements
06-Intro-Object-Oriented-Prog
– Each pixel object knows its color
22
Exploring Pictures
>>> file = pickAFile()
>>> beachPict =
makePicture(file)
>>>
explore(beachPict)
• Zoom in to see
individual pixels
• Move the cursor to
see the x and y
values
• Look at the red, 06-Intro-Object-Oriented-Prog
green, and blue
23
Encoding color
• Each pixel encodes color at that position in the
picture
• Lots of encodings for color
– Printers use CMYK: Cyan, Magenta, Yellow, and
blacK.
– Others use HSB for Hue, Saturation, and Brightness
(also called HSV for Hue, Saturation, and Value.
• We’ll use the most common for computers
– RGB: Red, Green, Blue
06-Intro-Object-Oriented-Prog
24
Encoding Color: RGB
• In RGB, each color has
three component colors:
– Amount of redness
– Amount of greenness
– Amount of blueness
• Each does appear as a
separate dot on most
devices, but our eye
blends them.
• In most computer-based
models of RGB, a single
byte (8 bits) is used for
each
– So a complete RGB color
is 24 bits, 8 bits of each
06-Intro-Object-Oriented-Prog
25
Making Colors with Light
• Type
>>> myColor = pickAColor()
• Try to create
– White
– Black
– Yellow
– Red
– Brown
– Purple
06-Intro-Object-Oriented-Prog
26
Basic Picture Functions
• makePicture(filename) creates and returns
a picture object, from the JPEG file at the
filename
• show(picture) displays a picture in a
window
• explore(picture) makes a copy of the
picture and shows it in the explorer
window
• We’ll learn functions for manipulating
pictures like getColor,
setColor, and
06-Intro-Object-Oriented-Prog
27
repaint
Writing a recipe:
Making our own functions
• To make a function, use the
command def
• Then, the name of the function,
and the names of the input
values between parentheses
(“(input1)”)
• End the line with a colon (“:”)
• The body of the recipe is
indented (Hint: Use two
spaces)
• Your function does
NOT exist for JES
until you load it 06-Intro-Object-Oriented-Prog
28
Saving Functions in Files
• Click on File and then Save Program
– Name it with some file name and .py at end
• You can define more than one function in
a file
– Maybe call these pictureFunctions.py
• You can later open these files up
– And use the Load Program button to load all
functions in the file
• You can build a library of python functions
for working with
pictures, sounds, movies,29
06-Intro-Object-Oriented-Prog
etc
Modifying a Pixel Color
• You can get the amount of red, green, or
blue
– redValue = getRed(pict)
– greenValue = getGreen(pict)
– blueValue = getBlue(pict)
• You can change the amount of red, green,
or blue
– setRed(pict,value)
– setGreen(pict,value)
– setBlue(pictValue)
06-Intro-Object-Oriented-Prog
30
Modifying Colors
• You can also get the color from a pixel
– myColor = getColor(pixel)
• You can create a new color by giving
values for red, green, and blue from 0 to
255
– newColor = makeColor(255,0,0)
• You can set a color using
– setColor(pixel,newColor)
06-Intro-Object-Oriented-Prog
31
How to change lots of pixels?
• If we want to change all the pixels in a picture
how can we do that?
– In a 640 x 480 picture that is 307,200 pixels
– That sounds too long to do one at a time
• Computers are very fast and can process billions
of instructions per second
– But we wouldn't want to name each pixel or modify
the color on each one by typing in the commands
307,200 times
• We can get an array of pixels to process
– Using getPixels(picture) and loop through the pixels in
the array
06-Intro-Object-Oriented-Prog
32
– It gets the first row of pixels followed by the second
What is an array?
• Space in memory for many values of the
same type
– Numbers, pictures, pixels, etc
• You can refer to the elements of an array
using an index
– Starting with 0 for the first element
– And length – 1 for the last element
06-Intro-Object-Oriented-Prog
33
Processing Pixels in an Array
>>> file="C:/ip -book/mediasources/barbara.jpg"
>>> pict=makePicture(file)
>>> show(pict)
>>> pixels = getPixels(pict)
>>> setRed(pixels [0], getRed(pixels [0]) * 0.5)
>>> setRed(pixels [1], getRed(pixels [1]) * 0.5)
>>> setRed(pixels [2], getRed(pixels [2]) * 0.5)
>>> setRed(pixels [3], getRed(pixels [3]) * 0.5)
>>> setRed(pixels [4], getRed(pixels [4]) * 0.5)
>>> setRed(pixels [5], getRed(pixels [5]) * 0.5)
>>> repaint(pict)
06-Intro-Object-Oriented-Prog
34
Use a loop!
Our first picture recipe
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Used like this:
>>> file="c:/ip-book/mediasources/katie.jpg"
>>> picture=makePicture(file)
>>> explore(picture)
>>> decreaseRed(picture)
>>> explore(picture)
06-Intro-Object-Oriented-Prog
35
It’s not iteration—it’s a set operation
• Research in the 1970’s found that people
are better at set operations than iteration.
– For all records, get the last name, and if it
starts with “G” then… => HARD!
– For all records where the last name starts with
“G”… => Reasonable!
• Because the Python for loop is a forEach,
we can start out with treating it as a set
operation:
– “For all pixels in the picture…”
06-Intro-Object-Oriented-Prog
36
How do you make an omelet?
• Something to do with eggs…
• What do you do with each of the eggs?
• And then what do you do?
All useful recipes involve repetition
- Take four eggs and crack them….
- Beat the eggs until…
We need these repetition (“iteration”)
constructs in computer algorithms too
- Today we will introduce one of them
06-Intro-Object-Oriented-Prog
37
Decreasing the red in a picture
• Recipe: To decrease the red
• Ingredients: One picture, name it pict and pass
it to the function where we will call it picture
• Step 1: Get all the pixels of picture. For each
pixel p in the set of pixels…
06-Intro-Object-Oriented-Prog
• Step 2: Get the value
of the red of pixel p, and
38
Use a for loop!
Our first picture recipe
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
06-Intro-Object-Oriented-Prog
The loop
- Note the
indentation!
39
How for loops are
written
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
• for is the name of the command
• An index variable is used to hold each of the
different values of a sequence
• The word in
• A function that generates a sequence
– The index variable will be the name for one value
in the sequence, each time through the loop
• A colon (“:”)
• And a block (the06-Intro-Object-Oriented-Prog
indented lines of code)
40
What happens when a for loop is executed
• The index variable is set to an item in the
sequence
• The block is executed
– The variable is often used inside the block
• Then execution loops to the for statement,
where the index variable gets set to the next
item in the sequence
• Repeat until every
value in the sequence 41
06-Intro-Object-Oriented-Prog
getPixels returns a sequence of pixels
• Each pixel knows
its color and place
in the original
def decreaseRed(pict):
picture
for p in getPixels(pict):
• Change the pixel
value=getRed(p)
setRed(p,value*0.5)
and you change
the picture
• So the loops here
assign the index
variable p to each
pixel in the picture
picture, one at a06-Intro-Object-Oriented-Prog
time.
42
Do we need the variable value?
• No
– We can calculate the original red amount right when we
are ready to change it.
– It’s a matter of programming style. The meanings are
the same.
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
def decreaseRed(pict):
for p in getPixels(pict):
setRed(p, getRed(p) * 0.5)
06-Intro-Object-Oriented-Prog
43
Let’s walk that through slowly…
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Here we take a picture
object in as a parameter
to the function and call it
pict
pict
06-Intro-Object-Oriented-Prog
44
Now, get the pixels
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
We get all the pixels from
the picture, then make p
be the name of each one
one at a time
pict
Pixel,
color
r=135
g=131
b=105
p
Pixel,
color
r=133
g=114
b=46
Pixel,
color
r=134
g=114
b=45
getPixels()
…
06-Intro-Object-Oriented-Prog
45
Get the red value from pixel
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
We get the red value of
pixel p and name it value
pict
Pixel,
color
r=135
g=131
b=105
p
Pixel,
color
r=133
g=114
b=46
Pixel,
color
r=134
g=114
b=45
getPixels()
…
value
= 135
06-Intro-Object-Oriented-Prog
46
Now change the pixel
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Set the red value of pixel
p to 0.5 (50%) of value
pict
Pixel,
color
r=67
g=131
b=105
p
Pixel,
color
r=133
g=114
b=46
Pixel,
color
r=134
g=114
b=45
getPixels()
…
value
= 135
06-Intro-Object-Oriented-Prog
47
Then move on to the next pixel
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Move on to the next pixel
and name it p
pict
Pixel,
color
r=67
g=131
b=105
Pixel,
color
r=133
g=114
b=46
p
Pixel,
color
r=134
g=114
b=45
getPixels()
…
value
= 135
06-Intro-Object-Oriented-Prog
48
Get its red value
Set value to the red
value at the new p, then
change the red at that
new pixel.
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
pict
Pixel,
color
r=67
g=131
b=105
Pixel,
color
r=133
g=114
b=46
p
Pixel,
color
r=134
g=114
b=45
getPixels()
…
value
= 133
06-Intro-Object-Oriented-Prog
49
And change this red value
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Change the red value at pixel
p to 50% of value
pict
Pixel,
color
r=67
g=131
b=105
Pixel,
color
r=66
g=114
b=46
p
Pixel,
color
r=134
g=114
b=45
getPixels()
…
value
= 133
06-Intro-Object-Oriented-Prog
50
And eventually, we do all pixels
• We go from this…
to this!
06-Intro-Object-Oriented-Prog
51
“Tracing/Stepping/Walking through”
the program
• What we just did is called “stepping” or “walking
through” the program
– You consider each step of the program, in the order
that the computer would execute it
– You consider what exactly would happen
– You write down what values each variable (name)
has at each point.
• It’s one of the most important debugging skills
you can have.
– And everyone has to do a lot of debugging, especially
at first.
06-Intro-Object-Oriented-Prog
52
Challenges
• Create an increaseRed function
– Start with the decreaseRed funciton
– Modify it to change the red value to 2 * the
original red value
• Create a clearBlue function
– Start with the decreaseRed function
– Modify it to change the blue value to 0
06-Intro-Object-Oriented-Prog
53
How does increaseRed differ from
decreaseRed?
• Well, it does increase rather than
decrease red, but other than that…
– It takes the same parameter input
– It can also work for any picture
• It’s a specification of a process that’ll work for any
picture
• There’s nothing specific
to any particular picture here. Practical programs =
parameterized processes
06-Intro-Object-Oriented-Prog
54
Can we modify more than one value?
• How do we turn this
beach scene into a
sunset?
• What happens at
sunset?
– At first, we tried
increasing the red, but
that made things like
red specks in the sand
REALLY prominent.
• Wrap-around
– New Theory: As06-Intro-Object-Oriented-Prog
the
sun sets, less blue and
55
A Sunset-generation Function
def makeSunset(picture):
for p in getPixels(picture):
value = getBlue(p)
setBlue(p, value * 0.7)
value = getGreen(p)
setGreen(p, value * 0.7)
06-Intro-Object-Oriented-Prog
56
Creating a negative
• Let’s think it through
– R, G, B go from 0 to 255
– Let’s say Red is 10. That’s very light red.
• What’s the opposite? LOTS of Red!
– The negative of that would be 245: 255-10
• So, for each pixel, if we negate each color
component in creating a new color, we
negate the whole picture.
06-Intro-Object-Oriented-Prog
57
Creating a negative
def negative(picture):
for px in getPixels(picture):
red = getRed(px)
green = getGreen(px)
blue = getBlue(px)
negColor = makeColor( 255-red, 255-green, 255-blue)
setColor(px, negColor)
06-Intro-Object-Oriented-Prog
58
Original, negative, double negative
(This gives us a quick way to test our function:
Call it twice and see if the result is equivalent
to the original)
06-Intro-Object-Oriented-Prog
We call this
a lossless transformation. 59
Converting to grayscale
• We know that if red=green=blue, we get gray
– But what value do we set all three to?
• What we need is a value representing the
darkness of the color, the luminance
• There are many ways, but one way that works
reasonably well is dirt simple—simply take the
average:
06-Intro-Object-Oriented-Prog
60
Converting to grayscale
def grayscale(picture):
for p in getPixels(picture):
sum = getRed(p) + getGreen(p) + getBlue(p)
intensity = sum / 3
setColor(p, makeColor(intensity, intensity, intensity))
Does this make
sense?
06-Intro-Object-Oriented-Prog
61
Why can’t we get back again?
• Converting to grayscale is different from
computing a negative.
– A negative transformation retains information.
• With grayscale, we’ve lost information
– We no longer know what the ratios are
between the reds, the greens, and the blues
– We no longer know any particular value.
Media compressions are one kind of transformation.
Some are lossless (like negative);
Others are lossy
(like grayscale)
06-Intro-Object-Oriented-Prog
62
But that’s not really the best grayscale
• In reality, we don’t perceive red, green,
and blue as equal in their amount of
luminance: How bright (or non-bright)
something is.
– We tend to see blue as “darker” and red as
“brighter”
– Even if, physically, the same amount of light is
coming off of each
• Photoshop’s grayscale is very nice: Very
similar to the way that our eye sees it
– B&W TV’s are also pretty good
06-Intro-Object-Oriented-Prog
63
Building a better grayscale
• We’ll weigh red, green, and blue based on
how light we perceive them to be, based on
laboratory experiments.
def grayscaleNew(picture):
for px in getPixels(picture):
newRed = getRed(px) * 0.299
newGreen = getGreen(px) * 0.587
newBlue = getBlue(px) * 0.114
luminance = newRed + newGreen + newBlue
setColor(px, makeColor(luminance, luminance, luminance))
06-Intro-Object-Oriented-Prog
64
Summary
• Name = expression creates a name (a variable)
that has a value
• You can create your own functions in python
– Using def name (params):
• You can execute functions using the function
name and passing in any required values
– decreaseRed(picture)
• You can modify pictures by modifying the pixels
red, green, and blue values
• You can use an array to hold many values of the
same type
• You can loop through all the values in an array
using a for variable in array:
65
• Blocks in Python06-Intro-Object-Oriented-Prog
are shown by indention