Introduction to Computer Science
and Media Computation
Barb Ericson
Georgia Institute of Technology
Oct 2010
01-IntroToMediaComp
1
Learning Goals
•
•
•
•
•
•
•
What is Computer Science?
What is Media Computation?
What is a function?
How do digital pictures work?
How can you manipulate a digital picture?
What is an array?
What is a loop?
01-IntroToMediaComp
2
What is Computer Science?
• The study of process, how to specify what is to
be done and define the stuff being processed.
• You can say that is the study of computational
recipes
– Ones that can be executed on a computer
– A recipe that runs on a computer is called a program
01-IntroToMediaComp
3
What do Computer Scientists do?
• Study if there are better ways to write recipes
– Algorithms – textual description of how to solve a problem
• Study how to structure the data in the recipes
– Data structures and databases
• Determine if there are recipes that can't be written? That
make machines intelligent?
– Theory, artificial intelligence
• Study how to make computers easier for people to use
– Human-computer interface
• Study how computers communicate with each other
– Networking
• Study how to create 3D models and simulations
– Graphics and scientific computing
01-IntroToMediaComp
4
Multimedia CS1 in Python
•
Focus: Learning programming and CS concepts
within the context of media manipulation and
creation
– Converting images to grayscale and
negatives, splicing and reversing sounds,
writing programs to generate HTML, creating
movies out of Web-accessed content.
– Computing for communications, not
calculation
01-IntroToMediaComp
5
We will program in JES
• JES: Jython
Environment for
Students
• A simple editor (for
entering in our
programs or
recipes): the
program area
• A command area
for entering in
commands for
Python to execute.
01-IntroToMediaComp
Editor or Program Area
Command Area
6
Naming parts – declaring variables
• You can name the result from a function call
– And then use the name as input to other functions
myFile = pickAFile() # name the picked file
myPict = makePicture(myFile) # name the picture
show(myPict)
• The value associated with that name is used
= doesn't mean equals here but assign the value for
myFile to the result of pickAFile()
01-IntroToMediaComp
7
Making our own functions
• To make a function, use
the command def
• Then, the name of the
function, and the names
of the input values
between parentheses
(“(input1)”)
• End the line with a colon
(“:”)
• The body of the recipe is
indented (Hint: Use three
spaces)
– That’s called a block
01-IntroToMediaComp
8
Making functions the easy way
• Get something working by typing
commands in the command area (bottom
half of JES)
– Use the up-arrow to bring up a statement
again in the command area
• Enter the def command in the editing
window (top part of JES)
• Copy-paste the tested commands up into
the recipe
01-IntroToMediaComp
9
A recipe for showing a picture
def pickAndShow():
myFile = pickAFile()
myPict = makePicture(myFile)
show(myPict)
Type this in the
program area (editor)
Note: myFile and myPict, inside pickAndShow(), are
completely different from the same names in the command
area. We say that they are in a different scope.
01-IntroToMediaComp
10
Blocking is indicated for you in JES
• Statements that are
indented the same, are
in the same block.
• Statements in the same
block as the cursor are
enclosed in a blue box.
• Type pickAndShow in
the command area and
all statements in the
block will be executed.
01-IntroToMediaComp
11
Image Processing
• Goals:
– Give you the basic understanding of image
processing, including psychophysics of sight,
– Identify some interesting examples to use
01-IntroToMediaComp
12
Light perception
• We perceive light with color sensors that peak
around 425 nm (blue), 550 nm (green), and 560
nm (red).
• Our brain figures out which color is which by
figuring out how much of each kind of sensor is
responding
• One implication: We perceive two kinds of “orange”
— one that’s spectral and one that’s red+yellow
(hits our color sensors just right)
• Dogs and other simpler animals have only two
kinds of sensors
– They do see color. Just less color.
01-IntroToMediaComp
13
Luminance vs. Color
• We perceive borders of
things, motion, depth
via luminance
– Luminance is not the
amount of light, but our
perception of the
amount of light.
– We see blue as “darker”
than red, even if same
amount of light.
– Contrast also plays a
role
Luminance is actually
color blind. Completely
different part of the brain.
01-IntroToMediaComp
14
Digitizing pictures
• We digitize pictures into lots of little dots
• Enough dots and it looks like a continuous
whole to our eye
– Our eye has limited resolution
– Our background/depth acuity is particularly low
• Each picture element is referred to as a pixel
• Pixels are picture elements
– Each pixel object knows its color
– It also knows where it is in its picture
01-IntroToMediaComp
15
Exploring Pictures
>>> file = pickAFile()
>>> beachPict = makePicture(file)
>>> explore(beachPict)
• Zoom in to see individual
pixels
• Move the cursor to see the x
and y values
• Look at the red, green, and
blue values
01-IntroToMediaComp
16
Encoding color
• Each pixel encodes color at that position in the
picture
• Lots of encodings for color
– Printers use CMYK: Cyan, Magenta, Yellow, and
blacK.
– Others use HSB for Hue, Saturation, and Brightness
(also called HSV for Hue, Saturation, and Value.
• We’ll use the most common for computers
– RGB: Red, Green, Blue
01-IntroToMediaComp
17
Encoding Color: RGB
• In RGB, each color has
three component colors:
– Amount of redness
– Amount of greenness
– Amount of blueness
• Each does appear as a
separate dot on most
devices, but our eye blends
them.
• In most computer-based
models of RGB, a single
byte (8 bits) is used for
each
– So a complete RGB color is
24 bits, 8 bits of each
01-IntroToMediaComp
18
Making Colors with Light
• Type
>>> myColor = pickAColor()
• Try to create
–
–
–
–
–
–
White
Black
Yellow
Red
Brown
Purple
01-IntroToMediaComp
19
Basic Picture Functions
• makePicture(filename) creates and returns a
picture object, from the JPEG file at the filename
pict = makePicture("c:/ip-book/mediasources/barbara.jpg")
• show(picture) displays a picture in a window
show(pict)
• explore(picture) makes a copy of the picture and
shows it in the explorer window
explore(pict)
• We’ll learn functions for manipulating pictures
like getColor, setColor, and repaint
01-IntroToMediaComp
20
Making our own functions
• To make a function, use the
command def
• Then, the name of the function,
and the names of the input
values between parentheses
(“(input1)”)
• End the line with a colon (“:”)
• The body of the recipe is
indented (Hint: Use two
spaces)
• Your function does
NOT exist for JES
until you load it
01-IntroToMediaComp
21
Saving Functions in Files
• Click on File and then Save Program
– Name it with some file name and .py at end
• You can define more than one function in a file
– Maybe call these pictureFunctions.py
• You can later open these files up
– And use the Load Program button to load all functions
in the file
• You can build a library of python functions for
working with pictures, sounds, movies, etc
01-IntroToMediaComp
22
Modifying a Pixel Color
• You can get the amount of red, green, or blue
– redValue = getRed(pict)
– greenValue = getGreen(pict)
– blueValue = getBlue(pict)
• You can change the amount of red, green, or
blue
– setRed(pict,value)
– setGreen(pict,value)
– setBlue(pictValue)
01-IntroToMediaComp
23
Modifying Colors
• You can also get the color from a pixel
– myColor = getColor(pixel)
• You can create a new color by giving values for
red, green, and blue from 0 to 255
– newColor = makeColor(255,0,0)
• You can set a color using
– setColor(pixel,newColor)
01-IntroToMediaComp
24
How to change lots of pixels?
• If we want to change all the pixels in a picture
how can we do that?
– In a 640 x 480 picture that is 307,200 pixels
• Computers are very fast and can process billions
of instructions per second
– But we wouldn't want to name each pixel or modify
the color on each one by typing in the commands
307,200 times
• We can get an array of pixels to process
– Using getPixels(picture) and loop through the pixels
one at a time
01-IntroToMediaComp
25
What is an array?
• Space in memory for many values of the same type
– Numbers, pictures, pixels, etc
• You can refer to the elements of an array using an index
– Starting with 0 for the first element
– And length – 1 for the last element
• You can also loop through all elements of an array using
a forEach loop
01-IntroToMediaComp
26
Our first picture recipe
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Used like this:
>>> file="c:/ip-book/mediasources/katie.jpg"
>>> picture=makePicture(file)
>>> explore(picture)
>>> decreaseRed(picture)
>>> explore(picture)
01-IntroToMediaComp
27
How do you make an omelet?
• Something to do with eggs…
• What do you do with each of the eggs?
• And then what do you do?
All useful recipes involve repetition
- Take four eggs and crack them….
- Beat the eggs until…
We need these repetition (“iteration”) constructs in
computer algorithms too
- Today we will introduce one of them
01-IntroToMediaComp
28
Decreasing the red in a picture
• Recipe: To decrease the red
• Ingredients: One picture, name it picture and pass it to
the function where we will call it pict
• Step 1: Get all the pixels of pict. For each pixel p in the
set of pixels…
• Step 2: Get the value of the red of pixel p, and set it to
50% of its original value
01-IntroToMediaComp
29
More about for each loops
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
• for is the name of the command
• An index variable is used to hold each of the different
values of a sequence
• The word in
• A function that generates a sequence
– The index variable will be the name for one value in the
sequence, each time through the loop
• A colon (“:”)
• And a block (the indented lines of code)
01-IntroToMediaComp
30
What happens when a for loop is executed
• The index variable is set to an item in the
sequence
• The block is executed
– The variable is usually used inside the block
• Then execution loops to the for statement,
where the index variable gets set to the next
item in the sequence
• Repeat until every value in the sequence is
used.
01-IntroToMediaComp
31
getPixels returns a sequence of pixels
• Each pixel knows
its color and place
def decreaseRed(pict):
in the original
for p in getPixels(pict):
value=getRed(p)
picture
setRed(p,value*0.5)
• Change the pixel
and you change
the picture
• So the loops here
assign the index
variable p to each
pixel in the picture
picture, one at a
time.
01-IntroToMediaComp
32
Do we need the variable value?
• No
– We can calculate the original red amount right when we
are ready to change it.
– It’s a matter of programming style. The meanings are
the same.
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
def decreaseRed(pict):
for p in getPixels(pict):
setRed(p, getRed(p) * 0.5)
01-IntroToMediaComp
33
Let’s walk that through slowly…
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Here we take a picture
object in as a parameter
to the function and call it
pict
pict
01-IntroToMediaComp
34
Now, get the pixels
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
We get all the pixels from
the picture, then make p
be the name of each one
one at a time
pict
Pixel,
color
r=135
g=131
b=105
p
Pixel,
color
r=133
g=114
b=46
Pixel,
color
r=134
g=114
b=45
getPixels()
…
01-IntroToMediaComp
35
Get the red value from pixel
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
We get the red value of
pixel p and name it value
pict
value = 135
Pixel,
color
r=135
g=131
b=105
p
Pixel,
color
r=133
g=114
b=46
Pixel,
color
r=134
g=114
b=45
getPixels()
…
01-IntroToMediaComp
36
Now change the pixel
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Set the red value of pixel
p to 0.5 (50%) of value
value = 135
Pixel,
color
r=67
g=131
b=105
p
pict
Pixel,
color
r=133
g=114
b=46
Pixel,
color
r=134
g=114
b=45
getPixels()
…
01-IntroToMediaComp
37
Then move on to the next pixel
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Move on to the next pixel
and name it p
value = 135
Pixel,
color
r=67
g=131
b=105
pict
Pixel,
color
r=133
g=114
b=46
p
Pixel,
color
r=134
g=114
b=45
getPixels()
…
01-IntroToMediaComp
38
Get its red value
Set value to the red
value at the new p, then
change the red at that
new pixel.
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
value = 133
Pixel,
color
r=67
g=131
b=105
pict
Pixel,
color
r=133
g=114
b=46
p
Pixel,
color
r=134
g=114
b=45
getPixels()
…
01-IntroToMediaComp
39
And change this red value
def decreaseRed(pict):
for p in getPixels(pict):
value=getRed(p)
setRed(p,value*0.5)
Change the red value at pixel
p to 50% of value
pict
value = 133
Pixel,
color
r=67
g=131
b=105
Pixel,
color
r=66
g=114
b=46
p
Pixel,
color
r=134
g=114
b=45
getPixels()
…
01-IntroToMediaComp
40
And eventually, we do all pixels
• We go from this…
to this!
01-IntroToMediaComp
41
Tracing/Stepping/Walking
• What we just did is called “stepping” or “walking
through” the program
– You consider each step of the program, in the order
that the computer would execute it
– You consider what exactly would happen
– You write down what values each variable (name)
has at each point.
• It’s one of the most important debugging skills
you can have.
– And everyone has to do a lot of debugging, especially
at first.
01-IntroToMediaComp
42
Challenges
• Create an increaseRed function
– Copy the decreaseRed function and rename it
– Modify it to change the red value to 2 * the original
red value
• Create a clearBlue function
– Copy the decreaseRed function and rename it
– Modify it to change the blue value to 0
01-IntroToMediaComp
43
Can we modify more than one value?
• How do we turn this
beach scene into a
sunset?
• What happens at sunset?
– At first, we tried
increasing the red, but
that didn't work very
well
– New Theory: As the
sun sets, less blue and
green is visible, which
makes things look
more red.
01-IntroToMediaComp
44
A Sunset-generation Function
def makeSunset(picture):
for p in getPixels(picture):
value = getBlue(p)
setBlue(p, value * 0.7)
value = getGreen(p)
setGreen(p, value * 0.7)
01-IntroToMediaComp
45
Creating a negative
• Let’s think it through
– R, G, B go from 0 to 255
– Let’s say Red is 10. That’s very light red.
• What’s the opposite? LOTS of Red!
– The negative of that would be 245: 255-10
• So, for each pixel, if we negate each color
component in creating a new color, we
negate the whole picture.
01-IntroToMediaComp
46
Creating a negative
def negative(picture):
for px in getPixels(picture):
red = getRed(px)
green = getGreen(px)
blue = getBlue(px)
negColor = makeColor( 255-red, 255-green, 255-blue)
setColor(px, negColor)
01-IntroToMediaComp
47
Original, negative, double negative
(This gives us a quick way to test our function:
Call it twice and see if the result is equivalent
to the original)
We call this a lossless transformation.
01-IntroToMediaComp
48
Converting to grayscale
• We know that if red=green=blue, we get gray
– But what value do we set all three to?
• What we need is a value representing the
darkness of the color, the luminance
• There are many ways, but one way that works
reasonably well is dirt simple—simply take the
average:
01-IntroToMediaComp
49
Converting to grayscale
def grayscale(picture):
for p in getPixels(picture):
sum = getRed(p) + getGreen(p) + getBlue(p)
intensity = sum / 3
setColor(p, makeColor(intensity, intensity, intensity))
Does this make
sense?
01-IntroToMediaComp
50
Why can’t we get back again?
• Converting to grayscale is different from
computing a negative.
– A negative transformation retains information.
• With grayscale, we’ve lost information
– We no longer know what the ratios are
between the reds, the greens, and the blues
– We no longer know any particular value.
Media compressions are one kind of transformation.
Some are lossless (like negative);
Others are lossy (like grayscale)
01-IntroToMediaComp
51
But that’s not really the best grayscale
• In reality, we don’t perceive red, green,
and blue as equal in their amount of
luminance: How bright (or non-bright)
something is.
– We tend to see blue as “darker” and red as
“brighter”
– Even if, physically, the same amount of light is
coming off of each
• Photoshop’s grayscale is very nice: Very
similar to the way that our eye sees it
– B&W TV’s are also pretty good
01-IntroToMediaComp
52
Building a better grayscale
• We’ll weigh red, green, and blue based on
how light we perceive them to be, based
on laboratory experiments.
def grayscaleNew(picture):
for px in getPixels(picture):
newRed = getRed(px) * 0.299
newGreen = getGreen(px) * 0.587
newBlue = getBlue(px) * 0.114
luminance = newRed + newGreen + newBlue
setColor(px, makeColor(luminance, luminance, luminance))
01-IntroToMediaComp
53
Summary
• Name = expression creates a name (a variable)
that has a value
• You can create your own functions in python
• You can execute functions using the function
name and passing in any required values
– decreaseRed(picture)
• You can modify pictures by modifying the pixels
red, green, and blue values
• You can use an array to hold many values of the
same type
• You can loop through all the values in an array
using a for variable in array:
• Blocks in Python are shown by indention
01-IntroToMediaComp
54