Media computation
as a context for
learning computing
Mark Guzdial
College of Computing/GVU
Georgia Institute of Technology
Story


Perlis’ challenge: Computer science is more
important than calculus
Evidence that we’re not there yet



And what the solution might look like
Our attempt: Introduction to Media Computation
How this might be used/useful elsewhere
Computer science is more
important than Calculus

In 1961, Alan Perlis argued
that computer science is more
important in a liberal
education than calculus



Explicitly, he argued that
all students should learn to
program.
Calculus is about rates, and
that’s important to many.
Computer science is about
process, which is important to
everyone
How close are we to being
able to teach everyone CS?

Not very


At many departments, CS retention rates are lower than the
rest of campus



CS1 is one of the most despised courses for non-majors
At Georgia Tech: 65% for 1995 cohort, vs. 73% for
Engineeering
Drop-out rates near 50% at many institutions
Female enrollment in CS has been dropping nationally
Why?

Several recent studies and books
claim that CS instruction tends to
dissuade anyone but white males

“Tedious,”
“taught without application
relevance,”
“boring,”
“lacking creativity,”
“asocial”
The potential of computing


How can we realize the potential impact of
computing if future professionals in other
disciplines despise computer science?
To realize that potential, we need practitioners in
other disciplines to understand computing.

At least some of those practitioners need to
understand it as a creative, exciting venture—as we
do!
The best uses for computing
technologies will come from others



Thomas Edison vs. D.W. Griffith
If we want computing technologies to become
useful, they have to get out of our hands.
It can’t be just through applications


That presumes that the current technologists
know how everyone else wants to do things and
should do things
Suggestion: D.W. Griffith knew things that Edison
didn’t.
The Challenges


We need to motivate CS, potential CS, and non-CS
students to care about computing
We need to make it
social,
creative,
relevant,
exciting, and
not tedious

Which is how many of us already see Computing, but
that’s not getting communicated
Intro Computing at Georgia Tech

CS1321 “Introduction to Computing”




Required of every student at Georgia Tech
Uses Scheme and How to Design Programs
Followed up by CS1322 “Introduction to ObjectOriented Programming” in Java
COE 1361 “Computing for Engineers”



Engineering problem-solving in MATLAB
Still in pilot stage: 75-90 students a term
800 new Engineering Freshmen a year enter Tech
Our Attempt: Introduction to
Media Computation

A course for non-CS and non-Engineering majors


120 students this semester,
planning 400-600 in the Fall


International Affairs, Literature, Public Policy,
Architecture, Management, Biology, etc.
2/3 female in this semester’s CS1315
Focus: Learning programming within the context of
media manipulation and creation
Motivating the Computing
As professionals, these students will often
the use the computer as a communications
medium.
 All media are going digital,
and digital media are manipulated with
software.
 Knowing how to program, then, is a
communications skill.

Programming as a
Communications Skill

Knowing how to program means to
understand one’s tools.
 Maybe
means can transfer tool skills more
easily
 Students tell us that they’re excited to
learn how PhotoShop works.

And it means that, if you have to, you may
be able to grow your own
Programming as
Communicating Process
A program is a succinct, executable process
description
 That makes valuable for explaining process

 We
use examples from Biology (e.g.,
explaining DNA transcription) and
Management (e.g., simulations) to make
this point
Python as the programming
language


Huge and contentious issue
Use in commercial contexts legitimatizes the
choice



Minimal syntax
Looks like other programming languages


Industrial Light & Magic, Google, Nextel, etc.
Potential for knowledge transfer
Actually using Jython (http://www.jython.org) for
Java class libraries
How the class was developed

Created in response to “recent unpleasantness”


Developed with an advisory board with
representatives from across campus


On-line surveys, meetings with students
Architecture, Math, Electrical and Computer
Engineering, Literature, Center for Enhancement of
Teaching and Learning
Trialed in faculty workshop in mid-December
Course Objectives

Students will be able to read, understand, and
modify programs that achieve useful
communication tasks



Not programming from a blank piece of paper
Students will learn what computer science is about,
especially data representations, algorithms,
encodings, forms of programming.
Students will learn useful computing skills,
including graphing and database concepts
Data structures

Some data structure discussions come naturally




Sounds are in arrays
Pictures are matrices of pixels, each of which has
red, green, and blue components
Files are stored in trees
Much of our discussion of data structures comes
in the form of encodings comparisons


RGB vs. CMYK for encoding color
22.1 Khz vs. 44.2 Khz for audio sampling rates
def clearRed(picture):
for pixel in getPixels(picture):
setRed(pixel,0)
def greyscale(picture):
for p in getPixels(picture):
redness=getRed(p)
greenness=getGreen(p)
blueness=getBlue(p)
luminance=(redness+blueness+greenness)/3
setColor(p,
makeColor(luminance,luminance,luminance))
def negative(picture):
for px in getPixels(picture):
red=getRed(px)
green=getGreen(px)
blue=getBlue(px)
negColor=makeColor(255-red,255-green,255-blue)
setColor(px,negColor)
def chromakey(source,bg):
for x in range(1,getWidth(source)):
for y in range(1,getHeight(source)):
p = getPixel(source,x,y)
# My definition of blue: If the redness + greenness < blueness
if (getRed(p) + getGreen(p) < getBlue(p)):
#Then, grab the color at the same spot from the new background
setColor(p,getColor(getPixel(bg,x,y)))
return source
Use a loop!
Our first picture recipe
def decreaseRed(picture):
for p in getPixels(picture):
value=getRed(p)
setRed(p,value*0.5)
original
Used like this:
>>> file="/Users/guzdial/mediasources/barbara.jpg"
>>> picture=makePicture(file)
>>> show(picture)
>>> decreaseRed(picture)
>>> repaint(picture)
Recipe to Increase the Volume
def increaseVolume(sound):
for sample in getSamples(sound):
value = getSample(sample)
setSample(sample,value * 2)
Using it:
>>> f="/Users/guzdial/mediasources/gettysburg10.wav"
>>> s=makeSound(f)
>>> increaseVolume(s)
>>> play(s)
>>> writeSoundTo(s,"/Users/guzdial/mediasources/louder-g10.wav")
A Sunset-generating function


How do we turn this
beach scene into a sunset?
What happens at sunset?


Tried increasing the red,
but that failed.
New Theory: As the sun
sets, less blue and green is
visible, which makes
things look more red.
A Sunset-generation Function
def makeSunset(picture):
for p in getPixels(picture):
value=getBlue(p)
setBlue(p,value*0.7)
value=getGreen(p)
setGreen(p,value*0.7)
Introducing IF:
Making Barb a redhead
def turnRed():
brown = makeColor(57,16,8)
file = r"C:\Documents and Settings\Mark Guzdial\My
Documents\mediasources\barbara.jpg"
picture=makePicture(file)
for px in getPixels(picture):
color = getColor(px)
if distance(color,brown)<50.0:
redness=getRed(px)*1.5
setRed(px,redness)
show(picture)
return(picture)
Original:
Generalizing Algorithms


We talk about algorithm complexity later in the
course, after the media is done.
We talk about different approaches to the same
problem, where the criteria might be aesthetics or
correctness, instead of speed or size


For example, generating greyscale
During the media, we point out similar themes in
different functions.

We refer to them as “sub-recipes”
Scaling the picture down
def copyBarbsFaceSmaller():
# Set up the source and target pictures
barbf=getMediaPath("barbara.jpg")
barb = makePicture(barbf)
canvasf = getMediaPath("7inX95in.jpg")
canvas = makePicture(canvasf)
# Now, do the actual copying
sourceX = 45
for targetX in range(100,100+((200-45)/2)):
sourceY = 25
for targetY in range(100,100+((200-25)/2)):
color = getColor(getPixel(barb,sourceX,sourceY))
setColor(getPixel(canvas,targetX,targetY), color)
sourceY = sourceY + 2
sourceX = sourceX + 2
show(barb)
show(canvas)
return canvas
Scaling the picture up
def copyBarbsFaceLarger():
# Set up the source and target pictures
barbf=getMediaPath("barbara.jpg")
barb = makePicture(barbf)
canvasf = getMediaPath("7inX95in.jpg")
canvas = makePicture(canvasf)
# Now, do the actual copying
sourceX = 45
for targetX in range(100,100+((200-45)*2)):
sourceY = 25
for targetY in range(100,100+((200-25)*2)):
color = getColor(getPixel(barb,int(sourceX),int(sourceY)))
setColor(getPixel(canvas,targetX,targetY), color)
sourceY = sourceY + 0.5
sourceX = sourceX + 0.5
show(barb)
show(canvas)
return canvas
Recipe for halving the
frequency of a sound
def half(filename):
source = makeSound(filename)
target = makeSound(filename)
This is how a
sampling synthesizer
works!
sourceIndex = 1
for targetIndex in range(1, getLength( target)+1):
setSampleValueAt( target, targetIndex,
getSampleValueAt( source, int(sourceIndex)))
sourceIndex = sourceIndex + 0.5
Here’s the
play(target)
return target
piece that
does it
Compare these two
def copyBarbsFaceLarger():
# Set up the source and target pictures
barbf=getMediaPath("barbara.jpg")
barb = makePicture(barbf)
def half(filename):
canvasf = getMediaPath("7inX95in.jpg")
source = makeSound(filename)
canvas = makePicture(canvasf)
target = makeSound(filename)
# Now, do the actual copying
sourceX = 45
for targetX in range(100,100+((200-45)*2)):
sourceIndex = 1
for targetIndex in range(1, getLength( target)+1): sourceY = 25
for targetY in range(100,100+((200-25)*2)):
setSampleValueAt( target, targetIndex,
color = getColor(
getSampleValueAt( source,
getPixel(barb,int(sourceX),int(sourceY)))
int(sourceIndex)))
setColor(getPixel(canvas,targetX,targetY), color)
sourceIndex = sourceIndex + 0.5
sourceY = sourceY + 0.5
sourceX = sourceX + 0.5
play(target)
show(barb)
show(canvas)
return target
return canvas
Both of them are sampling

Both of them have three parts:


A start where objects are set up
A loop where samples or pixels are copied from one
place to another
To decrease the frequency or the size, we take each
sample/pixel twice
 In both cases, we do that by incrementing the index by
0.5 and taking the integer of the index


Finishing up and returning the result
Using your personal pictures
And messin’ with them
Data-first Computing


Real users come to a user with data that they care about,
then they (unwillingly) learn the computer to manipulate
their data as they need.
CS1315 works the same.



We use pictures of students in class demonstrations.
Students do use their own pictures as starting points for
manipulations.
They started doing this in the second week


How often do students use their second week of CS1 on their own
data?
How does that change the students’ relationship to the material?
Rough overview of Syllabus


Defining and executing functions
Pictures


Sounds



Psychophysics, data structures, defining functions,
for loops, if conditionals
Text


Psychophysics, data structures, defining functions,
for loops, if conditionals
Converting between media, generating HTML,
“flattening” media and saving to a database
Movies
Then, Computer Science
Computer science as a solution
to their problems

“Writing programs is hard! Are there ways to
make it easier? Or at least shorter?”



“Movie-manipulating programs take a long time
to execute. Why?”


Object-oriented programming
Functional programming and recursion
Algorithmic complexity
“Why is PhotoShop so much faster?”


Compiling vs. interpreting
Machine language and how the computer works
Assignments encourage
collaboration
Homework are all collaborative
 Quizzes are preceded by nearly-identical,
collaborative pre-quizzes
 Two “take-home exams” (programming
assignments) are non-collaborative
 “Lablets” on application software are
collaborative

Assignments encourage
creativity

For several homeworks, the task is to manipulate
media in some way, but we don’t care what media



For example, creating a collage or building an
animation
Encouraging homework results to be posted to
CoWeb in galleries
We’ve purchased webcams to loan to students to
create their own media
First Homework assignment
(Due last Friday)
Homework 1:
Write a program named hw1 to accept a picture as input,
and change its pixels as follows:
•
•
•
Set the green component to 125% of its current value
Decrease the blue by 25%
Decrease the red by 75%
Solutions shared in the CoWeb
Grade distribution
Much better than
anticipated.
Tools to support media
computation
MediaTools for exploring media
JES: Jython environment for students
Assessing the effort

We don’t know if this will work

Will students learn?


Will they learn similarly to the other CS1 classes?
Will the class change their attitude toward CS?
Hopefully, something better than what we’ve been
seeing
 Will the differences appear among the women,
specifically?


If there are problems in student performance,
where are they?
Assessment measures

Comparing all of our CS1 classes in terms of
learning and motivation, broken out by gender
and major


Observational study of student performance to
understand problems and strategies


Surveys: Initial, midterm, and final
Watched two groups working on HW1
Interview study of impact on women
How can this be useful to you?

We’d like to find partners in developing and
trialing the course as a non-majors programming
course.



We have software, draft of book, and slides in
development.
We’d like to develop something that works more
than just at Tech.
We have Java classes to offer to majors courses
that implement most of the media interface.

Media focus may be motivating to our majors, and
may help with retention issues.
public greyScale(String filename) throws java.io.IOException // parms: string
defining the file path
{
JavaPicture src = new JavaPicture();
src.setFilename(filename);
src.loadImage();
// System.out.println("Height = " + src.getHeight() );
// System.out.println("Width = " + src.getWidth() );
for (int x=1; x < src.getHeight();x++)
{
for (int y=1; y < src.getWidth();y++)
{
JavaPixel p = src.getPixel(y,x);
int nC = (p.getRed()+p.getGreen()+p.getBlue() )/ 3;
p.setBlue(nC);
p.setRed(nC);
p.setGreen(nC);
}
}
src.saveImage("C:\\grey.jpg");
}
Summary

Perlis’ challenge suggests that CS is more important than
Calculus




But need to update our pedagogy to make it happen
Media Computation may be a useful context to motivate
student performance
Our class is aimed at addressing the challenges we’ve
identified, and we’re trying it this semester
The approach right now is aimed at non-majors, but
certainly could be used with majors, too.
Acknowledgements



Course materials development: Jason Ergle, Claire Bailey,
David Raines, Joshua Sklare, Adam Wilson, Andrea
Forte, Mark Richman, Matt Wallace, Alisa Bandlow.
Assessment: Andrea Forte, Rachel Fithian, Lauren Rich
Thanks to Bob McMath and the Al West Fund, to GVU
and CoC, and the National Science Foundation
For further information



Course CoWeb:
http://coweb.cc.gatech.edu/cs1315
Where we planned the course:
http://coweb.cc.gatech.edu/mediaComp-plan
guzdial@cc.gatech.edu