-Please grab a CD.
There are USB drives to
borrow – copy the folder
“MediaComp-CambridgeJune2010”
- Grab a seat in the back
with the flip-up tables.
Power is in the floor.
- We have a coffee break
at 10:15 with food. (The
catering folks assumed
you’d have breakfast at
the hotel. We will have
breakfast at Marriott
Thurs.)
- For WiFi, join
“Cambridge” network,
and use event code
sw616.
Mark Guzdial
College of Computing
Georgia Institute of Technology
guzdial@cc.gatech.edu
http://www.georgiacomputes.org
http://www.mediacomputation.org
Introductions
 Who are you?
 Where are you from?
 What do you teach?
 What do you want to get out of this workshop?
Workshop Plan:
Python on Wednesday
 9-9:30: Welcome and Introductions.
 9:30: Introduction to manipulation of images with Media Computation







in Python.
10:30-10:45: Coffee Break
11: Cropping and Scaling of Images
12-1: Lunch break (and play with images)
1-2: Introducing sound, sound manipulations, splicing sounds.
2-3: Integrating Media Computation with Alice
3-3:15: Coffee Break.
3:15-4:30. Try manipulating images and sounds, and carrying them in or
out of Alice.
Workshop Plan: Java on
Thursday
 9-10: Research results – what do we know about use of Media








Computation?
10-11:00: Mapping this all to Java. Manipulating images and sounds in
Java.
10:30-10:45: Coffee Break
11-12: Introducing objects with turtles and MIDI
12-1: Lunch
1-2: Making movies and doing digital video special effects.
2-3:15. Try movies and MIDI yourself.
3:15-3:30: Coffee Break.
3:30-4:30: Group Discussion: Which of these approaches could you use
in your classes?
How could I use any of this in
my courses?
 Trying to use MIDI even if can’t record their own voices.
 Can also bring in recordings on USB stick.
 Can use Python instead of JavaScript for simple introduction to
variables and looping and similar ideas.
 Using Media to motivate learning.
 Great way to use nested for-loops (for pictures and movies).
 Using for intro-to-programming for non-majors. May also for art
department’s Photoshop “How to manipulate images” class. Can we do
gened real art and real computing.
 You don’t have to have 1:1 art:computing students.
 Teaching in a design studio setting, so students are working and seeing
one-another’s work.
More using it in my classroom
 Can use a Wiki to share ideas, when don’t have a physical design studio.
 Feeling much less depressed teaching a more fun version of intro to
Java.
 Use pictures and sounds as data in more traditional assignments.
 Adding one or two projects can be too much overhead (learning the
libraries) for too little benefit.
 With Python, enough details are hidden so that you could do a LITTLE bit
of media.
 Can do a little Alice, as an extension – even without a book, even not
“officially.”
 Not enough exercises, not enough of the little bitty ones. “What does
this for loop do?” and “Trace this code…”
More on use of classroom
 Yeah, but am I going to give them “real” examples, too?
 What can’t we cover here that we might want to cover?
 Sorting. Engineering computations (e.g., do the area under the curve computation).
 Queen’s problem, chessboard setups.
 “Real” arrays. Moving things around in arrays.
 Writing complete applications, start-to-finish without turtles or multimedia.
 File processing is there, but hard to adapt to a CS0 class.
 Strings, lists, file I/O, networking etc. – radically different from the media stuff.
 Can students deal with the rest of the class after starting with media? The goal
of CS1 may be to improve the “survival” rate. Getting them excited is what
keeps them going.
 May result in better grasp of what they do cover.
Workshop Plan: Data structures
on Friday
 9-10: Introducing linked lists with MIDI
 10-11: Linked lists and trees of images.
 11-12: Try making a MIDI song or a linked list animation.
 12-1: Lunch.
 1-2: Making simulations in Greenfoot
 2-3: Making our own simulation package.
 3-3:15: Coffee break.
 3:15-4:30: Choice:
 Discrete event simulations (to introduce stacks and queues), or
 Exercise: Make simulations in Greenfoot or the MediaComp Data
Structures simulation package.
What’s on my
CD (or USB
stick)?
Overview of the books
 Introduction to Computing and
Programming in Python: A Multimedia
Approach, 2ed
 Introduction to Computing and
Programming in Java: A Multimedia
Approach
 Exploring Wonderland: Java with Alice
and Media Computation
 Problem Solving with Data Structures: A
Multimedia Approach
Detailed Overview Follows…
Multi-media CS 1 in Python
 Average 300/term
 Overall, CS1315 has been 51% female
 Required in Architecture, Management, Ivan Allen
College of Liberal Arts, and Biology
 Focus: Learning programming and CS
concepts within the context of media
manipulation and creation
 Converting images to grayscale and negatives, splicing
and reversing sounds, writing programs to generate
HTML, creating movies out of Web-accessed content.
 Computing for communications, not calculation
Using Python
 Huge issue
 Use in commercial contexts
authenticates the choice

IL&M, Google, Nextel, etc.
 Minimal syntax
 Looks like other programming
languages

Potential for transfer
Rough overview of Syllabus
 Defining and executing functions
 Pictures
 Psychophysics, data structures, defining functions, for loops, if
conditionals
 Bitmap vs. vector notations
 Sounds
 Psychophysics, data structures, defining functions, for loops, if
conditionals
 Sampled sounds vs. synthesized, MP3 vs. MIDI
 Text
 Converting between media, generating HTML, database, and
networking
 A little trees (directories) and hash tables (database)
 Movies
 Then, Computer Science topics (last 1/3 class)
def clearRed(picture):
for pixel in getPixels(picture):
setRed(pixel,0)
def greyscale(picture):
for p in getPixels(picture):
redness=getRed(p)
greenness=getGreen(p)
blueness=getBlue(p)
luminance=(redness+blueness+greenness)/3
setColor(p, makeColor(luminance,luminance,luminance))
def negative(picture):
for px in getPixels(picture):
red=getRed(px)
green=getGreen(px)
blue=getBlue(px)
negColor=makeColor(255-red,255-green,255-blue)
setColor(px,negColor)
Open-ended, contextualized homework in Media Computation CS1
Sound collage
16
Some Computer Science Topics inter-mixed
 We talk about algorithms across
media


Sampling a picture (to scale it) is the
same
algorithm as sampling a sound (to
shift frequency)
Blending two pictures (fading one
into the other) and two sounds is the
same algorithm.
 We talk about representations and
mappings (Goedel)

From samples to numbers (and into
Excel), through a mapping to pixel
colors
 We talk about design and
debugging

But they mostly don’t hear us
Computer Science Topics
as solutions to their problems
 “Why is PhotoShop so much faster?”
 Compiling vs. interpreting
 Machine language and how the computer works
 “Writing programs is hard! Are there ways to make it
easier? Or at least shorter?”
 Object-oriented programming
 Functional programming and recursion
 “Movie-manipulating programs take a long time to execute.
Why? How fast/slow can programs be?”
 Algorithmic complexity
Multimedia CS1 in Java
 Originally developed for AP CS
teachers in Georgia.

State of Georgia wanted to train Business teachers to
become computer science teachers.

Needed an approach to learning CS1 that was
successful with business and liberal arts students
Syllabus
 Introduction to Java
 Math operators, printing results, data types, casting,
relational operators, Strings, variables
 Introduction to Programming
 Creating and naming objects

Using a turtle and a world
 Creating new Turtle methods


Draw simple shapes
Using parameters
Syllabus - Continued
 Modifying Pictures using
Loops
 One-dimensional arrays
 Use for-each, while, and
for loops to
 Increase/decrease
colors, fake a sunset,
lighten and darken,
create a negative, and
grayscale
Syllabus - Continued
 Modifying Pixels in a
Matrix
 Two-dimensional arrays
 Nested loops
 Copying, mirroring,
blending, rotation,
scaling
Syllabus - Continued
 Conditionally Modifying
Pixels
 Boolean expressions
 Using && and ||
 Replacing a color,
reducing red-eye, edge
detection, sepia-toned,
posterize, highlight
extremes, blurring,
background
subtractions,
chromakey
Syllabus - Continued
 Drawing on Pictures
 Using existing Java
classes
 Inheritance
 Interfaces
 Drawing simple shapes,
drawing text, general
copy, general scale,
shearing, gradient paint,
general blending,
clipping
Syllabus - Continued
 Modifying all Samples in
a Sound
 1D arrays
 Loops
 Conditional execution
 Change volume,
normalizing a sound
(make it as loud as
possible), force to
extremes
Syllabus - Continued
 Modifying Samples
using Ranges
 Loops
 Clipping, splicing,
reversing, mirroring
Syllabus - Continued
 Combining and Creating
Sounds
 Class and private
methods
 Composing sounds,
blending sounds,
changing frequencies,
and creating echoes
 Creating sounds

Sine Waves, Square
Waves, Triangle Waves
 MP3 and MIDI
Syllabus - Continued
 Creating Classes
 Identifying objects and classes
 Defining a class
 Overloading constructors
 Creating and initializing an array
 Creating getters and setters
 Creating a main method
 Javadoc comments
 Reusing a class via inheritance
 ConfusedTurtle
Turtle ConfusedTurtle
Syllabus - Continued
 Creating and Modifying Text
 String methods
 Reading from and writing to files

Handling Exceptions
 Creating a form letter
 Modifying programs
 Getting text from networks
 Creating random sentences
 Using text to shift between
media
Syllabus - Continued
 Making Text for the Web
 Throwing exceptions,





“unnamed” package,
HashMap, Generics, and
Iterators
Generating HTML
Create a web page from a
directory
Create a web page from other
web pages
Databases
Creating a web page from a
database
Syllabus - Continued
 Encoding, Manipulating,
and Creating Movies
 Frame-based animations
with simple shapes and text
 Special effects – fade out,
fake sunset, and chromakey
Syllabus - Continued
 Speed
 What makes programs fast?




Compilers and Interpreters
Writing a graphics interpreter
and compiler
Searching
Algorithms that can’t be
written
 What makes computers fast?

Clock rates, Storage, Display
Syllabus - Continued
 Javascript
 Syntax
 User Interfaces
 Multimedia
Alice and Media Comp in Java
 Originally suggested by Randy
Pausch

In going from Alice to Java, “I’d use
Media Computation.”
 Java Media Computation as the
“special effects studio” for Alice
Intro to Objects, Classes, and Methods
 Create objects in Alice and
invoke methods on them
 Create a new method in
Alice
 And pass in a parameter
 Create objects in Java and
invoke methods on them
 Create a new method in
Java
 And pass in a parameter
35
Alice and
Media
Computati
on Intro
Sound Projects
 Create sound clips or splice sounds together to
create a soundtrack for your Alice movie
 Normalize all sounds recorded for character
dialog.
 Manipulate the sounds for an underwater movie to
sound as if they were produced under water.
 Blend two sounds together to add sound effects to
your sound track.
 Change the frequency of a sound and use the new
sounds for different Alice objects
Alice and Media Computation Intro
36
Sound Frequency Project
 Create an Alice world with
3 frogs
 Resize one to be bigger and
one to be smaller
 Using croak.wav
 Create a higher sound

Take every other sample
 Create a lower sound

Take every sample twice
 Import the sound files and
have the frogs play their
sounds and tap their foot
37
Alice and
Media
Computati
on Intro
Picture Projects
 Negate an Alice picture
 Darken or lighten an Alice picture
 Fade out a picture
 Change the colors on an Alice picture
 Make it look like sunset
 Mirror an Alice picture
 Make it look like a reflection in the water
 Do chromakey
 To put Alice objects on a normal photo
 Or a person in an Alice world
Alice and Media Computation Intro
38
Using Chromakey
 Create objects in Alice and
position them as desired
 Delete the ground
 set the color of the
background to green

in the world properties
 Use a conditional to
replace the green color
 You may also need to scale
one of the pictures
39
Alice and
Media
Computati
on Intro
Drawing Projects
 Draw something on an Alice
picture
 Like a hat, glasses, or a scarf
 Fireworks
 Create images in Media
Computation that you use as
textures in Alice
 like a gradient for the ground
 Clip an Alice picture
 To look like a porthole
 Create a poster advertisement
for your Alice movie.
 With your name on it
 Create a comic panel with Alice
pictures
40
Alice and
Media
Computati
on Intro
Comic Strip Project
 Create several scenes in Alice and take pictures of the
scenes
 Use Media Computation to create a Comic Panel
and/or Comic Strips
 Draw different types of text balloons

Speech balloon, thought balloon, scream balloon
Alice and Media Computation Intro
41
Movie Projects
 Blend two Alice pictures together with different
amounts to have one scene fade out and the other
fade in
 Change the background of an Alice movie using
chromakey
 Make an Alice movie look as if it was pencil-drawn
by running edge detection on every frame of the
movie.
 Add yourself to an Alice movie using chromakey
 Add an Alice character to a normal movie using
chromakey
Alice and Media Computation Intro
42
Add Yourself to an Alice Movie
 Use
MovieCapturePanel to
save out jpg frames
from an alice movie
 or use Alice 2.2
 Use chromakey to
position a picture of
yourself in an Alice
frame
 Change where you
place the picture of
you to make it look
like you are moving
Alice and Media Computation Intro
43
Add Alice Characters to a Real Movie
 Split out frames from a
movie of people
 Capture frames from
an Alice movie using
MovieCapturePanel
 Use chromakey to copy
the Alice items to the
real movie
 Use MovieWriter to
create a merged movie
Alice and Media Computation Intro
44
Media Computation Data Structures
 Developed because we finally had
Liberal Arts, Architecture, and
Management students who wanted
to go on to a second course
Multimedia CS2 in Java
 Driving question:
“How did the
wildebeests stampede
in The Lion King?”
 Spring 2005: 31
students, 75% female,
91% success rate.
Connecting to the Wildebeests
It’s all about data structures
Syllabus
 Introduction to Java and Media Computation
 Manipulating turtles, images, MIDI, sampled sounds.
 Insertion and deletion (with shifting) of sampled sounds
(arrays).
 Structuring Music
 Goal: A structure for flexible music composition
 Put MIDI phrases into linked list nodes.

Use Weave and Repeat to create repeating motifs as found in
Western Music

At very end, create a two-branched list to start on trees.
Swan
Bells
Canon
Fur Elise
HW2: Create a collage, but must use
turtles
Syllabus (Continued)
 Structuring Images
 Using linearity in linked list to
represent ordering (e.g., left to right)
 Using linearity in linked list to
represent layering (as in PowerPoint)
 Mixing positioned and layered in one
structure, using abstract super classes.
 Structuring a scene in terms of
branches—introducing a scene graph
(first tree)
 (We’ll see these slides as an example
later.)
Syllabus (Cont’d)
 Structuring Sound
 Collecting sampled
sounds into linked
lists and trees, as with
images.

But all traversals are
recursive.
 Use different
traversals of same tree
to generate different
sounds.
 Replace a sound inplace
Original
Scale the children
Scale the next
Syllabus (cont’d)
 Generalizing lists and trees
 Create an abstract class
“Linked List Node”
(LLNode) on top of the
sound and image class
hierarchies
 Make all image and sound
examples work the same
abstract LLNode
Knows next
Knows how to do
all basic list
operations
Syllabus (Cont’d)
JFrame
 GUIs as trees
 We introduce
construction of a Swing
frame as construction of a
tree.
 Different layout managers
are then different
renderers of the same tree.
JPanel
JPanel
 Traditional (binary)
trees as a specialization
 Can we make trees that
are faster to search?
JLabel “This is panel1!”
JButton
“Make a
picture”
JButton “Make
a sound”
Syllabus (cont’d)
 Lists that Loop
 Introduce circular linked lists as a way of create MarioBrothers’ style cel animations.
 Introduce trees that loop as a way of introducing graphs.
jSide.jpg
jSide.jpg
jWalk.jpg
jSide.jpg
jWalk2.jpg
Syllabus (cont’d)
 Introducing Simulations
 Introduce continuous and discrete event simulations, and Normal
and uniform probability distributions
 We do wolves and deer,
disease propagation,
political influence.
 Create a set of classes for simulation, then re-write our simulations
for those classes.
 Writing results to a file for later analysis
 Finally, Making the Wildebeests and Villagers
 Mapping from positions of our turtles to an animation frame.
 Creating an animation from a simulation.
HW7: Simulate European emigration to America
 Students are required
to try several different
scenarios, aiming for
historical accuracy.
 Counts of Europeans,
Americans, and intransit per year are
written to a file for
graphing in Excel
Syllabus (cont’d)
 Introduction to Discrete Event Simulations
 Create a discrete event simulation of trucks, factories,
salespeople, and markets.
 Use turtles to create an animated display.
 Now, the real focus is the simulation, and the animation
is just a mapping from the simulation.

Animation becomes yet another medium in which we can
review results, like data in an Excel spreadsheet, music, or
sound.