Research in the Collaborative
Software Lab
• Mark Guzdial
guzdial@cc.gatech.edu
http://www.cc.gatech.edu/~mark.guzdial
http://coweb.cc.gatech.edu/csl
• Story
– Collaborative Dynabooks—Why? What for?
– The CoWeb/Swiki, and its Variants
– Teaching the medium that Dynabooks make
possible
Dynabook to Personal
Computer
The Personal Computer as we know it today
was invented in pursuit of the Dynabook
(And object-oriented programming, too!)
Alan Kay’s Dynabook (1972)
• Alan Kay sees the Computer as Man’s first metamedium
– A medium that can represent any other media:
Animation, graphics, sound, photography, etc.
– Programming is yet another medium
• The Dynabook is a (yet mythical?) computer for creative
metamedia exploration and reading
– Handheld, wireless network connection
– Writing (typing), drawing and painting, sound
recording, music composition and synthesis
– End-user programming
Prototype Dynabook
(Xerox PARC Learning Research
Group)
A Dynabook is for Learning
• The Dynabook offered a new way to learn new
kinds of things and a better way of learning older
things
– Learning through creating and exploring media
– Dynamic systems (like evolution)
• Especially decentralized ones (Resnick, 1992)
– Knowledge representation (Papert, 1980)
– Programming (Kay & Goldberg, 1977)
• But needed a system for creative expression
– In a time when “windows” were made of glass, and
“mice” were undesirable rodents
Smalltalk-72
• For the
Dynabook,
WIMP was
invented:
– overlapping
Windows
– Icons
– Menus
– mouse Pointer
Squeak: Our tool of choice
• Based on the original Smalltalk-80 that Xerox PARC sent
to Apple in 1981 (by Alan Kay, Dan Ingalls, Ted Kaehler,
and others)
• Released to the net, and ported to Windows and UNIX
within five weeks
– Apple license allows commercial apps, but system fixes must be
posted
• Squeak Team moves to Disney, then VPRI
– In the end, it's about media.
• Squeak today (over 30 platforms)
– Media: 3-D graphics, MIDI, Flash, MPEG, sound recording
– Network: Web, POP/SMTP, zip compression/decompress
Consider the Dynabook
• Early-1970’s were at the start of the
“Cognitive Revolution”
• Now that we know what we know about
learning, do we still think that Alan is right?
– Will people learn with a Dynabook?
– Will it be useful in helping people learn what
they want to learn?
Do people learn from creating
media?
• Yes, more than simply watching media
– Children building educational software for other kids
learn fractions and science (Harel, 1988; Kafai &
Harel, 1990)
– Students building multimedia end up with higher
grades (Hay et al., 1994)
• “Multimedia literacy” may be a component of
expertise
– For example, expert chemists constantly shift
between media and representations (Kozma et al.,
1996)
How well do people learn
programming?
• Really, really badly
• Freshmen and Sophomores at Yale couldn’t
handle error conditions in loops in 1982
(Soloway et al.)
• Freshmen and Sophomores in 3 countries and 4
institutions can’t build calculators in 2001
(McCracken et al., 2001)
• Failure/withdrawl rate in CS1 is double-digits in
most institutions
– Percentage of women and minorities in CS is falling
The role of collaboration
• We are social beings
– An audience/critic/colleague
motivates/inspires/drives us
• Constructionism: Building a public
representation of one’s knowledge
facilitates learning (Papert, 1991; Resnick,
Bruckman, Martin, 1996)
Collaboration can even help
with learning programming
• MOOSE Crossing (Bruckman,
1997)
• Harel and Kafai studies
• STABLE (Guzdial & Kehoe, 1998)
Our Goal:
Collaborative Dynabooks
Hypothesis:
Collaboratively creating and
exploring multimedia facilitates
learning
Collaborative Dynabooks
• Current work:
– CoWeb: Collaborative Websites
– Used to support learning, collaboration, and even
collaborative multimedia in over 100 classes
– Empirical research on learning, collaboration, and
cost
• Where we’re going:
– Multimedia composition for CS Education:
“Introduction to Media Composition”
CoWeb:
Collaborative Websites
• Based on Ward Cunningham’s
WikiWiki Web
– Hence it’s “other” name: Squeak Wiki
-> Swiki
• Simple system:
– It’s a website
– Where any user can edit any page
(caveat “locks”)
– And any user can create new pages
Using the CoWeb
QuickTime™ and a
QuickDraw decompressor
are needed to see this picture.
Features to support collaboration:
Recent Changes and Attachments
Security
• We save
everything,
• But it’s mostly
social
Why multimedia composition in
Computer Science?
• AAUW report:
CS classes tend to
be overly technical,
with little
opportunity for
creativity
• Echoed in recent
SIGCSE papers
“Introduction to Media
Computation”
• A New Core Area B Course
– Why do we need one?
– Georgia Tech requirement of Computing for
every major
– Only one class currently meets that need:
CS1321 Introduction to Computing
• Currently the most reviled class on campus
• Why? We’re meeting our students’ needs, not
everyones.
The Argument:
Computation for Communication
• All media are going digital
• Digital media are manipulated with
software
• You are limited in your communication by
what your software allows
– What if you want to say something that
Microsoft or Adobe or Apple doesn’t let you
say?
Programming is a
communications skill
• If you want to say something that your tools
don’t allow, program it yourself
• If you want to understand what your tools can or
cannot do, you should understand what the
programs are doing
• If you care about preparing media for the Web,
for marketing, for print, for broadcast… then it’s
worth your while to understand how the media
are manipulated.
• PROGRAMMING KNOWLEDGE = FREEDOM
What’s computation good for
• Computer science is the study of recipes
• Computer scientists study…
– How the recipes are written (algorithms,
software engineering)
– The units used in the recipes (data structures,
databases)
– What can recipes be written for (systems,
intelligent systems, theory)
– How well the recipes work (human-computer
interfaces)
Specialized Recipes
• Some people specialize in crepes or
barbeque
• Computer scientists can also specialize on
special kinds of recipes
– Recipes that create pictures, sounds, movies,
animations (graphics, computer music)
• Still others look at emergent properties of
computer “recipes”
– What happens when lots of recipes talk to one
another (networking, non-linear systems)
KEY: The COMPUTER does the
recipe!
• Make it as hard, tedious, complex as you
want!
• Crank through a million genomes? No
problem!
• Find one person in a 30,000 campus?
Yawn!
• Process a million dots on the screen or a
bazillion sound samples…MEDIA
COMPUTATION!
Why should you need to
study “recipes”?
• To understand better the recipe-way of
thinking
– It’s influencing everything, from computational
science to bioinformatics
– Eventually, it’s going to become part of everyone’s
notion of a liberal education
• To work with and manage computer
scientists
• AND…to communicate!
– Writers, marketers, producers communicate
through computation
“But PhotoShop is great!”
• Okay, but so are ProAudio Tools,
ImageMagick and the GIMP, and Java and
Visual Basic
• You can learn lots of tools
OR
You can learn a key set of tools, and a
general set of principles and programming
skills AND learn new tools easier
It’s still computer science
• For example, all the “under the hood”
standard data structures are there
still:
– Sounds are arrays of samples
– Pictures are matrices (two-dimensional
arrays) of pixels (x, y, color in
red+green+blue)
– Movies are arrays of pictures!
Example: Opening a Picture
fp=pickAFile()
p=makePicture(fp)
show(p)
Example: A “Photoshop-like”
filter
def decreaseRed(pic):
for x in getPixels(pic):
setRed(x, 0.95 *
getRed(x))
Before
After
(twice)
Example: Background
Subtraction
• Let’s say that you
want to build a picture
of your daughter on
the moon.
• Take a picture of her
against the wall, then
just the wall.
Example Solution
• For each pixel (dot) in the picture
– Get the pixel in the picture
– Get the pixel in the frame (without the
kid)
– Are the colors pretty darn close?
• Must be where the kid ISN’T, so grab the
picture of the moon
Example code
#Picture with person, background, and
newbackground
def swapbg(pic1, bg, newbg):
for x in range(1,pic1.getWidth()):
for y in range(1,pic1.getHeight()):
p1px = getPixel(pic1,x,y)
bgpx = getPixel(bg,x,y)
if (distance(getColor(p1px),getColor(bgpx)) < 15.0):
setColor(p1px,getColor(getPixel(newbg,x,y)))
return pic1
Why doesn’t it look better?
• Can you figure out where the light
source was?
• The colors on the shirt and the colors
of the wall were awfully similar
New Approach:
Chroma key!
• It’s what weather
people do
• Pose in front of a blue
screen
• Swap all “blue” for the
background
Example Solution
def chromakey2(source,bg):
for p in getPixels(source):
if (getRed(p)+getGreen(p) < getBlue(p)):
setColor(p,
getColor(getPixel(bg,
getX(p),getY(p))))
return source
Another way of saying the
same thing
def chromakey(source,bg):
# source should have something in front of blue, bg is the new
background
for x in range(1,source.getWidth()):
for y in range(1,source.getHeight()):
p = getPixel(source,x,y)
# My definition of blue: If the redness + greenness < blueness
if (getRed(p) + getGreen(p) < getBlue(p)):
#Then, grab the color at the same spot from the new
background
setColor(p,getColor(getPixel(bg,x,y)))
return source
Can I do this by masking in
Photoshop?
• Of course!
–How do you think Photoshop does
it?
• But you can do it better,
differently, faster, and for more
kinds of pictures if you know how
it works
Example: Segmenting and
Splicing Sounds
• Recorded myself reading the first part of
the preamble
• Then used a sound editor to find the
sample number where each word ended
Let’s keep only the first part
•
•
•
•
•
•
•
•
•
•
#Segmenting
# Here's what I got from exploring with the editor
# Word Endpoint
# We 15730
# the 17407
# People 26726
# of 32131
# the 33413
# United 40052
# States 55510
def segmentMain():
# Grab the sound JUST up to "We the people of the United
States"
fs="/Users/guzdial/mediasources/preamble.wav"
s=makeSound(fs)
nfs = "/Users/guzdial/mediasources/sec1silence.wav"
#An empty sound
ns=makeSound(nfs) # This is where we'll build the new sound
nsi=1 # New sound index, starting from 1
for si in range(1,55510):
# Where the samples are in the sound
setSampleAt(ns,nsi, getSampleAt(s,si))
nsi = nsi + 1
play(ns)
writeSoundTo(ns,"preamble-start.wav")
Now, let’s make me say
something I didn’t say
• “We the UNITED People of the UNITED
States”
# Splicing
# Using the preamble piece, making "We the united
people"
def spliceMain():
fs="/Users/guzdial/mediasources/preamble-start.wav"
s=makeSound(fs)
ns=makeSound(fs) # This is where we'll build the new
sound
nsi=17408 # New Sound Index starts at just after "We
the" in the new sound
for si in range(33414,40052):
# Where the word "United" is in the sound
setSampleAt(ns,nsi, getSampleAt(s,si))
nsi = nsi + 1
for si in range(17408, 26726): # Where the word
"People" is in the sound
setSampleAt(ns,nsi, getSampleAt(s,si))
nsi = nsi + 1
play(ns) #Let's hear and return the result
return ns
Research Issues
• Hard to define usability here
– We don’t want to make the task easier;
We want to make the learning easier
– Within that definition, does it work?
• Does it help with learning?
– Compared to CS1321?
• More importantly, does it help with motivation?
– More than CS1321?
– More for women and minorities?
– If not, what should we change?
(Right now: What should we put in in the first place?)
Time Frame
• Fall 2002: Building everything: Book,
lecture slides, homework and project
definitions, examples, quizzes, exams…
• Spring 2003: Trial with 100 students (and
maybe IDT grad class?)
• Summer 2003: Study, iterate, and train 812 TAs
• Fall 2003: Full scale deployment: 2 x 250300 person sections
Summary: A Wonderfully
Exciting Time for CSL!
• Swiki/CoWebs are booming
– Do a search for “Swiki” in Google!
(I love the German and Japanese ones I can’t read :-)
– ChemEng just started their own server, and every
class gets its own Swiki
• “Introduction to Media Computation” is about
putting our money (time, effort…) where our
mouths are
– Can we teach people multimedia?
– Does it help them learn programming?