Challenges in Computing Education Research:
What We Need to Know to Reach Computational Thinking for All
Mark Guzdial
School of Interactive Computing
The Two Cultures
Story
•Research Questions of Computing Education
• Story 1: Teaching Computing to Computing-related
Majors
• Measuring knowledge, measuring naïve understanding,
changing the task
• Bottom line: It’s much harder than we think, and we
don’t really know why.
• Story 2: Teaching Computing to Everyone
• Computing for All at Georgia Tech
– 1999-2003: One course for all
– 2003-2008: Contextualized Computing Education
• Studying context and computing
• Evaluation Results
Computing Education Research
• Key Question:
How do people understanding computational
processes and devices; and how can we
improve that understanding?
• As a discipline, like other domain-specific
education research fields.
• Mathematics education, physics education,
chemistry education, etc.
• Applying the general findings of education
research, with the insights of a domain, to explain
learning about the domain.
The “Geek Gene” Question
• What influences people’s understanding of
computation and their ability to learn
computation?
• Why do so few people find programming easy to
learn, but those that do find it very easy?
• Hypothesis #1: Some people have the “geek
gene”
• Hypothesis #2: Some people accidentally get
the right background.
• Math education and “unitization”
Potential Causes for the “Geek Gene”
• Experiences with time, process,
representation, and agency
• Spatial Intelligence (Shea & Lubinsky, 2001)
6
“Why Johnny Can’t Compute”
• How should we be teaching computing?
• What is a developmentally appropriate
curriculum?
• What are the factors that influence that
developmental path, e.g., culture, motivation?
• Multiple studies have sought to measure
learning about programming in undergraduate
years.
• Results are always less than we anticipate
• Yet, people seem to have significant naïve
understanding of computing
7
Rainfall Problem: Elliot Soloway (1989)
• Students were given
some form of this
problem:
• Write a program that
repeatedly reads in
positive integers, until it
reads the integer
99999. After seeing
99999, it should print
out the average.
Results
• Results at Yale:
• CS1 Students: Only 14% correctly solved the
problem (correct other than syntactic errors).
• CS2: Only 36% got it correct.
• Junior/Senior Systems Programming Course: 69%
• This problem has been replicated at schools
like University of Michigan and Georgia Tech
• Always with the same results
• Why is this hard?
Large MIMN Studies
• McCracken ITiCSE Working Group 2001
• “Build a calculator.”
• Five institutions, four countries
• Average score: 22.89 out of 110
• Lister ITiCSE Working Group 2004
• Maybe the problem is design.
• Using Multiple Choice Questions in a multi-institutional,
multi-national study.
Lister Study
Naïve Understanding of Computing
• Studies by Miller (1981)
and Pane (2001) asked
people to tell a
person/computer how
to solve some task.
• Results tell us how
people think about
specifying process
before learning a
language.
“Make a list of those
employees who make more
than 8 dollars/hr. and also
are over 50 years old. List
should be organized by
employee name.”
12
Naïve Computing
• No one uses loops
• Some IF’s, never ELSE’s
• Preference for qualification instead of iteration
• “For all the last names that start with ‘G’”…
• No object-oriented programming
• There is a sense of instances with properties
• No notion of classes or methods.
• All programs from perspective of programmer, never of
agent/object
• Heavy use of production rules, event-driven
programming.
13
Replacing ELSE’s
Thomas R.G. Green (1977)
OPTION 1:
if (value < 10)
then value = value + 10;
else sum = sum + value;
end if.
OPTION 2:
if (value < 10):
value = value + 10;
not (value < 10):
sum = sum + value;
end (value < 10).
Parallelism in Commonsense Computing
Suppose we sell concert tickets over the telephone
in the following way – when a customer calls in
and asks for a number (n) of seats, the seller
1) finds the n best seats that are available, 2)
marks those n seats as unavailable, and 3) deals
with payment options for the customer (e.g. getting
credit or debit card number, or sending the
tickets to the Will Call window for pickup).
Suppose we have more than one seller working
at the same time. What problems might we see,
and how might we avoid those problems?
(Lewandowski et al. 2007)
Results:
Summary: CompEd Research
Questions for Computing Majors
• What is hard about programming?
• How do we measure student understanding about
computing?
• Where is the challenge about our tools versus about
computing?
• Can changes to our tools make programming more
obvious?
• Can we build on students’ naïve understanding?
17
1961 MIT Sloan School Symposium
Learn Programming
to Re-Think Process Everywhere
• Alan Perlis argued that
computer science should be
part of a liberal education.
– Explicitly, he argued that all
students should learn to
program.
• Why?
– Because Computer Science
is the study of process.
– Automated execution of
process changes everything
• Including how we think about
things we already know
The Power and Fear of Algorithms
• The Economist (Sept., 2007)
spoke to the algorithms that
control us, yet we don’t
understand.
• Synthetic Biology, Credit
Ratings, Adjustable Rate
Mortgages, Search Rankings
• C.P. Snow foresaw this in
1961.
• Those who don’t understand
algorithms, can’t understand
how the decisions are made.
“A handful of people, having
no relation to the will of
society, having no
communication with the rest
of society, will be taking
decisions in secret which
are going to affect our lives
in the deepest sense.”
The
challenge of
teaching
computing is
not just about
computing
majors
(Endy et al.,
Nature 2005)
21
Alan Kay’s
Dynabook (~1970)
•Alan Kay (2004 ACM
Turing Awardee) sees the
Computer as humanity’s
first metamedium
• A medium that can
represent all other media.
• Programming as an
important new medium
•The computer-asDynabook is for creative
metamedia exploration
and reading
Why should everyone learn computing?
• Perlis: To have a powerful new tool to think
with.
• Snow: Because it’s necessary to participate
and understand in the modern world.
• Kay: To have full expressive power with the
most powerful and creative medium humans
have ever invented.
23
Context and Computation
• Context: What students want to do with
computation. An application domain.
• Biology: Invent new medicines.
• Engineering: See and test models before realizing
them.
• Media: Manipulate sounds and images to express
new thoughts in new ways.
24
Relationship between
context and computing
Segregationist
Integrationist
2. Tools that
can be seen
from context as
a lens
1. A subject that
intersects a
context
3. A lens that
offers a new
way of seeing
and doing in
other contexts.
Synergist
Computing as
literacy
Lewis &
Smith,
inroads,
June
2005
25
Examples of
Context and Computing
•Tool Model
• Bio2010 (US NRC) calls for programming and
computational models for all Biology students.
• Physics teaching VPython for labs where they
solve three-body problems.
•Lens/Literacy Model:
• Jeanette Wing’s “Computational Thinking”
• Computer science provides the tools and
metaphors for understanding our world
Teaching Computing as Literacy
Boxer
Logo
Idit Harel and Instructional
Software Design Project
(ISDP)
27
Example: Teaching
Computing as Lens on Physics
Bruce
Sherin
Emile
28
Implementing the
Subject Model
•Fall 1999:
All students at Georgia Tech must take a
course in computer science.
• Considered part of General Education, like
mathematics, social science, humanities…
•Why did Georgia Tech make that decision?
• Computing is important
• Computing was a College.
•Solved a problem for Engineering
• Made a competitive distinction for Liberal Arts
One-class CS1: Pass (A, B, or C) vs.
WDF (Withdrawal, D or F)
Success Rates in CS1 from Fall 1999 to
Spring 2002 (Overall: 78%)
Architecture
46.7%
Biology
64.4%
Economics
53.5%
History
46.5%
Management
48.5%
Public Policy
47.9%
Contextualized Computing Education
• What’s going on?
– Research results: Computing is “tedious,
boring, irrelevant”
• Since Spring 2003, Georgia Tech
teaches three introductory CS
courses.
– Based on Margolis and Fisher’s
“alternative paths”
• Each course introduces computing
using a context (examples, homework
assignments, lecture discussion)
relevant to majors.
– Make computing relevant by teaching it in
terms of what computers are good for
(from the students’ perspective)
Contexts, Results,
and Themes
•Media Computation
•Computing for Engineers (MATLAB)
•Robotics for CS1
– Institute for Personal Robotics in Education
•Pre CS1
– Contexts with Girl Scouts
•Beyond CS1
– Impact on CS2
– Gameboy programming for Computer Organization
Media Computation:
Teaching in a Relevant Context
•Presenting CS topics with
media projects and
examples
– Iteration as creating negative
and grayscale images
– Indexing in a range as removing
redeye
– Algorithms for blending both
images and sounds
– Linked lists as song fragments
woven to make music
– Information encodings as sound
visualizations
33
Results:CS1“Media Computation”
Change in Success rates in CS1 “Media
11.410%2003 to
Computation”
from7.580%
Spring
Fall 2005
12.540% 10.270% 14.650% 9.370%
19.650% 17.100% 22.540%
(Overall 85%)
Architecture
46.7%
85.7%
91.940% 87.500%
Biology
64.4%
90.4%
86.470% 88.360% 84.710% 89.870%
80.330% 82.900%
Economics
54.5%
92.0%
History
46.5%
67.6%
Total Females Males
Fall03 Fall03 Fall03
Total Females Males
Sp04
Sp04
Sp04
Management
48.5%
Public Policy
47.9%
77.460%
Total Females Males
Fall04 Fall04 Fall04
87.8%
85.4%
WDF
Pass
Voices from Media
Computation Students
• Intl Affairs student (female): “I just wish I had more time to
play around with that and make neat effects. But JES [IDE for
class] will be on my computer forever, so… that’s the nice
thing about this class is that you could go as deep into the
homework as you wanted. So, I’d turn it in and then me and
my roommate would do more after to see what we could do
with it.”
• “I dreaded CS, but ALL of the topics thus far have been
applicable to my future career (& personal) plans—there isn't
anything I don't like about this class!!!"
• "Media Computation is a CS class but with less severity. The
media part of the class is extremely visually interesting. I
would only take another CS class if it were Media
Computation."
Survey One Year Later
• 19% of respondents had programmed since class ended
"Did the class change how you interact with
computers?”
• 80% say “Yes”
• “Definitely makes me think of what is going on behind
the scenes of such programs like Photoshop and
Illustrator.”
• 'I understand technological concepts more easily now; I
am more willing and able to experience new things with
computers now’
• 'I have learned more about the big picture behind
computer science and programming. This has helped
me to figure out how to use programs that I've never
used before.’
36
Results at Gainesville College
(Tew,(Tew,
Fowler,
Fowler,
Guzdial,
Guzdial,
SIGCSE
SIGCSE
2005)
2005)
Results at
U. Illinois-Chicago
(Sloan and Troy,
SIGCSE 2008)
A Context for CS1 for
CS majors: Robotics
• Microsoft Research has
funded the Institute for
Personal Robotics in
Education
• Tucker Balch, Deepak Kumar, Doug
Blank
• Joint between Bryn Mawr and
Georgia Tech
• http://www.roboteducation.org
• Goal is to develop a CS1
with robotics as the
context.
• Includes a camera and media
computation functions
Some Attitudes Results
• All students enjoyed the robot, were
comfortable with it, and found it easy to get
working.
• Personalizing the robot improved the course, in
students’ opinion.
• Reported that the class was about computer
science
• Found homework challenging
• BMC students did more on homework
“because it was cool.”
W’s vs. F’s,
statistically significant
Pre-CS1: “Georgia Computes!”
Use in Girl Scouts Workshops
What about beyond CS1?
•If students have a contextualized CS1, how do
they perform in a context-free CS2?
•Are students as well-prepared in CS2 after a
contextualized CS1?
•Is context useful in CS2 and beyond?
The Impact of
Contextualized CS1 on CS2 enrollment
Bryn Mawr College Data for 12 years of CS2
CS2 Data Structures Enrollment
Including last two years
(since robots)
CS2 Data Structures Enrollment
A Contextualized CS2:
MediaComp Data Structures
How did the
Wildebeests
charge over the
ridge in Disney's
"The Lion King"?
50
Research Question:
Is context still useful in a second course?
• Mixed model research design
• Interviews informing whole-class survey
• 11% agreed with “Working with media is a waste of time that
could be used to learn the material in greater depth.”
• “I didn’t take this class to learn how to make pretty pictures.”
• A majority of the class (70%) agreed or strongly agreed that
working with media makes the class more interesting.
• 67% of the students agreed or strongly agreed that they were
really excited by at least one class project
• 66% reported doing extra work on projects to make the
outcome look “cool.”
(Yarosh and Guzdial, JERIC,
Jan 2008)
Context:
Programming a Gameboy
• Traditional Computer Organization using an
imaginary processor, vs.
Computer Organization using a Gameboy
• Not all the same topics (e.g., Gameboy graphics modes)
• Common focus on C programming, bit manipulations,
model of the computer (including memory and I/O)
•Post-class survey comparison: No
significant performance difference.
Big motivation difference.
• 100% of students in Gameboy class did extra on at
least one homework, because it was fun.
• 30% of traditional students never put any extra
effort in (p=0.046).
(Tew, Dorn,
Leahy,
Guzdial,
JERIC,
2008)
Summary: What do we know about
contexts for Computing Ed?
• Including context has led to evaluation reports of:
• Increased retention (in CS1, across multiple schools, with
both majors and non-CS majors, across genders and
ethnicities),
• Some evidence of increased enrollment in CS2, and
• Student reports of increased time-on-task, even beyond
CS1.
• Not all contexts work for all students.
• Context is about personal engagement
• Some students do just fine with computing Fibonacci
numbers
• Therefore, we need to offer multiple contexts, for all
Snow’s “cultures”
Summary: Does Contextualized
Computing Lead to Literacy Models?
• The Computing-as-subject model is not viable
for all students from a motivation perspective.
• Programming is detail-oriented, unforgiving.
It’s hard.
• The Computing-as-tool model is motivating.
• Maybe it enhances learning of the context, too.
• Maybe it can be a step towards a literacy model.
• Computing-as-literacy model requires an
investment in becoming literate.
• What motivates that investment?
54
Summary: What don't we
know about context?
•What is a context?
•What do students perceive as a relevant
context?
•When are contexts successful? For how long?
For whom?
•When do contexts lead to valuing literacy?
Some PhD Student Work on Contexts
• Do contexts help with learning? Over time?
• Allison Elliot Tew is developing a reliable and valid languageindependent measure of CS1 knowledge.
• When the context is fixed, how do you teach computing within
that context?
• Brian Dorn is teaching computing to graphic designers who script
Adobe Photoshop
• What is needed for teachers to succeed with contextualized
computing education?
• Lijun Ni is studying teachers attending our workshops
56
Conclusions
• There is a lot we don’t know about teaching
computing to computing majors.
• Why it’s hard
• How to assess knowledge
• How to build tools that take advantage of naïve
understanding
• Computing is important for everyone,
We should aim to be able to teach computing to
everyone.
• We don’t yet know how
• Contextualized Computing Education has great promise for
achieving the goal of teaching everyone about computing.
Thank you!
http://www.cc.gatech.edu/~mark.guzdial
http://home.cc.gatech.edu/csl
http://www.georgiacomputes.org
For more on MediaComp approach:
http://www.mediacomputation.org
Supported by:
US National Science Foundation, Microsoft Research,
Georgia Tech's College of Computing , Georgia’s
Department of Education, GVU Center, Al West Fund,
President's Undergraduate Research Award,Toyota
Foundation
58
Media Computation
Impact on CS2
• Trial at Columbus State University
• Fall 2007, 3 “sections” of CS1 for Majors:
Media Computation Java vs. traditional
– We also taught two other CS1 sections without the Media Computation focus
(using the Lewis/Loftus book) and are assessing the difference. The Media
Computation class had a significantly higher completion and pass rate and
appears to have a higher student satisfaction rate. We will be able to better
compare the two methods after this semester. Students from both groups are
taking the same CS2 class. (Wayne Summers, Dept Chair)
• Spring 2008: Same CS2 for all 3 sections.
• Results:
– No difference on retention rates, midterm exam grades,
final exam grades. (ACM Southeast)