Understanding and Promoting
Interaction in the Classroom
UNIVERSITY LEVEL
Ph.D. Defense
Steven A. Wolfman
Computer Science & Engineering
University of Washington
The Blackboard
t
“…in the winter of 1813 & '14 … I attended a
mathematical school kept in Boston…On
entering [the] room, we were struck at the
appearance of an ample Black Board
suspended on the wall… I had never heard
of such
a thing
before.” ed tech
BENEFITS:
successful
[Samuel
J. May, 1855]
CHALLENGES:
few such successes
LEVERAGE PNT: public mediating artifact
Steve Wolfman
Understanding and Promoting Interaction
BIG EXCEPTION:
PowerPoint
2
Mediating Artifact [Vygotsky]
t
An external object or structure that
participates in cognition by supporting or
shaping thought.
Vygotsky: thought is social
DISTRIBUTED COGNITION
Steve Wolfman
Understanding and Promoting Interaction
3
Slides as Mediating Artifact
• In the classroom:
t
Persistent context for communication!
– facilitate communication
– structure discussion
• Outside the classroom:
– used as memory aid
– used as study guide
• Across terms:
– reify course knowledge
Steve Wolfman
Understanding and Promoting Interaction
4
Thesis Question
t
How can computer technology
exploit the mediating nature of
presentation slides to support and
shape interactive learning and
teaching?
UNIVERSITY LEVEL
Steve Wolfman
Understanding and Promoting Interaction
5
Research History
Classroom
Presenter
Gestural Model
of Ink
Distance & Large
Class Studies
t
Classroom
Feedback
System
Feedback Patterns
Structured Interaction
Presentation system
Steve Wolfman
Understanding and Promoting Interaction
6
Design Experiment Methodology [Brown]
t
Class studies
Evaluation &
user-centered design
Steve Wolfman
Understanding and Promoting Interaction
Theoretical
framework
System
design
7
Outline
t
• Introduction
• Classroom Presenter
• Classroom Feedback System (CFS) and
feedback patterns
• Structured Interaction Presentation System (SIP)
• Conclusions
Steve Wolfman
Understanding and Promoting Interaction
8
Classroom Presenter Goals
t
PETTT studies suggest like:
organization / preparation
• Maintain strengths of slideware
(organization, preparation, sharing, execution)
• Mitigate weaknesses of slideware
studies indicate need:
(inflexibility, immobility, passivity)Prelim
flexibility/contextual writing
• Secure classroom adoption
• Prepare for more ambitious systems
Explore slides as mediating artifact
Response to dist stud: INK in CONTEXT
Understanding and Promoting Interaction
TRANS: benefit from separating views (fract med art)
Steve Wolfman
9
t
Steve Wolfman
Understanding and Promoting Interaction
benefit from
separating views (fract med art)
10
Instructor view
with notes
Displayed view
without notes
COMPARE WRITTEN
NOTES WITH INK!
Innovations from
User-Centered Design
•
•
•
•
•
Instructor notes
Filmstrip and slide previews
“Whiteout”
“TV Talk Show” Tablet
Collective brainstorming
t
Flexibility/interaction enabled by exploiting:
SEPARATION OF VIEWS, INK IN CONTEXT,
SLIDES AS MED ART
Steve Wolfman
Understanding and Promoting Interaction
12
Classroom Deployments
Surveyed since Spring ’02:
t
– 37 courses
– 21 instructors
– 2,000+ students
– CSE courses: introductory to Master’s level
– UW, U. of Virginia, & U. of San Diego
Steve Wolfman
Understanding and Promoting Interaction
13
Survey Results
t
• Positive comments and repeat use by instructors
• Instructor survey: N = 9 Omits all project participants.
Students engaged in
lecture
Use in future
0% less
44% no change
56% more
0% no
33% maybe
67% yes
• Student surveys: N = 479
Attention to lecture
10% less
35% no change
55% more
Encourage future use
8% disc.
22% neutral
69% enc.
Steve Wolfman
Understanding and Promoting Interaction
14
Contributions (Presenter)
t
• Combined strengths of slides w/increased
flexibility, mobility, potential for interaction
• Developed features that exploit slides as
mediating artifact:
– Ink in context
– Separation of views
• Secured broad adoption
• Established basis for student extensions
Steve Wolfman
Understanding and Promoting Interaction
15
Outline
t
• Introduction
• Classroom Presenter
• Classroom Feedback System (CFS) and
feedback patterns
• Structured Interaction Presentation System (SIP)
• Conclusions
Steve Wolfman
Understanding and Promoting Interaction
16
CFS Goals
t
• Understand challenges to interaction
• Develop system exploiting slides as
mediating artifact to respond to challenges
• Evaluate impact of feedback
NOT system
FOCUS today.
• Understand how feedback system
changes interaction
Steve Wolfman
Understanding and Promoting Interaction
17
Challenge to Interaction:
Feedback Lag
t
A student hesitates to pose a question
until the instructor finishes a point. When
the instructor moves on, the question
seems out of place and is left unasked.
EVIDENCE: Survey/focus group responses (3/12 in pilot; 5/11 in final)
Lagged questions in video archives; Personal experience
Steve Wolfman
TRANS: built CFS in response to challenges WE
Understanding
andthis
Promoting
18
identified
like
one Interaction
Leaving Feedback on Current Slide
t
Steve Wolfman
Understanding and Promoting Interaction
19
Leaving Feedback on Last Slide
t
Steve Wolfman
Understanding and Promoting Interaction
20
Instructor View
t
Steve Wolfman
Understanding and Promoting Interaction
21
So, did it work?
“Retrospective” Feedback
t
Students’ response (n=12; 7 sessions;
150 students total):
Imagine n=150!
– 29 episodes of retrospective feedback
– CFS helped all who reported feedback lag
Instructor’s response:
Adoption lessons for classroom tech.
Not enough retrospect
– Retro. feedback is important; often responded
– Retro. feedback upset pacing
Steve Wolfman
Understanding and Promoting Interaction
22
“Prospective” Feedback
t
What if a student leaves feedback
ahead of the discussion?
“…if I’m smooth enough… the class will just think
‘Oh, he’s going to talk about [that] now.’ … [To
them,] here’s something that for some reason I
Steve Wolfman
Understanding
and Promoting
decided
to talk about
towards
theInteraction
end of the slide.”23
Prospective Feedback Episode
t
Steve Wolfman
Understanding and Promoting Interaction
24
Contributions (CFS)
t
•
•
•
•
•
Identified interaction challenges
Proposed slide context as interaction medium
Developed contextual feedback system
Established potential for student feedback
Discovered novel interaction patterns
– retrospective feedback: addressing feedback lag
– prospective feedback: enabled by computer-mediated
communication
Steve Wolfman
Understanding and Promoting Interaction
25
Outline
t
• Introduction
• Classroom Presenter
• Classroom Feedback System (CFS) and
feedback patterns
• Structured Interaction Presentation System (SIP)
• Conclusions
Steve Wolfman
Understanding and Promoting Interaction
26
“Conductor-of-Performances” Model
t
Computer-Supported Collaborative Learning
(CSCL) has been “a move from ‘sage-onthe-stage’ … to ‘guide-by-the-side’”.
New CSCL systems will be “much more like
the ‘conductor-of-performances’ for an
orchestra: students … [will contribute] to
an overall performance.”
Leader-of-theater?
[Roschelle & Pea]
Steve Wolfman
Understanding and Promoting Interaction
27
Goals of Structured Interaction
Presentation System (SIP)
t
Support the design, use, sharing, and
reflection on the “orchestra’s” score.
interactive exercises +
•
•
•
•
rich student data
Mitigate slides’ passivity, oversimplification
design
Maintain intuitive, flexible design w/PPT-style
+ widgets
Explore enabled interactions
Understand how integrated exercises
affect the classroom
Steve Wolfman
Understanding and Promoting Interaction
28
EXPLAINING SIP “by example”
Experimental class on “Risk Assessment”
24 “students”
50 minutes
6 SIP exercises
Took ~1 hour to convert static => interactive
Designed INSIDE PPT USING SIP, etc.
Steve Wolfman
Understanding and Promoting Interaction
t
29
SIP Architecture
Presenter + SIP exercises
Presentation
design
environment
SQL back-end for
reliability/archival reuse.
t
Instructor view
Presentation/
Widget
database
Interactive
widget design
environment
Pluggable widgets.
Support interactive
version of PPTUnderstanding
vision.
Steve Wolfman
and Promoting Interaction
Viewer
Viewer
scrnsht
Viewer
scrnsht
scrnshtviews
Student
30
REMIND: exercises from experimental
t
class; REAL DATA
Steve Wolfman
Understanding and Promoting Interaction
31
REMIND: exercises from experimental
t
class; REAL DATA
Steve Wolfman
Understanding and Promoting Interaction
32
t
Steve Wolfman
Understanding and Promoting Interaction
33
Distributed Human Computation (DHC)
t
Of those who died from
What are the death rates
receiving the vaccine, what for specific groups who
percentage had comproreceived this vaccine?
mised immune systems?
• Are these on the same or distinct topics?
• Which would you rather discuss?
Steve Wolfman
Understanding and Promoting Interaction
34
t
Steve Wolfman
Understanding and Promoting Interaction
35
DHC Results: Instructor’s View
t
Group
“winners”
Group
members
Steve Wolfman
Understanding and Promoting Interaction
36
Experimental Class Results
• Interaction analysis (video/audio/logs):
substantial engagement by students
• Student survey results
t
– Factors supporting interaction:
highlights particular strengths of integration
– Factors hindering interaction:
highlights important design lessons
Steve Wolfman
Understanding and Promoting Interaction
37
98% participation in SIP exercises
4-7 interactions reported/student (median)
3 recorded voicings/student (72 total)
Interaction Analysis
68% K-12 norm
t
teacher talk
(62%)
student talk
(15%)
student
discussion
(13%)
student
thinking
(5%)
other
(5%)
0:00:00
Steve Wolfman
0:10:00
0:20:00
0:30:00
0:40:00
Understanding and Promoting Interaction
0:50:00
1:00:00
38
78% cited supporting factors EXCLUDING NOVELTY
Factors Supporting Interaction (n=18)
t
Category
% students
Sharing responses w/ whole class 39
Participatory feel
33
Novelty
22
Anonymity
17
Forced to participate
17
Helps follow instructor
11
Neighbor discussion
6
None
6
“anonymity encouraged honest participation”
Wolfman
Interaction
39
“I feltSteve
as though
I … didn’t haveUnderstanding
much choiceand
but Promoting
to participate
… I think this is a good thing.”
Did not respond
6
50% cited aspect hindering interaction
Factors Hindering Interaction (n=18)
Category
% students
None
56
Distracting applications
22
Distracted looking at my slides
17
Did not respond
11
Reduced coverage
11
Lack of student control
11
Pace too fast
6
Pair discussions
6
Steve Wolfman
Understanding and Promoting Interaction
Anti-climactic exercise
6
t
40
Contributions (SIP)
t
• Proposed “score” as role of slides as
mediating artifact in “orchestra” CSCL model
• Developed prototype SIP system
• Designed novel interactive exercises
(e.g., DHC, “sampled quiz”)
• Identified advantages and pitfalls of
integrated interactivity
Steve Wolfman
Understanding and Promoting Interaction
41
Conclusions
• Developed and evaluated widely-adopted
Classroom Presenter system
• Developed and evaluated Classroom Feedback
System
• Developed and evaluated Structured Interaction
Presentation System
t
Demonstrated how to exploit slides as mediating
artifact across all three systems (e.g., separated
views, contextual feedback, “forced” participation)
Steve Wolfman
Understanding and Promoting Interaction
42
Systems FIT IN
DISCUSSRelated
HOW CP/CFS/SIP
• Ubiquitous Computing:
t
– eClass/Classroom 2000 [Abowd & Brotherton]
– ActiveClass [Griswold, Ratto, Truong, et al.]
– Cell-phone feedback [Brittain]
• Education/Educational Technology:
– ClassTalk [Dufresne]
– Debbie/DyKnow [Berque]
– WILD [Roschelle & Pea]
• HCI: Pebbles [Myers]
Steve Wolfman
Understanding and Promoting Interaction
43
Related Pedagogy
• Active learning [Bonwell & Eison]
• Active learning in CS [McConnell]
• Classroom Assessment Techniques
“CATs” [Angelo & Cross]
• CATs in CS [Schwarm & VanDeGrift]
• Collaborative Learning
[Johnson & Johnson]
Steve Wolfman
Understanding and Promoting Interaction
t
44
Acknowledgments
•
•
•
•
•
•
•
•
Richard Anderson and the Committee
Rachel Pottinger
Education and Educational Technology Group
Microsoft Research LST Group
Experiment participants
Faculty, staff, and students of UW CSE
Intel, MERL, Microsoft, and NSF for funding
Everyone
Steve Wolfman
Understanding and Promoting Interaction
t
45
SPARES
WARNING: overgeneralization, but borne out by research
Modern Pedagogy vs. Modern Practice
t
active learning
lecture
student-directed
instructor-dominated
interactive
disconnected
participatory
passive
~80-90% lectures Thielens, 1987
Many ways to resolve tension: reduce class size, retrain instructors, etc.
QUOTE
ON RQ SLIDE Understanding and Promoting Interaction
Steve Wolfman
47
Pedagogy of Active Learning
• Encourage “connected” learning
t
– Constructivism [Bruner]
– Social learning [Lave]
• Recapture flagging attention
– Attention studies [Stuart & Rutherford]
– Heart rate/memory [Bligh]
Smiley plot
– Skin conductivity [Picard]
• Address varied learning styles
– Index of Learning Styles [Felder & Silverman]
– Bloom’s taxonomy
Steve Wolfman
Understanding and Promoting Interaction
48
Bruner (constr.) Lave (soc.
Learning)
Attention studies [Stuart &
Rutherford]
Heart rate/memory [Bligh]
Skin conductivity [Picard]
ILS [Felder & Silverman]
Bloom’s taxonomy
Large class challenges
•
•
•
•
t
Maintaining attention
CFS: 2.4 voicngs/class,
Communication/Feedback
90-120 studs
Tech problems in
Spontaneous discussion
distance
Management of class activities
Our experience &
McConnell
Steve Wolfman
Understanding and Promoting Interaction
49
Slides as Externalization/
Mediating Artifact

Saljo: the significance of new technologies does not lie in t
their enhancing learning in a linear sense… the important
point about new technologies is that they, if they are
powerful enough, transform basic features of how people
communicate knowledge and skills in society and how
information is organised. In this sense, new media may
imply that learning will become different.
Technologies are ultimately about the regulation and
improvement of human relationships
Draw mental arith–
paper and pencil – mem
Elec calculator --alg:
communicates in familiar
symbolic representation
Steve Wolfman
Understanding
and Promoting
Interaction
HK Jade market
: comm
burden
50
t
Steve Wolfman
Understanding and Promoting Interaction
51
Slideware Strengths/Weaknesses
t
•
•
•
•
Strengths
Organization [PETTT]
Preparation [PETTT]
Sharing [Bligh]
Easy execution [Bligh]
Preliminary studies indicated instructors
needed: flexibility and contextual writing
Steve Wolfman
•
•
•
•
Weaknesses
Passivity [PETTT]
Simplified ideas [Tufte]
Inflexibility [VanDeGrift]
Immobility
PETTT studies suggest students like:
organization
and preparation.
Understanding and Promoting
Interaction
52
Slide previews with navigation
t
Steve Wolfman
Understanding and Promoting Interaction
53
t
Steve Wolfman
Understanding and Promoting Interaction
54
Design Experiment [Brown]
t
•
•
•
•
Discover what inhibits interaction
Understand what makes a good design
Design intervention
Evaluate
PIPE DREAM!
Iterative process: in particular, Ann Brown’s
“design experiment” style
Steve Wolfman
Understanding
Promoting in
Interaction
55
Still,
will address
the andsteps
this order.
Feedback on Student View
t
Steve Wolfman
Understanding and Promoting Interaction
56
Instructor View (2/3)
t
Steve Wolfman
Understanding and Promoting Interaction
57
Instructor View (3/3)
t
Steve Wolfman
Understanding and Promoting Interaction
58
Final evaluation iteration
CFS Evaluation
t
Intro. programming course, summer 2002:
– 150 students, 12 participants w/laptops
– 9 week course, 3 weeks with CFS
Data: observations, surveys, focus groups,
interview w/instructor, logs
Steve Wolfman
Understanding and Promoting Interaction
59
CFS increased interaction
Voicings Voicings All inter- All but
pre-CFS with CFS actions “Got it”
# per
class
p-value
Steve Wolfman
2.4
2.6
15.9
7.9
.91
.04*
.14
Understanding and Promoting Interaction
t
60
t
Steve Wolfman
Understanding and Promoting Interaction
61
REMIND: exercises from experimental
t
class; REAL DATA
Steve Wolfman
Understanding and Promoting Interaction
62
t
Steve Wolfman
Understanding and Promoting Interaction
63
t
TRANS: DESIGN
Steve Wolfman
Understanding and Promoting Interaction
64
t
Steve Wolfman
Understanding and Promoting Interaction
65
t
Steve Wolfman
Understanding and Promoting Interaction
66
t
Steve Wolfman
Understanding and Promoting Interaction
67
Experimental SIP Class
Topic: Risk Assessment
Duration: 50 minutes
Students: 24, CS students, faculty, staff,
each w/a Tablet PC
Presentation included
six SIP exercises.
Steve Wolfman
t
Results:
• System successful
• 98% participation in
SIP exercises
• 4-7 interactions per
student (median)
• 62% “teacher-talk”
Took ~1 hour to convert static => interactive
Designed INSIDE PPT USING SIP, etc.
62% T-T (compare to 68% K-12 norm)
Median of 3-5 interactions through SIP
Understanding and Promoting Interaction
68
Design Lessons
t
• SIP’s integrated exercises create “participatory feel”
• Students’ social expectations support participation
• SIP anonymity policies should be easy to specify
and understand
• Student interface should present few distractions
• Student interface should provide clear value
(independent navigation and notetaking)
• Instructors must still motivate interactive pedagogy
Steve Wolfman
Understanding and Promoting Interaction
69
t
Steve Wolfman
Understanding and Promoting Interaction
70