Integrated Collaborative
Learning Environments
with Dynamic Support
Carolyn Penstein Rosé
Language Technologies Institute/
Human-Computer Interaction
Institute
Design Principle
Know what
problem you
are trying to
solve!!
Passivity in Class
Discussions
Accountable Talk
A codified set of moves that
Ts and Ss can be taught
that build effective language
in the classroom
 Press individuals to explain and elaborate
 Require individuals to listen and respond to
each other
 Reveal misunderstandings, incomplete
ideas..
 Let Ss know how others are “hearing them”
 Spread participation through the group

Transactivity


Students explicitly
display their reasoning
Students orient their
contributions towards
previous contributions
 May
be competitive or
non-competitive
 May be
representational
operational
 May be oriented
towards self or other
Student 1: Alright, we put that the
negative and positive whole numbers.
Teacher: Alright, other ideas you want
to add on, yes Mario.
Student 2: I put a um numbers that
doesn't include decimal points or
fractions but they include, they can
include negative signs or positive
signs.
Accountable Talk elicits Transactivity in
group discussion
Accountable Talk Move
Revoicing
Restate Request
Reasoning Application
Request
Prompting for Further
Participation.
Request for Elaboration
Challenge or
Request for Counter-Example
Corresponding Transactivity
Move
Clarification, Competitive
Clarification, Refinement
Paraphrase, Competitive
Paraphrase
Extension, Competitive
Extension, Contradiction,
Reasoning Critique
Extension, Completion,
Clarification
Extension, Completion
Competitive Paraphrase,
Contradiction, Competitive
Extension
Accountable Talk elicits Transactivity in
group discussion
Eddie: Well, i don't think it matters what order the numbers are in. You
still get the same answer. But three times four and four times three
Reasoning
seem like they could be talking about different
things. Application Request
Justification
Request do you agree or disagree with what Eddie is saying?
Teacher: Rebecca,
Rebecca: Well, I agree that it doesn't matter which number is first,
because they both give you twelve. But I don't get what Eddie means
Request for Elaboration
about them saying different things.
Clarification
Teacher: Eddie, would you explain what you mean?
Eddie: Well, I just think that like three times four can mean three groups
of four things, like three bags of four apples. And four times three
means four bags of three apples, and those don't seem like the same
thing.Critique
Reasoning
Tiffany: But you still have the same number of apples, so they are the
Revoicing
same!
Teacher: OK, so we have two different ideas here to talk about. Eddie
says the order does matter, because the two orders can be used to
describe different situations. So Tiffany, are you saying that three times
four and four times three can't be used to describe two different
situations?
Unequal
Participation
Wizard-of-Oz Setup
Tutor
Student
Student
Problem Solving Interface
Interface allows
students to submit
Incorrect solutions
Reflection Interface
Scaffolding for Collaboration
• Prompts to encourage
the instructionally
beneficial behavior of
students
– Explanations
– Equal contributions
Explanation Prompt
(Student 1 corrects the answer for the confederate learner
without offering any explanation)
Student1: c is 1
Auto-Prompt (to student 1): Help the student understand your
correction.
Student2 : what does c mean?
Student1 : constant
Student2 : ok
Student1 : the constant in front of the variable which in our
case is variable t
Student2 : ok
Student1 : now for the second part we use the constant
function
Equal Contribution Prompt
(After the participant makes 5 correct problem actions without
confederate student’s contribution)
Auto-Prompt: It seems like the other student has not
contributed lately. Why don’t you see if they need help?
Student1: you want to do the other half?
Student2: ok
Learning effect

Significant benefit for prompting on student
learning with pretest as a covariate
 F(1,39)
= 4.47, p < .05, effect size .58
Process Losses
Another Domain: Earth Sciences


Collaborative idea
generation in the Earth
Sciences domain
Chinese TagHelper
learns hand-coded topic
analysis
 Human
agreement .84
Kappa
 TagHelper performance
.7 Kappa

Trained models used in
follow-up study to trigger
interventions and
facilitate data analysis
Example Dialogue
Individuals+ Individuals+
Feedback
NoFeedback
Pairs+
Feedback
Pairs+
NoFeedback
Speaker
Text
Student 1
People stole sand and stones to use for construction.
VIBRANT
Yes, steeling sand and stones may destroy the balance and
thus make mountain areas unstable. Thinking about
development of mountain areas, can you think of a kind of
development that may cause a problem?
Student 2
Development of mountain areas often causes problems.
Student 1
It is okay to develop, but there must be some constraints.
* Feedback during idea generation increases learning and
idea generation productivity (except during the first 5 minutes) (Wang et al., 2007)
Individuals+ Individuals+
Feedback
NoFeedback
Pairs+
Feedback
Process Analysis
Pairs+
NoFeedback
Unique Ideas
12
Individuals+Feedback
Individuals+NoFeedback
6
8
Pairs+Feedback
Pairs+NoFeedback
0
2
4
#Unique Ideas
10
Nom+N
Nom+F
Real+N
Real+F
0
Process loss Pairs vs
Individuals:
F(1,24)=12.22, p<.005, 1 sigma
Negative effect of Feedback:
F(1,24)= 7.23, p<.05, -1.03
sigma
5
10
15
Time Stamp
20
25
30
Process loss Pairs vs Individuals:
F(1,24)=4.61, p<.05, .61 sigma
Positive effect of feedback:
F(1,24)=16.43, p<.0005, 1.37 sigma
Deficient Help
Exchange
Why study help?
Offering of deep help and elaborated
explanations predicts post test
performance (e.g., Webb et al., 2002)
Help behavior mediates learning (e.g.,
Gweon et al., 2006; Gweon et al.,
2007)
Exchanging help in the context of
collaborative learning increases
identification with a learning
community and motivation, and
improves race relations (Sharan, 1980)
Providing help leads to feelings
empowerment for “low status” students
(Elbers & Hann, 2004)
How can we prompt helping
behavior?


DIRECT: Explicit prompts (Gweon et al., 2006)
LESS DIRECT: Manipulating availability of help
from problem solving environment (Gweon et al.,
2007)
 Girls
offered more help with delayed feedback from
environment
 Boys offered more help with immediate help from the
environment

SUBTLE: Conversation agents inject humor
(Kumar et al., 2007)
 Positive
effects on perceived help exchange (p<.05),
effective help exchange (p<.1), and Learning (p=.06)
Collaborative Problem Solving Environment
Jan packed several books to amuse herself on
along car ride to visit her grandma. After 1/5 of
the trip she had already finished 6/8 of the books
she brought. How many times more books should
she have brought than what she packed?
Extraneous Entertainment?
Coding Scheme (Gweon et al.,
2007)

Help provision mediates learning (Gweon et al.,
2006; Gweon et al., 2007)

(R) Help Requests: “Help me”, “I’m stuck”, “I don’t
get it.”
(P) Help Provisions: “Find the common
denominator”, “Try the flip and multiply strategy”
(C) Can’t help: “I don’t know”, “I’m stuck too”
(D) Deny help: “ask the teacher”, “you’re an idiot”,
“press the help button”
(N) Nothing




Examples
[R] Student 1: What do we put on top of the fraction?
[P] Student 2: Did you find a common denominator?
<student 1 correctly finds the common denominator>
[R] Student 1: I don’t get it
[D] Student 2: hold on
<then student 1 tried something and got negative feedback from the problem solving
environment.>
<finally student 1 tried something else, which was correct, and got positive feedback
from the problem solving environment>
[R] Student 1: Why 16?
[C] Student 2: I don’t know.
[R] Student 1: I don’t get it
<student 2 tries something and gets negative feedback from the problem solving
environment>
<student 2 tries something else and gets negative feedback from the problem solving
environment>
<student 2 clicks on the help button>
<student 1 tries something that is correct and gets positive feedback from the problem
solving environment>
Big findings:
More help related episodes in
experimental condition
More episodes where people got
help, and then managed to solve
the problem themselves in the
experimental condition
Vision
Vision






Support for collaborative
learning is like training
wheels
Effective support allows
learners to achieve better
collaboration
Unnecessary support can
be demotivating
Fading support is ideal
But too little support can
be detrimental as well
Trained human facilitators
are able to achieve the
right balance
Vision






Support for collaborative
learning is like training
wheels
Effective support allows
learners to achieve better
collaboration
Unnecessary support can
be demotivating
Fading support is ideal
But too little support can
be detrimental as well
Trained human facilitators
are able to achieve the
right balance
Vision






Support for collaborative
learning is like training
wheels
Effective support allows
learners to achieve better
collaboration
Unnecessary support can
be demotivating
Fading support is ideal
But too little support can
be detrimental as well
Trained human facilitators
are able to achieve the
right balance
Vision






Support for collaborative
learning is like training
wheels
Effective support allows
learners to achieve better
collaboration
Unnecessary support can
be demotivating
Fading support is ideal
But too little support can
be detrimental as well
Trained human facilitators
are able to achieve the
right balance
Vision






Support for collaborative
learning is like training
wheels
Effective support allows
learners to achieve better
collaboration
Unnecessary support can
be demotivating
Fading support is ideal
But too little support can
be detrimental as well
Trained human facilitators
are able to achieve the
right balance
Vision






Support for collaborative
learning is like training
wheels
Effective support allows
learners to achieve better
collaboration
Unnecessary support can
be demotivating
Fading support is ideal
But too little support can
be detrimental as well
Trained human facilitators
are able to achieve the
right balance
Architecture for
Dynamic
Collaboration Support
VMT-Basilica
** Students learn up to 1.25 standard deviations more when interactive
support is provided in the environment.
Basilica Architecture
A channel filter
collects all of the
events that occur at
the interface from
all students
involved in the
conversation
Basilica Architecture
Events are then
channeled to
special purpose
filters such as a text
processing filter
Other filters may
keep track of time
or other factors not
related to student
behavior
Basilica Architecture
Filters that are
related to decision
making for specific
types of
interventions
receive notifications
from the analysis
filters and pass
notifications on to
the Actors, which
are responsible to
launching
interventions
Basilica Architecture
Actors of different
types produce
events that are sent
to the Outgoing
Message Spooling
Filter
From there, events
are sent through the
Presence Actor to
the interface.
Issues to consider

What problem are you trying to solve?
 Formulate

analysis scheme
When should you intervene?
 Use
technologies like TagHelper and SIDE to
track interaction and trigger support

What should the intervention be?
 Technologies
support
like TuTalk can be used to offer
Questions?