Integrated Collaborative Learning Environments with Dynamic Support Carolyn Penstein Rosé Language Technologies Institute/ Human-Computer Interaction Institute Design Principle Know what problem you are trying to solve!! Passivity in Class Discussions Accountable Talk A codified set of moves that Ts and Ss can be taught that build effective language in the classroom Press individuals to explain and elaborate Require individuals to listen and respond to each other Reveal misunderstandings, incomplete ideas.. Let Ss know how others are “hearing them” Spread participation through the group Transactivity Students explicitly display their reasoning Students orient their contributions towards previous contributions May be competitive or non-competitive May be representational operational May be oriented towards self or other Student 1: Alright, we put that the negative and positive whole numbers. Teacher: Alright, other ideas you want to add on, yes Mario. Student 2: I put a um numbers that doesn't include decimal points or fractions but they include, they can include negative signs or positive signs. Accountable Talk elicits Transactivity in group discussion Accountable Talk Move Revoicing Restate Request Reasoning Application Request Prompting for Further Participation. Request for Elaboration Challenge or Request for Counter-Example Corresponding Transactivity Move Clarification, Competitive Clarification, Refinement Paraphrase, Competitive Paraphrase Extension, Competitive Extension, Contradiction, Reasoning Critique Extension, Completion, Clarification Extension, Completion Competitive Paraphrase, Contradiction, Competitive Extension Accountable Talk elicits Transactivity in group discussion Eddie: Well, i don't think it matters what order the numbers are in. You still get the same answer. But three times four and four times three Reasoning seem like they could be talking about different things. Application Request Justification Request do you agree or disagree with what Eddie is saying? Teacher: Rebecca, Rebecca: Well, I agree that it doesn't matter which number is first, because they both give you twelve. But I don't get what Eddie means Request for Elaboration about them saying different things. Clarification Teacher: Eddie, would you explain what you mean? Eddie: Well, I just think that like three times four can mean three groups of four things, like three bags of four apples. And four times three means four bags of three apples, and those don't seem like the same thing.Critique Reasoning Tiffany: But you still have the same number of apples, so they are the Revoicing same! Teacher: OK, so we have two different ideas here to talk about. Eddie says the order does matter, because the two orders can be used to describe different situations. So Tiffany, are you saying that three times four and four times three can't be used to describe two different situations? Unequal Participation Wizard-of-Oz Setup Tutor Student Student Problem Solving Interface Interface allows students to submit Incorrect solutions Reflection Interface Scaffolding for Collaboration • Prompts to encourage the instructionally beneficial behavior of students – Explanations – Equal contributions Explanation Prompt (Student 1 corrects the answer for the confederate learner without offering any explanation) Student1: c is 1 Auto-Prompt (to student 1): Help the student understand your correction. Student2 : what does c mean? Student1 : constant Student2 : ok Student1 : the constant in front of the variable which in our case is variable t Student2 : ok Student1 : now for the second part we use the constant function Equal Contribution Prompt (After the participant makes 5 correct problem actions without confederate student’s contribution) Auto-Prompt: It seems like the other student has not contributed lately. Why don’t you see if they need help? Student1: you want to do the other half? Student2: ok Learning effect Significant benefit for prompting on student learning with pretest as a covariate F(1,39) = 4.47, p < .05, effect size .58 Process Losses Another Domain: Earth Sciences Collaborative idea generation in the Earth Sciences domain Chinese TagHelper learns hand-coded topic analysis Human agreement .84 Kappa TagHelper performance .7 Kappa Trained models used in follow-up study to trigger interventions and facilitate data analysis Example Dialogue Individuals+ Individuals+ Feedback NoFeedback Pairs+ Feedback Pairs+ NoFeedback Speaker Text Student 1 People stole sand and stones to use for construction. VIBRANT Yes, steeling sand and stones may destroy the balance and thus make mountain areas unstable. Thinking about development of mountain areas, can you think of a kind of development that may cause a problem? Student 2 Development of mountain areas often causes problems. Student 1 It is okay to develop, but there must be some constraints. * Feedback during idea generation increases learning and idea generation productivity (except during the first 5 minutes) (Wang et al., 2007) Individuals+ Individuals+ Feedback NoFeedback Pairs+ Feedback Process Analysis Pairs+ NoFeedback Unique Ideas 12 Individuals+Feedback Individuals+NoFeedback 6 8 Pairs+Feedback Pairs+NoFeedback 0 2 4 #Unique Ideas 10 Nom+N Nom+F Real+N Real+F 0 Process loss Pairs vs Individuals: F(1,24)=12.22, p<.005, 1 sigma Negative effect of Feedback: F(1,24)= 7.23, p<.05, -1.03 sigma 5 10 15 Time Stamp 20 25 30 Process loss Pairs vs Individuals: F(1,24)=4.61, p<.05, .61 sigma Positive effect of feedback: F(1,24)=16.43, p<.0005, 1.37 sigma Deficient Help Exchange Why study help? Offering of deep help and elaborated explanations predicts post test performance (e.g., Webb et al., 2002) Help behavior mediates learning (e.g., Gweon et al., 2006; Gweon et al., 2007) Exchanging help in the context of collaborative learning increases identification with a learning community and motivation, and improves race relations (Sharan, 1980) Providing help leads to feelings empowerment for “low status” students (Elbers & Hann, 2004) How can we prompt helping behavior? DIRECT: Explicit prompts (Gweon et al., 2006) LESS DIRECT: Manipulating availability of help from problem solving environment (Gweon et al., 2007) Girls offered more help with delayed feedback from environment Boys offered more help with immediate help from the environment SUBTLE: Conversation agents inject humor (Kumar et al., 2007) Positive effects on perceived help exchange (p<.05), effective help exchange (p<.1), and Learning (p=.06) Collaborative Problem Solving Environment Jan packed several books to amuse herself on along car ride to visit her grandma. After 1/5 of the trip she had already finished 6/8 of the books she brought. How many times more books should she have brought than what she packed? Extraneous Entertainment? Coding Scheme (Gweon et al., 2007) Help provision mediates learning (Gweon et al., 2006; Gweon et al., 2007) (R) Help Requests: “Help me”, “I’m stuck”, “I don’t get it.” (P) Help Provisions: “Find the common denominator”, “Try the flip and multiply strategy” (C) Can’t help: “I don’t know”, “I’m stuck too” (D) Deny help: “ask the teacher”, “you’re an idiot”, “press the help button” (N) Nothing Examples [R] Student 1: What do we put on top of the fraction? [P] Student 2: Did you find a common denominator? <student 1 correctly finds the common denominator> [R] Student 1: I don’t get it [D] Student 2: hold on <then student 1 tried something and got negative feedback from the problem solving environment.> <finally student 1 tried something else, which was correct, and got positive feedback from the problem solving environment> [R] Student 1: Why 16? [C] Student 2: I don’t know. [R] Student 1: I don’t get it <student 2 tries something and gets negative feedback from the problem solving environment> <student 2 tries something else and gets negative feedback from the problem solving environment> <student 2 clicks on the help button> <student 1 tries something that is correct and gets positive feedback from the problem solving environment> Big findings: More help related episodes in experimental condition More episodes where people got help, and then managed to solve the problem themselves in the experimental condition Vision Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance Vision Support for collaborative learning is like training wheels Effective support allows learners to achieve better collaboration Unnecessary support can be demotivating Fading support is ideal But too little support can be detrimental as well Trained human facilitators are able to achieve the right balance Architecture for Dynamic Collaboration Support VMT-Basilica ** Students learn up to 1.25 standard deviations more when interactive support is provided in the environment. Basilica Architecture A channel filter collects all of the events that occur at the interface from all students involved in the conversation Basilica Architecture Events are then channeled to special purpose filters such as a text processing filter Other filters may keep track of time or other factors not related to student behavior Basilica Architecture Filters that are related to decision making for specific types of interventions receive notifications from the analysis filters and pass notifications on to the Actors, which are responsible to launching interventions Basilica Architecture Actors of different types produce events that are sent to the Outgoing Message Spooling Filter From there, events are sent through the Presence Actor to the interface. Issues to consider What problem are you trying to solve? Formulate analysis scheme When should you intervene? Use technologies like TagHelper and SIDE to track interaction and trigger support What should the intervention be? Technologies support like TuTalk can be used to offer Questions?