Initiative in Dialogue (draft) Mark Core An Informal Definition • OED.com: to take the initiative: to take the lead, make the first step, originate some action • Sometimes referred to as “control” • dialogue initiative identifies who is leading the dialogue at any given point Why do people study initiative? • Dialogue System Builders • for some applications, system may need to reason about whether to take the initiative • systems need to take and give away initiative smoothly like humans • system acts differently when it does and does not have initiative • Linguists • open question: what factors influence when dialogue participants take initiative Outline • Defining and Annotating Initiative • Discourse Structure • Whittaker and Stenton • Strayer and Heeman • Linell et al. • What is Known about Initiative • Initiative and Current Dialogue Systems • Going Beyond Initiative Discourse Structure and Initiative • “dialogue initiative ... tracks the lead in determining the current discourse focus” (Chu-Carroll and Brown 1997, p. 263) • topic-based segmentation – who initiated the topic. Grosz and Sidner (1986) • dialogue moves – who produced an initiating move. Grosz and Sidner (1986) • Discourse grammar (syntax) • Discourse => Discourse_Segment+ • Discourse_Segment => utterance+ • Discourse grammar (semantics) • there is one unique ‘foundational’ purpose to the dialogue and each discourse segment • meant to be recognized by listener • the discourse segment purposes (DSPs) support the discourse purpose • utterances support discourse support the DSP Structural Relations • dominance: discourse is hierarchical. one DSP may be intended to provide part of the satisfaction of another • satisfaction-precedence: order of DSPs may be significant DS1 E: First you have to remove the flywheel.... loosen the two allen head setscrews... DS2 A: I can only find one screw. Where’s the other one? .... A: I can find them both now DS3 E: Use the wheelpuller. Do you know how to use it? ... E: The wheel should slide off [p. 186] Attentional State • focus spaces – contain salient entities (mentioned explicitly or implicitly) • Entities could be properties, objects, relations, as well as the discourse segment purpose • Let’s just look at objects Focus Space Stack DS1 DS1 E: First you have to remove the flywheel.... loosen the two allen head setscrews... DS2 DS2 A: I can only find one screw. Where’s the other one? .... A: I can find them both now DS3 DS3 E: Use the wheelpuller. Do you know how to use it? ... E: The wheel should slide off screw1 wheel screw puller2 flywheel screw1 screw2 FOCUS SPACE STACK Dialogue Moves and Games • Dialogue Moves • also called speech acts or dialogue acts • capture intention behind an utterance • examples • initiating: e.g., command, statement, question • responding: acknowledgement, answer, accept, reject • Dialogue Games • start with an initiating move and end when the goal of that move is accomplished or abandoned • games can be nested Dialogue Games Example [1] T: [2] S: [3] [4] T: [5] S: [6] S: [7] T: [8] [9] Do you still believe [your answer]? I suppose not. But doesn’t I=V/R? Yes. That is true. because V=IR So why doesn‘t I = 75 / the resistence? If P=IV, then you know that I = P/V. I can only equal P/R if R=V. Do you have a reason to believe that R=V? [10] S: no Complimentary Approaches • Can group dialogue games based on topic • In tutoring ‘how to measure current’ we can group dialogue games around the topics: • preparing the circuit (turn off power and break circuit) • connect leads across break (+ and – leads must be connected correctly) Whittaker and Stenton (1988) • Simpler procedure for initiative assignment • Use four dialogue moves • Assertions – “Declarative utterances used to state facts” • Commands • Questions • Prompts – “Utterances which did not express propositional content, such as Yeah, Okay, Uh-huh” • Initiative assignment • Assertion – speaker has initiative unless responding to a question • Command – speaker has initiative • Question – speaker has initiative unless responding to a question or command • Prompt – hearer has initiative Walker and Whittaker (1990) • “third person and one anaphors cross [these] boundaries extremely rarely” p. 73-74 • discourse segment boundaries marked by initiative changes Strayer and Heeman (2001) • Compared initiative (annotated with Whittaker and Stenton’s rules) to dialogue structure (as defined by Grosz and Sidner) • Whittaker and Stenton rules are more fine-grained than Grosz and Sidner segmentation • Forward Acknowledgments: listener takes initiative by completing the speaker’s utterance • Other-Contributions: listener and speaker building utterances together • Usually initiative goes back to the original speaker after the interjection Linell et al. (1988) • each utterance is ranked based on how much “they can be regarded as governing or steering the ensuing dialogue and as being governed or commanded by the preceding dialogue” p. 419 • highest rank (6): not a response in any way but requires a response from the listener • lowest rank (2): invite no response and give no more information than required Whittaker and Stenton (again) • Whittaker and Stenton (1988) • benefits • • • • relatively simple to annotate correlates well with discourse structure gives us insight into “taking” and “giving away” initiative extendable (consider additional dialogue moves) • drawbacks • does not account for answers giving more information than requested • never considers clarification requests as taking initiative • redundant if you know discourse structure Outline • Defining and Annotating Initiative • Discourse Structure • Whittaker and Stenton • Strayer and Heeman • Linell et al. • What is Known about Initiative • Initiative and Current Dialogue Systems • Going Beyond Initiative Walker and Whittaker (1990) • Distinguish between types of control shifts (i.e., speaker 2 takes initiative after...) • Abdication: speaker1 utters a prompt signaling initiative is released • Repetition/Summary: speaker1 signals the end of their contribution with a repetition or summary • Interruption: speaker1 has not invited speaker2 to take initiative • Rationale: listener knows a relevant fact, listener detects a problem in the speaker’s proposal, or listener finds the utterance ambiguous Walker and Whittaker (1990) Finance Support Pump-Phone Pump-Chat Turns/Seg 7.49 8.03 15.68 11.27 Exp-Initiative 60% 51% 91% 91% Abdication 38% 38% 45% 28% Summary 23% 27% 7% 6% Interrupt 38% 36% 48% 67% Guinn (1996) • used simulated conversations to argue that the most efficient problem-solving dialogues are those where the participant who knows the most about the current subtask takes initiative Distribution of Expert Initiative Domain Expert Control Finance 60% Computer Support 51% Pump Repair 91% TRAINS-93 39.5% Airline 58.1% Maptask 86.6% Switchboard 59.9% Socratic Tutoring 90% Didactic Tutoring 79% Initiative and Learning Gain % Student Initiative 50 40 30 20 10 0 0 5 10 15 20 25 Learning Gain Pearson’s r = -.0689, n=23, NS 30 35 40 Initiative Shifts • Chu-Carroll and Brown (1997) • Explicit cues • Discourse cues • silence, repetitions, prompts, questions, obligation fulfilled • Analytical cues • invalid action/belief, sub-optimality, ambiguity • Giving more information than requested Factors affecting initiative • • • • • • • • • Individual or joint goals? Collaborative or competitive task? Distribution of knowledge/expertise Complexity of task Task-based roles Social roles Social projection (face) Group discussion v. one-on-one Mediation/modality Outline • Defining and Annotating Initiative • Discourse Structure • Whittaker and Stenton • Strayer and Heeman • Linell et al. • What is Known about Initiative • Initiative and Current Dialogue Systems • Going Beyond Initiative Horvitz (1999) • email reading system (graphical) – based on the content of the email should the system (1) bring up the calendar, (2) ask the user first, (3) do nothing • utility-based approach – penalties for interrupting user, bringing up calendar unnecessarily, and reward for bringing up the calendar when needed Simple Speech-based Systems • Here, initiative affects the system questions: • system initiative: “What city do you want to leave from?” • user initiative: “How may I help you?” • initiative also affects how the system deals with extra information e.g., S: What city do you want to leave from? U: LA on November 17th • system initiative: ignores extra info • mixed/user initiative: processes all information • the major factor is speech recognition performance Tutorial Dialogue Systems • Tend to be system initiative although sometimes students can ask questions about definitions: “What is a CPU?” • EDGE system (Cawsey 1989) • if user question is related to a presentation goal, system will request that the user wait • Duke Programming Tutor (Keim et al. 1997) • topics picked on basis of • • • • • believed student understanding importance of topic distance from current topic how many times topic has been discussed student interest Outline • Defining and Annotating Initiative • Discourse Structure • Whittaker and Stenton • Strayer and Heeman • Linell et al. • What is Known about Initiative • Initiative and Current Dialogue Systems • Going Beyond Initiative Tutorial Dialogue Systems • (Core et al. 2003) study showed no relationship between initiative and learning • Shah (1997) found more student initiative in students’ first sessions • Graesser and Person (1994) • in the first half of a course, found a negative correlation between number of student questions and exam scores • in second half of course, found a positive correlation between exam scores and proportion of student questions that were deep-reasoning and knowledge-deficit questions Task Initiative • Who has initiative can be quite independent of what’s happening with the task (e.g., putting together the water pump, learning physics) • (Chu-Carroll and Brown 97) & (Jordan and Di Eugenio 97) • If the dialogue concerns a task, it is possible to lead the dialogue but not contribute to the task • Chu-Carroll and Brown separate these ideas into dialogue initiative and task initiative Chu-Carroll and Brown Example S: I want to take NLP to satisfy my seminar course requirement Who is teaching NLP? A1: Dr. Smith is teaching NLP A2: You can’t take NLP because you haven’t taken AI, which is a prerequisite for NLP. A3: You can’t take NLP because you haven’t taken AI, which is a prerequisite for NLP. You should take distributed programming to satisfy your requirement, and sign up as a listener for NLP. Simple Task Initiative • MIMIC (Chu-Carroll and Nickerson 2000) • simple spoken dialogue system (for movie information) • task initiative means system gives directions • e.g., “Please say the name of the movie or theatre or town you would like information about” • task initiative means system makes suggestions • e.g., “Terminator is not playing at <theatre> Terminator is playing at <alternative theatre> at <time1> • Mixed-initiative MIMIC system outperformed systeminitiative MIMIC (better user satisfaction and task efficiency) Chu-Carroll and Nickerson (2000) • Mixed-initiative MIMIC reasons about cues • TakeOverTask (user gives more info than requested) • NoNewInfo • InvalidAction/InvalidActionResolved • AmbiguousAction/AmbiguousActionResolved Future Work • “Although a system needs to reason about how helpful it needs to be, it is unclear whether this can be done through a single variable [task initiative] that is tied to dialogue initiative” (Strayer and Heeman p. 7 of pdf) • How to apply this principle to more complex domains such as tutoring • student model contains all facts about domain • student has task initiative if he demonstrates knowledge of such a fact • also if student recognizes their own error • may want to model hints which to various degrees give away the answer