Interaction Challenges
for Intelligent Assistants
Jim Blythe
USC Information Sciences Institute
How to build “truly useful assistants”?
Personalized, Learn, Engender trust,
Become partners
Organizer: Neil Yorke-Smith
Committee: Pauline Berry, Timothy Bickmore,
Mihai Boicu, Justine Cassell, Ed Chi, Mike Cox,
John Gersh, Jihie Kim, Jay Modi, Donald
Patterson, Debra Schreckenghost, Richard
Simpson, Stephen Smith, Sashank Varma
28 accepted papers
Topics
•
•
•
•
•
•
Trust
When to assist?
Learning
Modeling
Desktop assistants
Panel with symp. on multidisciplinary
collaboration for socially assistive robots
• Panel with intentions in intelligent systems
How To Make Users Happy
• And avoid annoying users
- Brad Myers’ invited talk
User Happiness?
Hu = f (Performance)
User Happiness?
Hu = f (Performance, Trust)
User Happiness!
Hu = f (EAssistant ENegative EPositive EValue EUser
ECorrected EBy-hand ECost EAvoided
EApparentness ECorrect-difficulty ESensible WQuality
WCommitment TBy-hand TBy-Hand-start-up TByHand-per-unit TAssistant TTraining-start-up TAssistantper-unit TInteraction-per-unit TMonitoring TCorrecting
TResponsiveness TSystem-Training TUser-training
TAverage-for-each-correction AError-rate Nunits
PPleasantness UPerceive UWhy UProvenance
UPredictability IAssistant-interfere IScreen-space
ICognitive IAppropriate-Time CAutonomy CCorrecting
SSensible-Actions SUser-models SLearning
RSocial-Presence DHand VImportance)
A Tale of Two Associates
• Pilot’s Associate
(1985-1991)
– Single Pilot
– Direct pilot interaction
with associate meant
added workload
– Design philosophy
minimized direct pilot
interaction with
associate
– Moderate user
acceptance
The Pilot is
ALWAYS in
charge.
The Effort
required of the
pilot to control
the associate
must be less
than the effort
saved by the
associate
• Rotorcraft Pilot’s Associate
(1994-1999)
– Two Pilots
– 1/3 of human activity is crew
coordination
– Design philosophy included
some direct pilot interaction
with associate
– Improved User Acceptance
Why and how to model multi-modal
interaction for a mobile robot companion
Shuyin Li & Britta Wrede
Best paper
• Tested policies with users
interacting with a robot
• Communicate pre-interaction
attention
• Need to make social remarks
with non-verbal methods
(because people tend to
reply in kind)
Biron and Barthoc
Interaction Challenges
for Agents with Common Sense
Invited talk from Henry Lieberman
• We now have several sources of common
sense knowledge, e.g. Cyc, Open Mind,
ThoughtTreasure
• Some strategies and examples of
exploiting common sense to build better
interfaces
Strategies for using common sense
in interfaces
• Find underconstrained situations
• Find situations where every little helps
• Know a little about everything, but not too
much about anything
• Make better mistakes! Not just ‘right’ and
‘wrong’, being reasonable is better
– Plausible mistakes can increase trust
• Set user expectations
Examples of interfaces using
common sense
• ARIA photo agent: more powerful
matching of tags using common sense
• Predictive typing:
“I’m having landlord problems because my
roommate was late with my r..”
• BEAM
(Gil & Chklovski)
Trust
• Openness and understanding more important
as systems become more complex.
• Methods to improve understanding:
explanations [McGuinness et al.]
• HTN metamodel [Wallace]
• Patterson: would I trust a fork? a bridge? a
space shuttle?
– predictability, understandability, similarity, liability,
social/emotional
Learning (and Trust)
• Adaptive (Learning) vs Adaptable
(Instructed by user)
– important for believability and trust
Supporting interaction in Robocare
intelligent assistant agent
Cesta et al.
Best application paper
Use of multiagent technology
The Interaction Skills
Endowed with human like I/O channels
by engineering state of the art
components
•Face: Lucia (Piero Cosi, ISTC, Pd)
•Voice: Sonic (Univ.Colorado)
•Simple Interaction Manager
The Motion Skills
Robust continuous behavior
at home with person
Multiple Intelligent Systems
Supporting interaction in Robocare
intelligent assistant agent
Integrates multiple systems to produce a
socially acceptable robotic care assistant
• Interesting DCOP solution to allow multiple
systems and guarantee coherent
behaviour
• System follows a STN to notice deviations
from expected behaviour
• Experiments in face/no-face in RoboCare
• People prefer no-face
– “less artificial”, “more integrated in the
domestic environment”
Desktop assistants
• Many papers on desktop assistants
– 6 from the Calo project
PeXA architecture
Towel todo manager
• Towel [Conley et al]:
taking an IM approach
to give access to tasks
Inspired by
Diamond
Help [Rich
et al. 06]
Did Ken sacrifice himself
to User Testing?
• Registered to give talk at AAAI Spring
symposium
Should Ken have worked on
meeting scheduling?
• Registered to give talk at AAAI Spring
symposium
• Booked another trip in same week
Should Ken have worked on
meeting scheduling?
• Registered to give talk at AAAI Spring
symposium
• Booked trip to Hawaii in same week
• Ultimate in user testing?
You decide..