Video links
Aggregation:
http://www.youtube.com/watch?v=9foi342LXQE
Brian Blessed GPS:
http://www.youtube.com/watch?v=-JpKuYbJQK4
1
“Are you talking to me?”
What to say when you are talking to a robot.
Dr. Nava Tintarev
Dept. of Computing Science
University of Aberdeen
2
We are the SAsSY project
This is short for Scrutable Autonomous Systems
There are six of us:
Logician/Computer scientist on reasoning
Computer scientists on generating text from data
Computer scientists on human computer interaction
Psychologist, or rather a psycholinguist
3
We are the SAsSY project
4
Why I got into
computing…
“Yes, this system is a little bit finicky. It won’t let me
put this in directly”
“Machine at train station will not let me buy a
ticket!”
“Why is it picking this route when the other one is
about 10 miles shorter?”
5
The problem…
The system needs to tell us things
What if the system sounded like Brian Blessed (1.152.00)?
And we need to tell it some things back
Or ask it questions – “this one?”
No, not of the @?$!% kind…
6
Let’s talk about Robots….
ASIMO immediately recognizing customers' intention by a show of
hands, Honda.com, 26 June
7 2013
What’s a computer?
8
Autonomous systems
Computers that do not look like humans
But they can
‘see’ and
‘think’ (calculate) and
‘react’ (according to a program) and
‘learn’ (collect new information) and
‘talk’ (send information to) people or other computers
on their own.
9
Autonomous systems
They can do things that we cannot do
Too boring, too complicated, or maybe too dangerous
Like Fukushima
Sort of like a robot…
10
What could possibly go
wrong?
11
What could possibly go
wrong
The U.S. states of Nevada, Florida and California permit the
operation of autonomous cars .
The first license for an autonomous car was given in May 2012.
An unmanned aerial vehicle (UAV), also known as a drone, is an
aircraft without a human pilot on board.
The United States government has made hundreds of attacks on
targets in northwest Pakistan since 2004 using drones (unmanned
aerial vehicles).
Drones also used for policing and firefighting, and nonmilitary
security work, such as surveillance of pipelines.
With great power comes great responsibility…
12
The problem
Autonomous systems act on behalf of the user
Should the GPS have a mind of its own?
The system’s decisions are often opaque to the user
Why did it turn off here?
The user should be able to view and challenge decisions
“Hey car, that’s not right! Why are we turning off at the next
junction?”
Solution: Keep people in the loop. Give them explanations!
“There’s a traffic jam coming up, you’ll get home quicker
taking this country road!”
13
Pilot Authority Control of Tasks
(PACT)
Human
Monitors
PACT
level
5b
Computer autonomy
Levels of Human Machine Interface
(Modified from Taylor, 2001)
Computer monitored by
pilot
Computer does everything autonomously
5a
4b
Computer chooses action, performs it and informs
human
Computer backed up by
pilot
Computer chooses action and performs it unless
human disapproves
4a
Human
Action
Computer chooses action and performs it if human
approves
3
Pilot backed up by
computer
Computer suggests options and proposes one of them
2
Pilot assisted by computer
Computer suggests options to human
1
Pilot assisted by computer
only when required
Human asks computer to suggest options and human
selects
0
Pilot
Whole task done by human except for actual
operation
14
A solution: Explanations
From Keith, you should go to Elgin instead of Aberlour.
This is because Keith to Aberlour is blocked by snow.
15
Example: Logistics
Deliver a package from Aberdeen to Inverness
User prefers to go via Aberlour to see the Highlands
but the system routes them through Elgin.
Elgin
Aberlour
16
Put another way…
17
Two plans
A (Aberdeen - Keith - Elgin - Inverness)
B (Aberdeen - Keith - Aberlour - Inverness)
18
And a set of reasons and arguments….
From Keith – to can go to either Elgin or Aberlour
You cannot do both.
If Keith to Aberlour is impassable
Then you should not go through Aberlour.
And you should go through Elgin
Aberlour
Elgin
Snow in
Aberlour!
19
Which gives us a plan
and a dialog
Two plans:
A (Aberdeen - Keith - Elgin - Inverness)
B (Aberdeen - Keith - Aberlour - Inverness)
System suggests A
User asks why not B, which is preferable
System says that one of the actions of B is not executable
because of counter-argument (impassable)
The user re-instates plan B by giving a counter-counterargument (ploughed)
20
Discussion
User: why not Aberlour
System: Keith to Aberlour is blocked by snow
(morning report)
User: the road has just been ploughed (new
knowledge)
System: ok
System: Drive from Keith to Aberlour
21
A slight detour…
Natural Language Generation…
22
Where it gets tricky
Size: Thousands of steps, hundreds of decisions
What is possible and needed?
A person cannot drive for more than 3 hrs straight.
You can only refuel at certain places
Resources
Number of trucks a deliver company has
Multiple parties
Different delivery companies have different interests
23
Where it gets tricky
Tailoring
The driver will need different information from the
coordinator!
And different information before and during his/her
time on the road.
Information presentation
Aggregation and summaries
Graphics or text
How best to combine
24
Some of the things
we’ve done so far
25
Information presentation
Before we can ask why, we need to understand the
what…
Mostly language (English), but also graphs.
How hard is the information to understand?
26/30
Demo: plans
STRIP plans are difficult to read in standard notation
27/30
Demo: plans
But there are ways to remove redundant text using aggregation
Text is more natural and easier to read
28/30
“What have the romans
ever given us?”
How many objects can we join together?
http://www.youtube.com/watch?v=9foi342LXQE
“Load the truck. Load the van. Load the car” vs
“Load the truck, the van and the car.”
No known limit
Similarity of words likely to matter
Load the ship and the dishwasher.
Load the truck and the van.
29/30
So we are working on
Argumentation
Distributed planning
Information presentation (NLG)
User modelling
All informed by experiments with people!
30
Vision
Complex systems
Share information
Making important decisions
But we still need to know what is going on
AND add our input
Checking it with people in experiments
31
Come talk to us!
Blog: http://sassyproject.wordpress.com/
Official: http://www.scrutable-systems.org
Nava Tintarev, n.tintarev@abdn.ac.uk
32