Natural Language Agents
Why is it so hard?
The Turing Test
Turing’s Prediction
1950 Turing predicted that in about fifty years "an
average interrogator will not have more than a 70
percent chance of making the right identification
after five minutes of questioning".
Was Turing Right?
Let’s try a conversation with Alice :
Modern Language Systems
• WikiAnswers
• Newsblaster
Machine Translation
A Babelfish example from about 2004:
Austin Police are trying to find the person responsible for robbing a
bank in Downtown Austin.
El policía de Austin está intentando encontrar a la persona
responsable de robar un banco en Austin céntrica.
The police of Austin is trying to find the responsible person to rob a
bank in centric Austin.
Machine Translation
A Google Translate example in 2008:
Austin Police are trying to find the person responsible for robbing a
bank in Downtown Austin.
De Policía de Austin están tratando de encontrar a la persona
responsable de robar un banco en el centro de Austin.
Austin Police are trying to find the person responsible for robbing a
bank in downtown Austin.
http://translate.google.com/translate_t?hl=en#
Machine Translation – Another Old One
A Florida teen charged with hiring an undercover policeman to shoot
and kill his mother instructed the purported hitman not to damage
the family television during the attack, police said on Thursday.
Un adolescente de la Florida cargado con emplear a un policía de
la cubierta interior para tirar y para matar a su madre mandó a
hitman pretendida para no dañar la televisión de la familia durante
el ataque, limpia dicho el jueves.
An adolescent of Florida loaded with using a police of the inner
cover to throw and to kill his mother commanded to hitman tried not
to damage the television of the family during the attack, clean said
Thursday.
Machine Translation – 2008
A Florida teen charged with hiring an undercover policeman to shoot
and kill his mother instructed the purported hitman not to damage
the family television during the attack, police said on Thursday.
Un adolescente de Florida acusado de contratar a un policía
encubierto para disparar y matar a su madre encargó a la supuesta
Hitman no dañar a la familia de televisión durante el ataque, la
policía dijo el jueves.
A Florida teenager accused of hiring an undercover police to shoot
and kill his mother instructed the purported Hitman no harm to the
family television during the attack, police said on Thursday.
Machine Translation – Today
A Florida teen charged with hiring an undercover policeman to shoot
and kill his mother instructed the purported hitman not to damage
the family television during the attack, police said on Thursday.
Un adolescente de Florida acusado de contratar a un policía
encubierto para disparar y matar a su madre instrucciones al
presunto asesino a sueldo de no dañar el televisor de la
familia durante el ataque, dijo la policía el jueves.
A Florida teenager accused of hiring an undercover policeman to
shoot and kill his mother instructed the alleged murderer paid not to
damage the family television during the attack, police said Thursday.
Machine Translation
I have just returned from a visit to my landlord--the solitary neighbour that I
shall be troubled with. This is certainly a beautiful country! In all England, I do
not believe that I could have fixed on a situation so completely removed from
the stir of society. A perfect misanthropist's heaven: and Mr Heathcliff and I are
such a suitable pair to divide the desolation between us. A capital fellow! He
little imagined how my heart warmed towards him when I beheld his black eyes
withdraw so suspiciously under their brows, as I rode up, and when his fingers
sheltered themselves, with a jealous resolution, still farther in his waistcoat, as I
announced my name.
Acabo de regresar de una visita a mi dueño - el solitario vecino que voy a ser
perturbada con. Esta es, sin duda, un hermoso país! En Inglaterra todos, no creo
que me podría haber fijado en una situación tan completamente eliminado de la
conmoción de la sociedad. Un perfecto misanthropist del cielo: y el señor
Heathcliff y yo somos una pareja adecuada para dividir la desolación entre
nosotros. Una capital compañeros! El poco imaginar cómo mi corazón caliente
hacia él cuando vi sus ojos negro retirar de manera sospechosa en virtud de sus
cejas, como ya he cabalgó, y cuando sus dedos protegidos ellos mismos, con un
celoso de resolución, todavía más lejos en su chaleco, como ya he anunciado mi
nombre .
Machine Translation
I have just returned from a visit to my landlord--the solitary neighbour that I
shall be troubled with. This is certainly a beautiful country! In all England, I do
not believe that I could have fixed on a situation so completely removed from
the stir of society. A perfect misanthropist's heaven: and Mr Heathcliff and I are
such a suitable pair to divide the desolation between us. A capital fellow! He
little imagined how my heart warmed towards him when I beheld his black eyes
withdraw so suspiciously under their brows, as I rode up, and when his fingers
sheltered themselves, with a jealous resolution, still farther in his waistcoat, as I
announced my name.
I just returned from a visit to my landlord - the solitary neighbor that I shall
be troubled with. This is certainly a beautiful country! In all England, I do
not think I could have set in a situation so completely removed from the
commotion of society. A perfect misanthropist of heaven: and Mr. Heathcliff
and I are a couple suitable to divide the desolation between us. A capital
fellow! The little imagined how my heart warm to him when I saw his black
eyes withdraw so suspicious by virtue of his eyebrows, as I rode, and
when his fingers protected themselves with a jealous resolution, still farther
in his waistcoat, as I announced my name.
Machine Translation
I have just returned from a visit to my landlord--the solitary neighbour that I
shall be troubled with. This is certainly a beautiful country! In all England, I do
not believe that I could have fixed on a situation so completely removed from
the stir of society. A perfect misanthropist's heaven: and Mr Heathcliff and I are
such a suitable pair to divide the desolation between us. A capital fellow! He
little imagined how my heart warmed towards him when I beheld his black eyes
withdraw so suspiciously under their brows, as I rode up, and when his fingers
sheltered themselves, with a jealous resolution, still farther in his waistcoat, as I
announced my name.
私 は私の家主への訪問から帰国している-は、孤独な隣人は私に悩まされるもの
とする。これは確かに美しい国です！すべてのイングランドでは、私は完全に
社会 の状況をかき立てるから削除に固定している可能性があるとは思わない。
完璧な人間嫌いの天国： Heathcliff氏と私たちの間の荒廃に分割するような適切
なペアがあります。資本やつ！私が乗った彼は少し私の心は彼に向けてどのよ
うに私は彼の目 は怪しい黒色のまゆの下で撤退を見た暖め、想像が自分の指を
保護、嫉妬の分解能では、まだ遠く彼のチョッキで、私は自分の名前を発表。
Machine Translation
I have just returned from a visit to my landlord--the solitary neighbour that I
shall be troubled with. This is certainly a beautiful country! In all England, I do
not believe that I could have fixed on a situation so completely removed from
the stir of society. A perfect misanthropist's heaven: and Mr Heathcliff and I are
such a suitable pair to divide the desolation between us. A capital fellow! He
little imagined how my heart warmed towards him when I beheld his black eyes
withdraw so suspiciously under their brows, as I rode up, and when his fingers
sheltered themselves, with a jealous resolution, still farther in his waistcoat, as I
announced my name.
I returned from a visit to the landlord is - a lonely neighbor and I are plagued by.
This is certainly a beautiful country! In all England, I completely removed from
society to stir up the situation may be fixed and I do not think there. I do not like a
perfect paradise for human beings: Heathcliff's torn between our appropriate to
split into pairs. Capital fellow! He got me a little bit in my heart for him how I'm
suspicious of his black eyes under the withdrawal eyebrows looked warm and
protect the fingers of his imagination, a jealous resolution, In his waistcoat far
yet, I announced my name.
Going Both Ways
Notice that these applications require that we process
language in two directions:
•Understanding
•Generation
But also notice that it is possible to do well without going
through any meaning representation.
When Meaning is Critical
English:
Put the kid’s cereal on the bottom shelves.
Java
import java.util.ArrayList;
public class GroceryStore
{
private int[][][] shelves;
private ArrayList products;
public void placeProducts(String productFile)
{ FileReader r = new FileReader(productFile);
GroceryItemFactory factory = new GroceryItemFactory();
while(r.hasNext())
products.add( factory.createItem(r.readNext()));
ThreeDLoc startLoc;
GroceryItem temp;
for(itemNum = 0; itemNum < products.size(); itemNum++)
{ temp = (GroceryItem)(products.get(itemNum))
startLoc = temp.getPlacement(this);
shelves[startLoc.getX()][startLoc.getY()][startLoc.getY()]=
tempgetIDNum();
}
}
}
Java, Continued
public class ChildrensCereal
{
private static final int
private static final int
private static final int
extends GroceryItem
PREFERRED_X = -1;
PREFERRED_Y = 0;
PREFERRED_Z = 0;
public ThreeDLoc getPlacement(GroceryStore store)
{
ThreeDLoc result = new ThreeDLoc();
result.setX(store.find(this));
result.setY(PREFERRED_Y);
result.setZ(PREFERRED_Z);
return result;
}
}
It’s All about Mapping
What Are We Going to Map to?
English:
Do you know how much it rains in Austin?
The database:
Months
Month
Days
RainfallByStation
Stations
year
station
month
City
station
rainfall
English:
What is the average rainfall, in Austin,
in months with 30 days?
SQL:
SELECT Avg(RainfallByStation.rainfall) AS
AvgOfrainfall FROM Stations INNER JOIN
(Months INNER JOIN RainfallByStation
ON Months.Month =
RainfallByStation.month)
ON Stations.station =
RainfallByStation.station
HAVING (((Stations.City)="Austin")
AND ((Months.Days)=30));
Doing it Without Meaning
How much does it rain in Austin?
WikiAnswers
So Where’s the Holy Grail?
Can we just bite the bullet, solve all the hard problems
and be done with it once and for all?
Ambiguity – the Core Problem
• Time flies like an arrow.
Ambiguity – the Core Problem
● Chris
likes the girl with the cat.
Ambiguity – the Core Problem
● Chris
likes the girl with the cat.
Ambiguity – the Core Problem
● Chris likes the girl with the cat.
● Chris shot the bear with a rifle.
Ambiguity – the Core Problem
● Chris
likes the girl with the cat.
● Chris shot the bear with a rifle.
Ambiguity – the Core Problem
● Chris
likes the girl with the cat.
● Chris shot the bear with a rifle.
● Chris shot the bear with a rifle.
Ambiguity – the Core Problem
• I saw a Boeing 747 flying to New York.
Ambiguity – the Core Problem
• I saw the Grand Canyon flying to New York.
Ambiguity – the Core Problem
• I know more beautiful women than Kylie.
Ambiguity – the Core Problem
• The boys may not go out for recess.
Ambiguity – the Core Problem
• You can have potatoes or rice and beans.
or
Ambiguity – the Core Problem
• You can have potatoes or rice and beans.
or
and
Ambiguity – the Core Problem
• You can have potatoes or rice and salad.
or
and
Ambiguity – the Core Problem
• Is there water in the fridge?
Ambiguity – the Core Problem
• Who cares?
Language Requires Knowledge
Headlines
Beijing Stops Construction for Olympics
Language Requires Knowledge
Headlines
Beijing Stops Construction for Olympics
Hospital Pays $100,000 to Overdose Victim’s Family
Language Requires Knowledge
Headlines
Beijing Stops Construction for Olympics
Hospital Pays $100,000 to Overdose Victim’s Family
TV ads boost eating of obese children.
One Kind of Knowledge
Concepts like:
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
Chair?
The bottom line?
A Mapping Function for NL Understanding
• Morphological analysis and POS tagging
* The womans goed home.
A Mapping Function for NL Understanding
• Morphological analysis and POS tagging
* The womans goed home.
I hit the bag.
A Mapping Function for NL Understanding
• Morphological analysis and POS tagging
• Syntactic analysis (Parsing)
* Fishing went boys older
A (Tiny) Grammar of English
S  NP VP
NP  the Nominal | a Nominal | Nominal |
ProperNoun | NP PP
Nominal  N | Adjs N
N  cat | dogs | bear | girl | chocolate | rifle
ProperNoun  Chris | Fluffy
Adjs  Adj Adjs | Adj
Adj  young | older | smart
VP  V | V NP | VP PP
V  like | likes | thinks | shots | smells
PP  Prep NP
Prep  with
Structure in English
S
NP
VP
Nominal
Adjs
V
N
Nominal
Adj
the
smart
NP
N
cat
smells
chocolate
Why Cats Paint
Why Paint Cats
The Importance of Parsing Even When
We’re Not Doing Full Understanding
Find me all the:
Lawyers whose clients committed fraud
vs
Lawyers who committed fraud
vs
Clients whose lawyers committed fraud
The Dictionary is Important
Mary gave Lucy the food.
The Dictionary is Important
Mary gave Lucy the food.
* Mary donated Lucy the food.
The Dictionary is Important
Mary demanded a raise.
The Dictionary is Important
Mary demanded a raise.
* Mary asked a raise.
The Dictionary is Important
Mary demanded a raise.
* Mary asked a raise.
Mary asked for a raise.
The Dictionary is Important
Mary demanded a raise.
* Mary asked a raise.
Mary asked for a raise.
* Mary demanded for a raise.
Recall the Parsing Problem
● Chris
likes the girl with the cat.
● Chris shot the bear with a rifle.
● Chris shot the bear with a rifle.
A Harder One
John saw a boy and a girl with a red wagon with one blue
and one white wheel dragging on the ground under a tree
with huge branches.
8064 different parse trees
A Mapping Function for NL Understanding
• Morphological analysis and POS tagging
• Syntactic analysis (Parsing)
• Extracting meaning (Semantics)
Colorless green ideas sleep furiously.
Semantics: The Meaning of Words
Getting it right for the target application:
month  RainfallByStation.month
Semantics: The Meaning of Words
Dealing with ambiguity:
spring 
Semantics: The Meaning of Words
Dealing with ambiguity:
spring 
or
Semantics: The Meaning of Words
Dealing with ambiguity:
spring 
or
or
Semantics: The Meaning of Words
Dealing with ambiguity:
spring 
stamp 
or
or
Semantics: The Meaning of Words
Dealing with ambiguity:
spring 
stamp 
or
or
or
Semantics: The Meaning of Phrases
Semantics is (mostly) compositional.
Olive oil
(oil (made-from olives))
Another One
Peanut oil
(oil (made-from peanuts))
Another One
Coconut oil
(oil (made-from coconut))
Another One
Sesame oil
(oil (made-from sesame))
But What About This One?
Baby oil
But What About This One?
Baby oil
(oil (used-on baby))
And Another One
Cooking oil
(oil (used-for cooking))
And Another One
Riding jacket
Leather jacket
Letter jacket
Rain jacket
Idioms Don’t Work This Way
• I’m going to give her a piece of my mind.
• He bent over backwards to make the sale.
• I’m going to brush up on my Spanish.
Putting Phrases Together
Bill cooked the potatoes.
The potatoes cooked in about an hour.
The heat from the fire cooked the potatoes in 30 minutes.
(cooking-event (agent
)
(object
)
(instrument
)
(time-frame
)
Putting Phrases Together
Bill cooked the potatoes.
The potatoes cooked in about an hour.
The heat from the fire cooked the potatoes in 30 minutes.
(cooking-event (agent
)
(object
)
(instrument
)
(time-frame
)
* Bill and the potatoes cooked.
A Mapping Function for NL Understanding
• Morphological analysis and POS tagging
• Syntactic analysis (Parsing)
• Extracting meaning (Semantics)
• Putting it all in context
My cat saw a bird out the window.
It batted at it.
A Mapping Function for NL Understanding
• Morphological analysis and POS tagging
• Syntactic analysis (Parsing)
• Extracting meaning (Semantics)
• Putting it all in context
My cat saw a bird out the window.
It had almost totally frosted over but …
A Mapping Function for NL Understanding
• Morphological analysis and POS tagging
• Syntactic analysis (Parsing)
• Extracting meaning (Semantics)
• Putting it all in context
• Coherence
Coherence
Winnie doesn’t like melted ice cream. He always dreads August.
* Winnie doesn’t like melted ice cream. He always dreads January.
Winnie wanted to go to the store. He went to find Christopher Robin.
* Winnie wanted to go to the store. He started leafing through a new
book.
Winnie walked into the room. Christopher Robin looked up and
smiled.
* Winnie walked into the room. The earth rotates around the sun.
Coherence
Winnie doesn’t like melted ice cream. He always dreads August.
* Winnie doesn’t like melted ice cream. He always dreads January.
Winnie wanted to go to the store. He went to find Christopher Robin.
Winnie wanted to go to the store. He started leafing through a new
book instead.
Winnie walked into the room. Christopher Robin looked up and
smiled.
* Winnie walked into the room. The earth rotates around the sun.
Conversational Postulates
Grice’s maxims:
• The Maxim of Quantity:
Be as informative as required.
Don’t be more so.
• The Maxim of Quality:
Do not say what you believe to be false.
Do not say that for which you lack sufficient evidence.
• Maxim of relevance: Be relevant
• Maxim of manner:
Avoid obscurity of expression
Avoid ambiguity
Be brief.
Be orderly.
Conversational Postulates and Scalar Implicature
A: Have you done the first math assignment yet?
B: I’m going to go buy the book tomorrow.
Another Example of Scalar Implicature
A: When did you get home last night?
B: I was in bed by midnight.
Conversational Postulates and Inference
A: Joe doesn't seem to have a girl-friend these days.
B: He's been going to Dallas a lot lately.
Conversational Postulates and Inference
A: Let’s go to the movies tonight.
B: I have to study for an exam.
Conversational Postulates and Inference
Reviewer of new book:
It is well-bound and free of typographical errors.
Conversational Postulates and Inference
Writer of recommendation letter for an important new job:
He comes to work every day and is well-liked.
Conversational Postulates and Inference
A: What do you think of my new dress?
B: It’s interesting.
Sometimes it is Very Subtle
A Bangkok dry cleaner asks its customers to, “Drop your
trousers here for best results.”
Sometimes it is Very Subtle
A Bangkok dry cleaner asks its customers to, “Drop your
trousers here for best results.”
A Norwegian cocktail lounge asks, “Ladies are requested
not to have children in the bar.”
Sometimes it is Very Subtle
A Bangkok dry cleaner asks its customers to, “Drop your
trousers here for best results.”
A Norwegian cocktail lounge asks, “Ladies are requested
not to have children in the bar.”
In the window of a Barcelona travel agency, “Go away.”
Sometimes it is Very Subtle
A Bangkok dry cleaner asks its customers to, “Drop your
trousers here for best results.”
A Norwegian cocktail lounge asks, “Ladies are requested
not to have children in the bar.”
In the window of a Barcelona travel agency, “Go away.”
In a laundry in Rome, “Ladies, leave your clothes here
and spend the afternoon having a good time.”
Sometimes it is Very Subtle
A Bangkok dry cleaner asks its customers to, “Drop your
trousers here for best results.”
A Norwegian cocktail lounge asks, “Ladies are requested
not to have children in the bar.”
In the window of a Barcelona travel agency, “Go away.”
In a laundry in Rome, “Ladies, leave your clothes here
and spend the afternoon having a good time.”
Underpaid guards in a Budapest zoo, “Please do not feed
the animals. If you have any suitable food, give it to the
guards on duty.”
Spoken Language
1
0.8
0.6
Thedis Four transform of a real val sig is con sym
me
ju
ued nal
crete ier
gate
tric
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
0
2
4
6
8
10
12
4
x 10
Speech Understanding
http://www.nuance.com/talk/
Text to Speech
http://www.research.att.com/~ttsweb/tts/demo.php
Text-To Speech
http://www.naturalreaders.com/?gclid=CJW_85uCwZUCFQEuxwoddiru0g
The horse raced past the barn fell down.
A Recent Loebner Prize Winner
> Why do you think people want to go to the moon?
Jabberwacky: You don't. Plastic and metal, perhaps. No
flesh.
> Can you imagine life before plastic?
Jabberwacky: Can you?
Sure. Milk came in glass bottles.
Jabberwacky: You can come here whenever you want. I will
welcome you.