Beating Common Sense into
Interactive Applications
Henry Lieberman, Hugo Liu, Push Singh, Barbara Barry
AI Magazine, Winter 2004
As (mis-)interpreted by Peter Clark
for Boeing KR Group
Introduction
• Claim: Commonsense applications are closer than
you think
• Problems with CommonSense (CS) applications:
– Even large KBs have sparse coverage
– Inference is unreliable
Their Common Sense KB:
Open Mind Common Sense (OMCS)
• 750k NL assertions
from 15k contributors
• ConceptNet: A
semantic net built
from these
– 20 link types
Against Question-Answering…
• Question answering is a bad CS domain:
– User expects a direct answer to all his/her questions
– System has to be right (almost) all of the time
– Got to be fast (few seconds)
• Alternative: intelligent interfaces
– Assists user when it can
– “fail soft” - user can ignore it if he/she wants
– But: Is yet another paperclip?
1. ARIA: Annotation and Retrieval
Integration Agent
• Helps annotate photos, and find
photos
– Similar to Thesaurus search
– Photos are annotated with
keywords
– a. People, places and events
are recognized in text
– b. Use the semantic net to find
“close” photos to text
– Text also adds to the net
(system learns)
• “My sister’s name is Mary”
→ “Joe –sister→Mary”
2. Detecting Moods (“affect”) in Text
“My wife left me; she took the kids and the dog”
• Approach:
– Mood keyword (e.g., “sad”)
→ mine a “small society of
linguistic models of effect”
from the KB (=?)
• Applications:
– Empathy Buddy: (purpose=?)
– Summarizing a collection of
reviews about a topic

3. Cinematic Commonsense:
Video Capture and Editing
• Videographer shoots, adds NL annotations
– E.g., “a street artist is painting a painting”
• Send annotations to KB for elaboration
– “after painting, you clean the brushes”
– “during painting, you might get paint on your hands”
• Elaborations
– suggest new shots for the videographer
– Also are stored for improved retrieval
– Can help order shots into temporal/causal
• (isn’t temporal ordering already done?)
• But: need more complex story understanding to create
effective suggestions for the filmmaker.
4. Common Sense Storytelling: StoryIllustrator
• Continuosly retreive photos relevant to user’s typing
• Use Yahoo image search, not annotations, for Web images
• CSK for query expansion, E.g., “baby” ↔ “milk”
5. Common Sense Storytelling:
OMAdventure
• Generates dungeons-and-dragons game on the fly
• E.g., in kitchen
→ what do you find in kitchen?
→ Other associated locations?
• E.g., oven
→ what can you do with an
oven?
• Hence oven, cooking are
“moves” for user.
Associated locations are
“exits”.
• User can add objects (e.g.,
“blender”) → extend KB
(“blenders are in kitchens”)
7. Common Sense Storytelling:
StoryFighter
• System and user take turns to contribute lines to a
story to get from A to B, e.g.,
– “John is sleepy” (start)
– “John is in prison” (end)
• Must avoid “taboo” words (e.g., “arrest”)
• CSK deduces consequences of an event
– “If you commit a crime, you might go to jail”
• CSK also picks obvious taboo words
8. Topic Spotting
• Task: Given speech, identify situation
• E.g., “fries”, “lunch”, “Styrofoam” → “eating in a
fast-food restaurant”
• Use Bayesian inference + ConceptNet
• Used in collaborative storytelling with kids
– Computer starts the story
– Kid continues
– Computer can’t fully understand kid’s speech, but can at
least identify the topic → generate plausible continuation
• E.g., “bedroom” → “Jane’s parents walked into the bedroom while
she was hiding under the bed”
9. Globuddy: A Tourist Phrasebook
• Type in your situation
– “I’ve just been arrested”
• It retrieves and translates
associated CS (?)
– “If you are arrested, you
should call a lawyer”
– “Bail is a payment that
allows an accused person to
get out of jail until a trial”
10. Predictive typing/phrase completion
• E.g., for a cellphone
keyboard
• Use ConceptNet to find
next word that “makes
sense”
– E.g., “train st” → “train
station”
11. Search: GOOSE and Reformulator as
Google adjuncts
• Infer user’s search goals and add keywords,e,g,:
– “my cat is sick” → “Did you mean to look for veterinarians?”
• Currently interactive. Later, will suggest better query.
12. Semantic Web
• Given user’s goals, find services that might accomplish
subgoals, e.g.,
– “Schedule a doctor’s appt” → look up directory of doctors,
check reputation, geographic lookup, lookup schedules, etc.
13. Knowledge Acquisition
• Criticism: Many OpenMind sentences are
decontextualized
– “At a wedding, bride and groom exchange rings” is
culturally specific
• → develop a prompt-based interface to have user’s
make context explicit.
Reflections
• Logic: What inferences are possible
• Commonsense: What inferences are plausible
• Qn: How well does OpenMind support this? E.g.,
– “People live in houses”
– “Things fall down rather than up”
– “Acid irritates skin”
Same for our own database…
“There is a rocket”
=?
+
Reflections (cont): Limitations
• Spottiness of subject coverage in OpenMind
• Inference is unreliable → reluctant to use it
– Need new inference methods
– E.g., “interleave context-sensitive inference with
retrieval in a breadth-first manner”
• CS suggestions may be distracting
– But trials suggest otherwise (people tolerate wrong but
plausible suggestions better than stupid ones)
Some additional thoughts…
• Domain-specific vs. domain-general applications
– Domain-specific – how much CS is needed?
• CycSecure
• Oil exploration
• etc.
– Domain-general – still need task-specific algorithm
– Unusual to find a domain- and task-general application
• “Scenario completion” is a good task
– newswire, incident reports, etc.