Artificial Intelligence Adv., October 1st, 2014
1. Introduction
Toyoaki Nishida
Kyoto University
Copyright © 2013 Toyoaki Nishida All Rights Reserved.
What is AI?
 Example
→ Enthiran (The Robot), 2010
AI in the movies
Year
Title
Who
Wikipedia
1968
2001: A Space Odyssey
HAL9000
http://ja.wikipedia.org/wiki/2001%E5%B9%B4%E5%AE%87%E5%AE%99%E3%81%AE%E6%97%85
1977
Star Wars
C‐3PO, R2‐D2
http://ja.wikipedia.org/wiki/%E3%82%B9%E3%82%BF%E3%83%BC%E3%83%BB%E3%82%A6%E3%82%A9%
E3%83%BC%E3%82%BA%E3%83%BB%E3%82%B7%E3%83%AA%E3%83%BC%E3%82%BA
1982
Blade Runner
Replicants
http://ja.wikipedia.org/wiki/%E3%83%96%E3%83%AC%E3%83%BC%E3%83%89%E3%83%A9%E3%83%B3%
E3%83%8A%E3%83%BC
1984
The Terminator
The terminator
http://ja.wikipedia.org/wiki/%E3%82%BF%E3%83%BC%E3%83%9F%E3%83%8D%E3%83%BC%E3%82%BF%
E3%83%BC_(%E6%98%A0%E7%94%BB)
1987
RoboCop
RoboCop (cyborg)
http://ja.wikipedia.org/wiki/%E3%83%AD%E3%83%9C%E3%82%B3%E3%83%83%E3%83%97
1993
War Games
WOPR: War Operation Plan Response
http://ja.wikipedia.org/wiki/%E3%82%A6%E3%82%A9%E3%83%BC%E3%83%BB%E3%82%B2%E3%83%BC%
E3%83%A0_(%E6%98%A0%E7%94%BB)
1994
Disclosure
Angel
http://ja.wikipedia.org/wiki/%E3%83%87%E3%82%A3%E3%82%B9%E3%82%AF%E3%83%AD%E3%83%BC%
E3%82%B8%E3%83%A3%E3%83%BC_(%E6%98%A0%E7%94%BB)
1998
Bicentennial Man
Andrew, a new NDR‐114 robot DR114
1999
The Matrix
Computer
2001
A.I. Artificial Intelligence David , achild Mecha
2002
Minority Report
Insect robots
(user interface is interesting,
too)
http://ja.wikipedia.org/wiki/%E3%83%9E%E3%82%A4%E3%83%8E%E3%83%AA%E3%83%86%E3%82%A3%
E3%83%BB%E3%83%AA%E3%83%9D%E3%83%BC%E3%83%88
2004
I, ROBOT
V.I.K.I . , Sonny
http://ja.wikipedia.org/wiki/%E3%82%A2%E3%82%A4,%E3%83%AD%E3%83%9C%E3%83%83%E3%83%88
2009
ATOM Astro boy
Astro boy
http://ja.wikipedia.org/wiki/ATOM_(%E6%98%A0%E7%94%BB)
2009
Avatar
(tele‐existence)
http://ja.wikipedia.org/wiki/%E3%82%A2%E3%83%90%E3%82%BF%E3%83%BC
2009
Surrogate
(tele‐existence)
http://ja.wikipedia.org/wiki/%E3%82%B5%E3%83%AD%E3%82%B2%E3%83%BC%E3%83%88_(%E6%98%A0
%E7%94%BB)
2014
Transcendence
(Technical singularity)
http://ja.wikipedia.org/wiki/%E3%83%88%E3%83%A9%E3%83%B3%E3%82%BB%E3%83%B3%E3%83%87%
E3%83%B3%E3%82%B9
http://ja.wikipedia.org/wiki/%E3%82%A2%E3%83%B3%E3%83%89%E3%83%AA%E3%83%A5%E3%83%BCN
http://ja.wikipedia.org/wiki/%E3%83%9E%E3%83%88%E3%83%AA%E3%83%83%E3%82%AF%E3%82%B9_(
%E6%98%A0%E7%94%BB)
http://ja.wikipedia.org/wiki/A.I.
[Nishida 2012b] Year
AI
History of AI research in contrast with ICT
ICT
1936: Turing Machine, 1947: von Neumann Computer, 1948: Information Theory, by C. Shannon and W. Weaver, 1948: Cybernetics by Wiener
1940～
Universal Turing Machine
1950～
1952‐62: Checker program by A.Samuel
1956: Dartmouth Conference
1960～
1961: Symbolic Integration program SAINT by J.Slagle
1962: Perceptron by F.Rosenblatt
1966: The ALPAC report against Machine Translation by R. Pierce
1967: Formula Manipulation System Macsyma by J.Moses
1967: Dendral for Mass Spectrum Analysis by E.Feigenbaum
1968: Mouse and Bitmap display for oN Line System (NLS) by D.C.Engelbart
1969: ARPA‐net
1971: Natural Language Dialogue System SHRDLU, by T.Winograd
1973: Combinatorial Explosion problem pointed out in The Lighthill report
1974: MYCIN by T.Shortliffe
Mid 1970’s: Prial Sketch and Visual Perceptron by D.Marr
1976: Automated Mathematician (AM) by D.Lenat
1979: Autonomous Vehicle Stanford Cart by H.Moravec
1970: ALOHAnet
1970: Relational Database Theory by E.F.Codd
1972: Theory of NP‐completeness by S.Cook and R.Karp
Mid 1970’s: Alto Machine by A.Kay and A.Goldberg
1976: Ethernet
1979: Spreadsheet Program Visicalc by D.Bricklin
1984: The CYC Project by D.Lenat
Mid 1980’s: Back‐propagation algorithm was widely used
1985: the Cybernetic Artist Aaron by H.Cohen
1986: Subsumption Architecture by R.Brooks
1989: An Autonomous Vehicle ALVINN by D.Pomerleau
1982:TCP/IP Protocol by B.Kahn and V.Cerf
Mid 1980’s: First Wireless Tag Products
1987: UUNET started the Commercial UUCP Network Connection Service
1988: Internet worm (Morris Worm)
1989: World Wide Web by T.Berners‐Lee
1989: The number of hosts on the Internet has exceeded 100,000. 1990: Genetic Programming by J.R.Koza
Early1990’s: TD‐Gammon by G.Tesauro
Mid 1990’s: Data Mining Technology
1997: DeepBlue defeated the World Chess Champion G.Kasparov
1997: The First Robocup by H.Kitano
1999: Robot pets became commercially available
1992: The number of hosts on the Internet has exceeded 1,000,000.
1994: Shopping malls on the Internet
1994: W3C was founded by T. Berners‐Lee
1997: Google Search
1998: XML1.0（eXtensible Markup Language) by W3C
1998: PayPal
2000: Honda Asimo
2001: Wikipedia.
2003: Skype / iTunes store
2004: Facebook
2005: YouTube / Google Earth
2006: Twitter
2007: Google Street View
Heuristics
- Intelligence as Clever Computing
1970～
1980～
1990～
Symbolic
Computing
1982: Fifth Generation Computer Project
- Knowledge-based Systems
Embodied & Network Intelligence
- Like human and society
2000～
2004: The Mars Exploration Rovers (Spirit & Opportunity)
Service Copying
2010～
- Machine Learning / Data Mining
2010: Google Driverless Car / Kinect
2011: IBM Watson Jeopardy defeated two of the greatest champions
2012: Siri
- from Hardware to Software
1957: FORTRAN by J.Backus
1961: Mathematical theory of Packet Networks by L. Kleinrock
1963: Interactive Computer Graphics by I.Sutherland
Exponential Growth
- Moore’s Law
- Expanding Internet World
Generic Computing Engines
- from Software to Data
Successful Topics of AI
DeepBlue (1997)
http://www.research.ibm.com/deepblue/

Large-scale Search

Knowledge-based Systems

Language, Speech, Vision

Planning

Machine Learning and Data Mining

Using AI in Creating Works of Art
IBM Watson (2011)
http://www-03.ibm.com/innovation/us/watson/index.shtml
The Mars Exploration Rovers (2003)
http://marsrovers.jpl.nasa.gov/mission/spacecraft_surface_rover.html
ALVINN （An Autonomous
Land Vehicle in a Neural
Network） on Navlab (1989)
http://dl.acm.org/citation.cfm?id=89891
Experiments in Musical Intelligence (1987)
http://arts.ucsc.edu/faculty/cope/
http://artsites.ucsc.edu/faculty/cope/mp3page.htm
AARON (1985)
http://www.kurzweilcyberart.com/
Recent Trends
Landmarks
1997: Deep Blue defeated G. Kasparov
… Source: IBM Deep Blue, wikipedia
1997: Official opening of Robocup (H. Kitano)
… Source: robocup.org, robocup.or.jp, Wikipedia
1997: Mars Pathfinder
… Source: NASA Mars Pathfinfer, Wikipedia
1999: Robot pet SONY AIBO
… Source: SONY AIBO, Wikipedia
1999: OpenCV Project
… Source: OpenCVWiki，wikipedia
2000: Honda ASIMO
… Source: Honda ASIMO, Wikipedia
2004: Mars Exploration Rovers
… Source: NASA‐JPL
2010: Google Driverless Car
… Source: Wikipedia, Sebastian Thrun’s home page, 3P
2010: Kinect
… Source: xbox.com, Wikipedia
2011: IBM Watson defeated two Jeopardy! champions
… Source: IBM Watson, Wikipedia
2011: iPhone Siri
… Source: Apple, Wikipedia
2011: Google Voice Search
… Source: Google
2012: Zen Takemiya Masaki with 4 stones … Source: Sig ECS, UEC
2012: Google Glass Project
… Source: Project Glass
2012: NTT DOCOMO’s Shabette Concier
… NTT DOCOMO
Other： Smarter than You Think (New York Times) ‐ From philosophy to science‐technology
‐ Toy problems to the real world
‐ Powerful tools
‐ Grand challenges and competitions
Conventional Society
Service producers
Corporation
Service consumers
Technology Society
Agents
Service producers
Corporation
Service consumers
AI Society
Corporation
People
Dark side of super intelligence
 Technology abuse
 Responsibility flaw
 Moral in crisis
 Over‐dependency on technology
 Technical Singularity
 End of Work
(Technological) singularity
 The day when machine intelligence surpasses human intelligence.
 IEEE Spectrum June 2008 issue
http://spectrum.ieee.org/biomedical/ethics/signs‐of‐the‐singularity
 “The AI Scenario: We create superhuman artificial intelligence (AI) in computers.”
 “The IA Scenario: We enhance human intelligence through human‐to‐computer interfaces‐‐that is, we achieve intelligence amplification (IA).”
 The fear of utopia So what?
 Dismiss as nonsense.
 Neo‐Luddism.
 Invent a better solution.
Future Direction: Cultivating Human and Social Potential
‐ Human potentials: find meaningful goals and sustain life for the achievement of these goals; ability to
‐ take action
‐ Imagine
‐ induce emphasis
‐ Sustain endeavor
‐ be ethical
‐ be positive
‐ be aesthetic
‐ Social potentials: the potential of the society as a whole being able to maintain a good balance and regularity to incorporate and foster member's activities.
‐ tolerance (or broad‐mindedness)
‐ ability to achieve a goal
‐ Conviviality
‐ well‐being and peace ‐ Fairness
‐ Diversity
‐ connectedness
Reality – Stories - Games
Interpretation, Imagination
Reality
Stories
Understanding
Gamification
Understanding
Scripts
Memory
Games
Towards Empathic Agents
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Traditional AI：
High competence
Future AI：
High empathy
DeepBlue
Entertain with a game
IBM Watson
Entertain with a game
Siri
Conversation partner
(AI that can pass
entrance exam)
Effective and affective tutor
Communicative Intelligence for Bridging People and CI
Super Intelligence
People
Communicative Intelligence
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Empathy
The ability to understand others’ emotions and/or perspectives and, often, to resonate with others’ emotional states.
or …
An affective response that is identical, or very similar, to what the other person is feeling or might be expected to feel given the context: a response stemming from an understanding of another’s emotional state or condition.
[Eisenberg 2010]
Sharing hypothesis
The more common ground is shared, the more empathy will be gained. ... the universe of discourse, first‐person view, knowledge and skills, the communication style and rituals, the value system, ...
[Nishida 2013c]
A road to empathic agents
Empathy
Sharing hypothesis
Primordial soup of conversation
Common ground
Conversational intelligence
Engagement
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Our approach: Building a Primordial Soup of Conversation Partially mechanized humans & Partially humanized machines live together and converse with each other.
Conversation environment #3
Conversation environment #2
Content provision
Training data
Acquired content
Conversation environment #1
Conversation management
‐ Conversation environment management
‐ Sensing human behaviors
‐ Controlling agents
Long‐term goal
Challenge: A robot that can participate in conversation
Not just conversational but also empathic
Conversation is a complex business
Conviviality
Trust
Social networks
Proposing
Asking
Negotiating
Eye gaze
Facial expression
Hand gesture
Posture
The Architecture of Empathic Agent?
Mind
Self
Inter cultural
Consciousness
Empathic
Social
Interactive
Judgment
Language
cognition
Memory
Imagination
Perception
Internal theater
Embodiment
Sensors
Motors
Learning skills
Evolutional system History of conversational systems development
Embodied Conversational Agents / Intelligent Virtual Human
The Knowledge Navigator
Multi‐modal dialogue systems
Speech dialogue systems
Interactional systems
Natural language dialogue systems
Conversational Systems
Natural language question answering systems
Transactional systems
Story Understanding systems
Cognitive systems
Affective Computing
t
1970
1980
1990
2000
2010
Conversational Informatics
Application
Conversational interactions
Content production
Platform
Theory
Model building
Evaluation
Measurement
Analysis
Applied Intelligence Information Processing Group
Interaction Measurement, Analysis and Modeling We aim at uncovering principles of verbal and nonverbal interactions that people engage everyday as a part of intellectual activities and developing a suite of technologies to help people benefit from conversational interactions.
Cognitive Design
We aim at designing the expressions, functions, control, and interactions of artifacts based on a physio‐cognitive approach.
Designing high‐level cognitive interaction
Annotation tool
Conversation corpus
Reconstructing 3D conversation scene Learning by imitation
Virtualization of physical space
Augmented Conversation Environment
We aim at building a smart environment that integrates conversation in physical and virtual spaces.
Eye tracker (wearable)
Cluster computer
SmartInFill
Eye tracker
Polygraph
Multiparty conversation recorder
Optical motion capture systems
IMADE: Interaction Measurement, Analysis, and Design Environment
Conversation in the physical environment
ICIE: Immersive Collaborative Interaction Environment
Conversation in the virtual environment
IMADE: a real‐world interaction measurement, analysis and design environment
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
3D conversation capture – over the shoulder
[Yano 2012]
Non‐intrusive Eye Gaze Tracking (EGT) by corneal imaging
Corneal Image Feature Matching
Problem:
Local feature correspondence + RANSAC does not work due to large noise in eye images
Corneal Imaging Camera
Scene camera
Eye camera
• Lightweight and versatile system • Appl.: Google Glass like HMD, unconstrained setups
Approach:
1. Formulate problem as registration of 3D spherical light maps of eye and scene image
2. Single point algorithm for robust alignment
Experimental settings for capturing human behaviors and physiological indices
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
ICIE – immersive collaborative interaction environment
[Nishida-Nakazawa-Ohmoto-Mohammad 2014]
Projecting the real world into the virtual world
[Mori 2011]
Virtualized physical world
[Mori 2012]
Immersive WOZ environment
Tele-operated robot
The conversation place
WOZ operator
Feedback
generation
Motion mapping
User motion sensing
Head recognition
Gesture recognition
Face model
Human body model
WOZ operating environment
[Ohmoto, Ohashi, Saiga]
Experimental setting for studying human‐robot interaction
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
iCorpusStudio
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Collaborative annotation system
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Learning by imitation
[Mohammad 2009]
Learning by imitation
Constrained Motif Discovery:
• Given a time series X(t)
find recurring patterns of length between L1 and L2 using distance
function D subject to the constraint P(t), where P(t) is an estimation of the probability that a motif occurrence exists near time step t.
P(t)
Change point discovery
likely
unlikely
[Mohammad 2009]
Learning by imitation
Robust Singular Spectrum Transform
H t   seq t  n  ;...; seq t  1 
G t   seq t  1 ;...; seq t  n  
Past
t
H (t )  U (t ) S (t )V (t )T
Future
Find optimal lP
G (t )G (t )T u g  u g
Find optimal lF
H
 i (t )  ui g , i  l F and  j  1   j   j 1
U lU lT  (t )
 i (t ) 
, i  lf
T
U lU l  (t )
G
cs i t   1   i t  i t 
T
lf
Future
Change angle
xˆ t  

i
i 1
 cs i
lf

i 1
i
~
x (t )  xˆ (t )   F (t )   P (t )   F (t )   P (t )
[Mohammad 2009]
Immersive games
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Virtual Basketball
[Lala 2013]
Tacit social signals
Intentions are dynamic and tacit
Facilitative agent that can track dynamic and tacit intentions
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Summary
1.
2.
3.
This course centers on conversations.
Why conversations? ‐> Foundation of thought and communication
Why do people converse with each other? ‐> As a part of social interaction, for creating and maintaining stories, for fun.
4. The complexity of conversations: ‐> Multi‐level, multi‐layered, polysemy, polymorphism, coordination of multi‐modality.
5. People are not only proficient in expressing ideas but also skillful in interpreting utterances, thereby they learn from each other.
6. Building empathic agents helps people effectively communicate with each other by conversations.
7. Conversational informatics addresses science and engineering of communicative intelligence.
8. Conversational intelligence can be counted as an important branch of artificial intelligence.
9. We will draw on a data‐intensive approach. 10. Conversation quantization as a foundation of data‐intensive approach to conversational informatics.
Agenda
Credits:
Will be awarded based on one or more reports on subjects given at the class.
Agenda (planned)
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Introduction (October 1st) History of Conversational System Development (October 8th)
Methodologies for Conversational System Development ‐ 1 (October 15th)
Methodologies for Conversational System Development ‐ 2(October 22nd)
Smart Conversation Space (October 29th)
Measurement, Analysis and Modeling (November 5th)
Cognitive Design (November 12th)
Computer Vision Techniques (November 19th)
Affective Computing (November 26th)
Motif Discovery (December 3rd)
Change Point Discovery and Some More Advanced Subjects (December 10th)
Aspects of Conversation ‐ 1 (December 17th)
Aspects of Conversation ‐ 2 (January 7th)
Speaking Turn Taking System (January 14th)
Synergy and Wrap up (January 28th)
For more details …
http://www.springer.com/engineering/signals/book/978‐4‐431‐55039‐6