Sakura Science Plan
CHATBOT SYSTEM IN THE SIMULATED WORLD
Student: Nguyen Minh Phuong
Hanoi University of Science and Technology, Vietnam
Lecturer: Associate Professor. INAMURA Tetsunari
1
▶ Impression
▶ Major
of NII
research
▶ Natural Language Processing in Chatbot
▶ Unity framework
▶ Integrate Chatbot and Unity System
▶ What
I have done ?
▶ Futures
Outline
2
➢ Every people friendly
➢ Comfortable study environment
➢ Have modern device for research
➢ Many good professors
➢ ...
Impression of NII
3
MAJOR RESEARCH
▶ Natural Language Processing in Chatbot
▶ Unity framework
▶ Integrate Chatbot and Unity System
4
➔ Natural language processing (NLP) is an area
of computer science and artificial
intelligence concerned with the interactions
between computers and human (natural)
languages. [Wikipedia]
➔ Virtual Assistant (or chatbot) is a software
agent that can perform tasks or services for
an individual [Wikipedia].
➔ Samples: Siri-apple, Cortana-google, ...
NLP - Chatbot
5
5 step general to process natural language:
❖ Lexical Analysis − It involves identifying and analyzing
the structure of words.
❖ Syntactic Analysis (Parsing) − It involves analysis of
words in the sentence for grammar.
❖ Semantic Analysis − It draws the exact meaning or the
dictionary meaning from the text
❖ Discourse Integration − The meaning of any sentence
depends upon the meaning of the sentence just
before it.
❖ Pragmatic Analysis − During this, what was said is reinterpreted on what it actually meant.
NLP - Chatbot
6
In chatbot system, 2 main task:
❖ Named Entity Recognize problem.
Given a stream of text, determine label of each word
about name some type of object (PERSON, LOCATION, ..)
❖ Semantic Parsing[3] problem.
Converting a natural language utterance to a logical
form: a machine-understandable representation of its
meaning.
NLP - Chatbot
7
- Deep learning model: Bidirectional Recurrent Neuron
Network [1]
- Architecture:
- Sample:
I come from Hanoi Vietnam => is LOCATION
NLP - Named Entity Recognize
8
- Deep learning model: Sequence to sequence [2]
- Architecture:
- Sample:
I come from Hanoi Vietnam => come_from ( LOCATION )
NLP - Semantic parsing
9
- Deep learning model: Sequence to sequence [2]
- Architecture:
- Sample:
I come from Hanoi Vietnam => come_from ( 'Hanoi Vietnam' )
NLP - Semantic parsing
10
I come from Hanoi Vietnam
NLP module
HUMAN
Rules base
(KB)
Chatbot system
11
Can understand and communicate
but can't do anything.
● Can’t see anything
● Can't feel the world
● Can't do physic action
Chatbot - Limitation and Difficulties
12
- Environment to simulation
the real world
- Law of physics
- Easy to access
and learning.
- Useful in Robotics
Unity Framework
13
- Make a bot in the simulated
Environment.
- Make bot do simple task by
natural language sentence
- The way for chatbot feel the
real world in the future.
Integrate Chatbot - Simulated World
14
SHRDLU: developed by Terry Winograd at MIT in 1968–1970.
In it, the user carries on a conversation with the computer, moving
objects, naming collections and querying the state of a simplified
"blocks world", essentially a virtual box filled with different blocks.
- The same:
- Limited world of bot
- The world have basic law physic
- The difference:
- SHRDLU: The system does not have any specific perspective,
but all the information about the world is accessible.
- My system: Agents are deployed in the VR to simulate a
specific perspective. The agent access to limited world.
Previous work
15
I come from Hanoi Vietnam
NLP module
HUMAN
Simulated
World
Rules base
(KB)
Integrate Chatbot - Simulated World
16
- Simple conversation:
- Player: Hi
- Robot: hello :D
- Player: what can u do
- Robot: Now, I just can
take a photo for u.
Sorry and I love u <3.
Integrate Chatbot - Simulated World
17
- Simple conversation:
- Player: ok, Can u take
a photo
- Robot: I'll take a
photo for u.
Integrate Chatbot - Simulate World
18
- Simple conversation:
- Player: What can u
see now
- Robot: This is
something i’m seeing.
- Blue box
- White box
Integrate Chatbot - Simulate World
19
- Simple conversation:
- Player: can u see blue
box
- Robot: Yes, I’m seeing
blue box
Integrate Chatbot - Simulate World
20
- Simple conversation:
- Player: pls come back
the position which u see
blue box.
- Robot: Yes, I'll come
back position which can
see blue box.
Integrate Chatbot - Simulate World
21
What I have done?
•
Make a simple chatbot system in
English.
•
Learn Unity framework and C#.
•
Integrated chatbot and Unity
system.
22
FUTURES
•
Make a chatbot system more
perfect
•
Create a chatbot which can
cognitive abilities
•
Create a robot auto learning in
Unity world and live in real world
23
● [1] Schuster et al., 1997 Bidirectional Recurrent
Neural Networks Trans. Sig. Proc., 45, 2673–2681
● [2] Sutskever et al., 2012 Sequence to Sequence
Learning with Neural Networks, NIPS’14, 3104–3112.
● [3] Robin Jia et al., 2016 Data Recombination for
Neural Semantic Parsing, CoRR, abs/1606.03622
REFERENCE
24
THANKS
25