A Computational Mental Lexicon Model based on Principles of
Information Processing in Human Brain
Kichun Nam, Dept. of Psychology, Korean University
Heui-Seok Lim, Dept. of Information & Communication, Cheonan University
Abstract: We proposed a computational mental
lexicon model which can explain some linguistic
effects in human language processing. The proposed
model is based on modified trie data structure of
which nodes are sorted by phoneme frequency. The
model can explain frequency effect, length effect,
lexical status effect, and similarity effect with easy
because of its inherent characteristics of trie data
structure. In addition, when we trained the model
with 10 million Eojeol size corpus, the correlation
between lexical decision time and frequency of input
lexical was very similar that of human language
processing.
Background:
Researches
on
knowledge
representation and its structure of the mental lexicon
play very important role in development of
computational model in human language processing.
There are some findings that are consistently
obtained in human language processing such as
frequency effect, length effect, lexical status effect,
and similarity effect. Frequency effect is that highly
frequent word is accessed more rapidly than low
frequent word. Length effect is that longer word
requires more time to be retrieved than shorter word.
Lexical status effect means that non-word takes much
time to be retrieved than normal word. Finally
similarity effect is that non-word similar to normal
word takes much time in lexical decision test. We
claim that a computational mental lexicon model
should explain the above linguistic characteristics. In
this research, we proposed a new computational
mental lexicon model and showed the validity of our
model through simulation experiment.
The proposed model: We use trie data structure as
basic structure for our model. The origin of the name,
trie is from the middle section of the word
"reTRIEeval", and this origin hints on its usage. The
trie data structure is based on two principles: a fixed
set of indices and hierarchical indexing. The first
requirement is usually met when we can index
dictionary items by the alphabet. Each of the nodes'
elements may point to another 26-element node, and
so on. One of the advantages of using the trie as a
basic structure for a computational mental lexicon is
that length effect, lexical status effect, and similarity
effect are explained with easy through the inherent
characteristics of its data structure. But it is hard to
explain frequency effect with general trie structure.
We proposed a modified trie structure called
frequency based trie(FB-trie) which enables to
explain frequency effect as well as other linguistic
effects. The proposed FB-trie is a trie which satisfies
the following requirements.
1) The structure of a node is linked list structure.
2) The alphabet in a node is a set of Korean
phonemes.
3) The alphabet elements in a node are sorted by
descending order of frequencies of the alphabets
in a Korean corpus.
Nodes in the FB-trie are sorted by descending order
of frequencies of the alphabets as described in the
above the 3rd requirement. The 3rd requirement is
needed to model frequency effect which means more
frequent words are accessed more rapidly by visiting
the minimal number of nodes. We tried to model
language proficiency of Korean by adjusting the size
of training corpus in indexing the FB-trie; large
corpus for proficiency of an adult or an expert and
small for that of an infant or novice.
Results of researches: Our experiment was focused
on whether the proposed model can explain
frequency effect because FB-trie is a kind of trie and
it can explain the other linguistic effect naturally. We
used three different size of corpora as training data; a
million Eojeol size, 5 million Eojeol size, and ten
million Eojeol size. We use the same test data in [2]
to compare the performance of the proposed model
and that of human. The experiment was very
promising that correlation between frequency and
lexical access time was very similar to that of human
language processing. Especially, results with ten
million Eojeol corpus, the similarity of the correlation
is the most. The reason, we think, is that the training
data represents real world text more appropriately as
the size of training data is increased
[1] Marcus Taft, Reading And The Mental Lexicon,
Essays in Cognitive Psychology, Lawrence Erlbaum
Associates, Publishers, 1991.
[2] 남기춘, 서광준, 최기선, 한글 단어 재인에
서의 단어 길이 효과, 한국인지과학회지:실험
및 인지, Vol. 9, No. 2, 1-18, pp. 1-18, 1997.
A Computational Mental Lexicon Model based on Principles of Information Processing in Human Brain
87
Functional Neuroanatomical Study of Inhibitory ResponseControl: Activation and Neurochemical PET Brain Imaging
Sang Eun Kim, Department of Nuclear Medicine, Seoul National University Bundang Hospital
Abstract: We developed the technologies for
constructing neural networks involved in the
inhibitory response-control by positron emission
tomography
(PET)
brain
mapping
and
neuropsychological studies, which aimed at the
ultimate goal of the project, which is “To construct
the neuroanatomical and neurochemical circuits
involved in the inhibitory response-control and to
develop
an
integrated
neuroanatomicalneurochemical cognitive model for the inhibitory
response-control”.
Background: Inhibitory response-control is an
important cognitive function for the efficient
information processing. Thus, the understanding of
neural networks involved in the process of inhibitory
response-control is essential for the development of
human-like artificial intelligence system. Because
neurochemical factors are involved in cognitive
function, it is necessary to develop a cognitive model
reflecting both neurochemical and neuroanatomical
factors. However, neurochemical approaches to the
cognitive mechanisms are very restricted. This
project aims to construct neuroanatomical and
neurochemical circuits involved in inhibitory
response-control and to develop an integrated
neuroanatomical-neurochemical cognitive model for
inhibitory response-control.
ields of Research: This project consists of
neuroanatomical mapping studies using O-15 water
and fluorodeoxyglucose (FDG) PET, neurochemical
mapping studies mainly focusing on the dopamine
system
using
neurotransmitter/receptor
PET,
neuropsychological evaluation and behavioral studies
for the proper interpretation of the mapping studies,
and the development of neuropsychological tasks for
the assessment of inhibitory response-control
function. Based on these studies, an integrated
neuroanatomical-neurochemical cognitive model for
the inhibitory response-control will be developed.
Contents of Research Fields:
1. Mapping of regional cerebral glucose metabolism
associated with normal and abnormal processes of
inhibitory response-control using FDG PET in
healthy controls and patients with attention deficit
hyperactivity disorder (ADHD), obsessive
compulsive disorder (OCD), and frontotemporal
dementia (FTD)
2. Development of neuropsychological tasks and
rating scales for the assessment of inhibitory
response-control function
3. Study of the central dopaminergic circuitry
associated with rewarded motor learning through
the mapping of dopamine release with
[11C]raclopride PET
Results of Researches:
1. Construction of a functional neuroanatomical
circuit associated with the inhibitory responsecontrol through the mapping of abnormal
metabolism in the frontal lobe, which plays an
important role in inhibitory response-control and
executive functions, in patients with ADHD, OCD,
and FTD (Fig. 1)
2. Development of inhibitory CPT and modified
Go/NoGo tasks, which proved useful for the
evaluation of the inhibitory response-control
function.
3. Standardization of the Korean version of CAARS
for the screening of adult ADHD
4. Mapping of dopamine release evoked by rewarded
motor learning as a basis for the construction of the
central dopaminergic circuitry associated with
inhibitory response-control (Fig. 2)
Fig. 1. rCMRglu mapping in patients with ADHD and
OCD
Fig. 2. Mapping of dopamine release evoked by
rewarded motor learning
Functional Neuroanatomical Study of Inhibitory Response-Control: Activation
and Neurochemical PET Brain Imaging
88
Knowledgebase Prototype Construction and its Application for
Human Knowledge Processing Modeling
Key-Sun Choi, Department of Computer Science & KORTERM, KAIST
Abstract The goal of the project is to construct a
prototype of a domain-specific knowledgebase for
modeling human knowledge processing. We perform
research about knowledge extraction, event relation
representation and knowledge retrieval and
transformation. For its application, we construct
question and answering system. In the second phase,
we focus on knowledge extraction from documents
and
modeling
knowledge
retrieval
and
transformation.
generality and efficiency for knowledge retrieval and
construct knowledge. In the representation model,
knowledge is represented by concept hierarchy,
domain model, and event network
Background: Information and knowledge are more
and more increasing. Moreover, they are reproduced
by human activity. The point of human activity is to
analyze information written in texts and to formalize
it into knowledge. Human brain is the system that
performs language understanding, knowledge
acquisition, and knowledge representation effectively.
A system for storing and retrieving knowledge is
necessary to acquire and represent knowledge. For
language understanding, a multi-level language
processing system that uses various knowledge
resources is needed. This project aims at the system
that mirrors human brain processing for language
understanding, knowledge acquisition and knowledge
representation
3. Modeling for Knowledge retrieval and Knowledge
transformation: Design a knowledge retrieval model
and a knowledge transformation model.
Mirroring
Texts
Language
Understanding
Language
Resources
Knowledge
Acquisition
Texts reflecting
Human knowledge
World
knowledge
Domain-Specific
Knowledge
Knowledge
Retrieval
Personal
Knowledge
Knowledge base
Knowledge
Transformation
Texts containing new
knowledge
Human Knowledge
2. Tools for Knowledge acquisition: Develop tools
for knowledge acquisition and acquire knowledge
about 1,000 nodes from the domain specific texts.
Knowledge acquisition tool is composed of event
extractor, concept mapper, and domain model
generator.
4. Modeling for question answering system: Design a
question answering (QA) system using knowledge
retrieval and transformation model. The QA system
generates answer using knowledge acquired from
domain specific text.
5. Generalization:
Generalize
knowledge
acquisition/retrieval/transformation model for other
domain.
Results of Researches: In the second phase, we
automatically extract knowledge from medical
domain texts - medical encyclopedia (1,000
knowledge nodes are extracted). The acquired
knowledge is stored in database with knowledge
representation format. Knowledge retrieval and
transformation model also designed. A question and
answering (QA) system based on the knowledge
retrieval and transformation model is developed and
is tested using the extracted 1,000 knowledge nodes.
Our QA system retrieves knowledge based on
similarity between question and extracted knowledge
and automatically generates an answer based on
scripts..
Alternative ego
Fig. 1. System architecture
Fields of Research:
To perform research, sentence-level semantic
categorization, information extraction, representing
event relation, generation, human brain processing
based question answering system will be necessary.
Contents of Research Fields:
1. Research on a knowledge representation format:
Design a knowledge representation model that has
Knowledgebase prototype construction and its application for human knowledge processing modeling
89
Intelligent Visual System Design Based on Text Extraction in
Natural Scene Images
Yeong-Woo Choi, Department of Computer Science, Sookmyung Women’s University
Abstract: In this research we develop text extraction
methods on natural scene images relatively independent
to text font style, location, orientation, size, color and
background complexities. And we will apply the
developed methods to design intelligent visual systems
with speed and accuracy of minimizing user’s
inconveniences. In the second year of this research, we
develop a hierarchical feature combining and
verification method to improve the text extraction
accuracy and speed in natural scene images.
Background: Texts in the natural scene images often
contain important summarized information about the
scene. If we can find these items accurately in real time,
we can design the vision systems for assisting
navigations of the moving robots or the blinds. Many of
the previous researches have focused on extracting texts
that are artificially inserted into the image like captions,
thus the developed methods have many difficulties to be
used in the real applications. In this research, we are
developing the text extraction methods on the natural
scene images that can be used in the real world
application systems.
Proposed Method: <Fig. 1> shows an overview of the
research. In the first year of this research, text extraction
methods based on the low level image processing are
developed. In the second year, we developed the text
extraction method both using the low-level image
features of color continuity, gray-level variation and
color variance and by verifying the extracted regions
with the high-level text feature such as stroke. And the
two level features are combined hierarchically. The
color variance feature is added since the text strokes are
distinctive in their color values to the background, and
this value is more sensitive than the gray-level
variations. The text level stroke features are extracted
using a multi-resolution wavelet transforms on the local
image areas and the feature vectors are input to a
SVM(Support Vector Machine) classifier for the
verification. The proposed method can even extract the
text regions in the images of uneven illumination,
different text orientations, and skewed texts.
Contents and Results:
Developing methods for information selection: 1) In the
color reduced images by clustering the colors of the
small number of histograms to the nearest colors and by
improving the heuristic rules of the clustering method,
we obtained the clustering results that are not sensitive
to the initial selection of the colors. Thus, the clustering
results are stabilized and the extraction accuracies are
improved. 2) In the edge images the long line removal
method was improved by considering the 4-directional
histogram and the X/Y projections of the lines. Also, the
emphasizing and combining the candidate text regions
are improved. These improvements make possible the
extraction of the vertical text regions, and also improve
the text extraction accuracy. 3) To use the color variance
of the text regions in the image, both the horizontal and
vertical color variances are measured and their results
are logically ANDed to remove noisy objects. With this
feature the text extraction accuracy was improved a lot.
4) The combining method for the three candidate text
regions is developed to improve the accuracy of the
extraction. 5) Using multi-resolution wavelets
transformations 36 features are extracted from the 16x16
image blocks that represent the directions of the
character strokes. Only 12 distinctive features were
selected for the verification purposes with Baysian
method. Also, the SVM classifier was used for verifying
the text candidate regions.
1. Improving the extraction efficiency: 1) Developed
the skew correction method using the shearing
transformation by clustering the similar skew lines
according to the line locations. Also, developed the
method for correcting the perspective of the text regions
by finding the one or two vanishing points. 2) Improved
the text quality up to the level of possible recognition by
enlarging the low-resolution images, by developing the
filtering method to remove the noises, and by
developing the proper binarization method. 3) Improved
the processing time of the Canny edge detection up to
1/5, 1/10 by improving the non-maximum suppression
step. The quality of the edges in the image is maintained
by controlling the variances of the Gaussian filter.
We have tested the developed method using various
kinds of the natural images and have confirmed that the
extraction rates are very high even in complex images.
Fig. 1. The proposed method
Intelligent Visual System Design Based on Text Extraction in Natural Scene Images
90
A Study on Classifiers for Face Recognition and Face Detection
using Color Model
Hyeran Byun, Dept. of Computer Science, Yonsei University
Abstraction: Face recognition at a distance requires
both efficient face detection technique and classifier
with good generalization performance. For real time
facd detecting using skin-color information, we
proposed multi-channel skin-color model and applied
automatic whitening technique for the proposed
method to be robust to color temperature. Then, we
applied gaussian mixture model for real time
detection. Face images have a large variations in
appearances, so we should adopt a complex classifier
such as SVM. We studied fundamentals of SVM and
proposed a new method for multiclass problem.
Motivations: For real-time face detection, it is
essential to use color information. Since skin-color is
classified with the color of artifact, we can perform
face detection efficiently though color information.
Unfortunately, because object recognition based on
color information is very sensitive not only
illumination but also color temperature. It causes the
problem of Color Constant. Th conventional face
recognition uses the simplest classifier such as
Nearest Neighbor. It cannot apply efficiently to the
variation of facial images. Therefore, We need to
adopt complex classifier to have robust performance
against such variations. SVM is one of solutions.
However, we cannot apply it to a muti-class problem
like face recognition.
Real-time Skein-color Detection using Gaussian
Mixture Model: We made a skin-color model
containing only chrometic components to achieve
robust detection against illumination changes. In this
process, thorough analyzing the feature of skin-color,
we can use combination of each color of red, blue
and green channel. Th red component is distributed
widly among the face skin-colors while blue and
green components are concentrated on narrow
regions, so we made the skin-color model using
gaussian mixture model based on blue and green
components after extracting candidate region by red
component. The skin-color is very sensitive not only
illumination but also color temperature, so we adopt
automatic whitening technique to handle the
variations due to the color temperature. We can
separate skin-color region efficiently using multichannel color model with whitened images.
The Methods adopting SVM as Multiclass
classifiers: We adopt output coding method for
applying SVM to multiclass problems. The output
coding method can be described as follows: a
complex multiclass problem is decomposed into a set
of binary problems and then the outputs of binary
classifiers for each binary problem are reconstructed.
The conventional output coding methods are OPC
and All-Pairs. The OPC method separates one class
from all other classes and the All-Pairs method
separates only two classes for each possible pair of
classes. We propose a new method, the N-Division
output coding method, which is a generalized form of
OPC and All-Pairs. It divides problems produced by
OPC into parts of N. In this way, we can make the
decomposition which varies from OPC to All-Pairs
by controlling the value of N. As N decreases, the
decomposition becomes similar to OPC. If N is
increasing, it becomes similar to All-Pairs. Fig. 2
shows the framework of our N-Division method.
Results: The Fig. 1 showed skin-color detection
results by skin-color model which is based on
proposed color. In order to show the performance of
our proposed output coding method, we conducted
experiment on the ORL face dataset. The proposed
method have good properties such as problem
complexity although the difference in classification
performance is not significant. And we presented that
a value of 2 or 3 for N is desirable.
Fig. 1 Face skin-color detection by using
skin moded and gaussian mixture model.
proposed
Fig 2. N-Division Output Coding Method.
A Study on Classifiers for Face Recognition and Face Detection Using Color Model
91
Noise-induced Multimode Behavior in Excitable System
D. E. Postnov1 , O. V. Sosnovtseva1, S.K. Han2, and W.S. Kim2, 1Dept. of Physics, Saratov State
University, Russia, 2Dept. of Physics, Chungbuk Nat'l. University, Korea
Abstract: Based on experiments with electronic
circuits, we show how a system of coupled excitable
units can posses several noise-induced oscillatory
mode. We characterize the multimode organization in
term of the coherence resonance effect.
Background: While generation and entrainment of
single-mode deterministic or stochastic oscillations
are well understood, the dynamics of systems with
many oscillations with different modes is less studied
Many living systems perform oscillations with
different modes [1,2,3].
We focus on time scales that are delivered and
controlled by noise and that did not exist in the
deterministic case.
[1] X.J. Wang, Neuroscience 59, 21, 1994
[2] A. Neiman and D.F. Russel, Phys. Rev. Lett. 86,
3443, 2001
[3] S.K.Han, T.G.Yim, D.Postnov, and O.Sosnovtseva,
Phys. Rev. Lett. 83, 1771, 1999
[4] D.Postnov, O Sosnovtseva, S.K.Han, and W.S.Kim,
Phys. Rev. E 66, 016203, 2002
Results: We examined different implementations of
coupled identical excitable units with different types
of coupling as shown in Figure 1 [4].
Fig. 1. Different implementation of coupled exci-table
units. (a) Monovibrator;(b)mutually coupled units;(c)
circle configuration
Figure 2. demonstrates different collective behaviors
when the coupling strength and noise intensity of two
symmetrically coupled excitable units is varied. It is
clearly seen that two-mode behavior is observed
through the resonant and nonresonant ration between
the noise-induced frequencies.
In Figure3, three different frequencies is observed
when three functional excitable units are coupled as
circle shape (Figure1(c)). From this results, we can
state that the three-unit system is able to generate
three-mode stochastic behavior.
For more realistic neuronal excitable system, we
consider the Figure4(a) which emulate the simplest
circuit model of snail’s breathing rhythm generator.
As shown in Figure 4(c) , it can generate two-mode
stochastic oscillation behavior.
Discussions: We have shown that a simple system of
coupled excitable functional units can generate a few
oscillatory modes that are induced and controlled by
noise and coupling.
Noise-induced multimode behavior in excitable system
Fig. 2. Two-mode collective response in the system of
two mutually coupled monovibrators[Figure1(b)]. (a)
D=0.475V2. (b)D=0.77V2.
Fig. 3. Power spectrum illustration three-mode
collective behavior in a system of three interacting
excitable unit[figure1(c)] at D=0.35V2 and g=0.03.
Fig. 4. (a) Two monovibrators with delayed inhibitory
couplings imitate the simple neural circuit.(b)
Stochastic spike trains generated by the first and
second exitable units.
92
Neural Network Modeling for Intelligent Novelty Detection
Sungzoon Cho, Department of Industrial Engineering, Seoul National University
Abstract: In this research, we developed three novelty
detection models based on auto associative neural
networks (AANNs) and support vector machines (SVMs).
The first was applied to typing pattern based password
identity verification problem, the second to trading of
index future and the third to default company detection
problem. All three problems share the characteristic fit
for novelty detection in that training data set is heavily
or completely imbalanced. The first reduced the error
rate 5 times while the third reduced the error by half.
The second resulted in an excessive profit of 50 points in
a simulated trading for a year.
Background: Novelty detection is to identify novel
patterns out of normal ones. There are many situations
where normal patterns can be easily obtained yet novel
ones can not. For instance, the way one types his
password can be captured and stored as timing vectors.
However, it is neither easy nor desirable to obtain such
patterns from the rest of the population including
potential imposters. Although the problem can be cast as
a binary classification problem, scarcity or lack of data
from one class prevents one from using typical two class
classifiers. A traditional neural network algorithm
handling this novelty detection problem is auto
associative neural network. The AANN, which was
trained only with the normal patterns, produces a small
error on a normal pattern while a large error on a novel
pattern. Employing a proper threshold, one can
distinguish a normal from a novel pattern based on the
error value. Lately, SVM has gained popularities
because of its clear cut theory and good performance.
Another algorithm which can be applied to novelty
detection problem is one-class SVM. In this research,
we developed novelty detection models based on both
AANNs and SVMs. The models were applied to typing
pattern based password identity verification, index
future trading and default company detection. The
password identity verification system detects imposters’
typing patterns from user’s. There were no training
patterns available of non-user. We additionally
developed a feature selection algorithm and integrated it
into the system. For financial applications, a sudden
change in future price trends or a company having
abnormal financial status was defined as novelty. In both
cases, the number of novel patterns was relatively quite
small, thus novelty detection approach was employed.
The research details were published in the papers [1]
through [4].
Approaches: (A) Typing pattern based password
identity verification: We collected normal typing
patterns from 21 users and novel ones from 15 imposters.
Novelty detection algorithms based on AANN and SVM
were implemented independently, and the results were
compared [1]. In addition, we studied feature selection
algorithms based on wrapper approach [2]. In the
proposed wrapper feature selection algorithm, SVM was
used as the induction algorithm and GA was adopted as
the search mechanism. Use of SVM as the induction
algorithm allowed us to adopt wrapper approach since
its training time was 3 orders of magnitude smaller than
that of AANN. (B) Future trading and default
company detection: First, KOSPI future prices from
1999.01 to 2001.12 were used for training and
simulation [3]. We extracted four technical indicators
(Volume Ratio, Relative Strength Index, Rate Of
Change, Stochastic Slow %D), which were used as input
variables. Up-trend and down-trend were separately
trained and later combined to produce trading signals.
Second, the balance sheets of 2000 solvent companies
were used for training. [4]. A neural network based
bagging model was built as a novelty detector.
Results: (A) Typing pattern based password identity
verification: Fast learning SVM allowed a wrapper
based feature selection, which resulted in error reduction
to FRR=3.54% from FRR=15.78% (when FAR=0). (B)
Future trading and default company detection: The
simulated trading made an annual profit of more than 50
points without considering trading cost [3]. For default
company detection, novelty detection AANN reduced
the error by half compared with the conventional
approach where imbalanced data set was used [4].
[1] E. Yu and S. Cho, "Novelty Detection Approach for
Keystroke Dynamics Identity Verification", Fourth
International Conference on Intelligent Data Engineering
and Automated Learning, IDEAL 2003, HongKong.
[2] E. Yu, and S. Cho, "GA-SVM Wrapper Approach for
Feature Subset Selection in Keystroke Dynamics Identity
Verification", 2003 INNS-IEEE International Joint
Conference on Neural Networks, July 2003, Portland, USA.
[3] J. Lee, S. Cho, and J. Baek, “Trend Detection Using
Auto-Associative Neural Networks: Intraday KOSPI 200
Futures,” IEEE 2003 International Conference on
Computational Intelligence for Financial Engineering
(CIFEr2003), Mar 21-23, 2003, Hong Kong.
[4] J. Baek and S. Cho, “`Bankruptcy Prediction for Credit
Risk Using an Auto-Associative Neural Network in Korean
firms,” IEEE 2003 International Conference on
Computational Intelligence for Financial Engineering
(CIFEr2003), Mar 21-23, 2003, Hong Kong.
Neural Network Modeling for Intelligent Novelty Detection
93