Introduction to Neural Networks
and Fuzzy Logic
Lecture 1
Dr.-Ing. Erwin Sitompul
President University
http://zitompul.wordpress.com
2 0 1 3
President University
Erwin Sitompul
NNFL 1/1
Introduction to Neural Networks and Fuzzy Logic
Textbooks
Textbook:
“Neural Networks. A Comprehensive Foundation”, 2nd
Edition, Simon Haykin, Prentice Hall, 1999.
“Fuzzy Systems Theory and Its Application”,
Toshiro Terano et. al., Academic Press, 1992.
President University
Erwin Sitompul
NNFL 1/2
Introduction to Neural Networks and Fuzzy Logic
Grade Policy
Final Grade = 20% Homework + 20% Quizzes +
30% Midterm Exam + 30% Final Exam +
Extra Points
 Homeworks will be given in fairly regular basis. The average of
homework grades contributes 20% of final grade.
 Written homeworks are to be submitted on A4 papers, otherwise
they will not be graded.
 Homeworks must be submitted on time. If you submit late,
< 10 min.
 No penalty
10 – 60 min.  –20 points
> 60 min.
 –40 points
 There will be 3 quizzes. Only the best 2 will be counted. The
average of quiz grades contributes 20% of final grade.
 Midterm and final exam schedule will be announced in time.
 Make up of quizzes and exams will be held one week after the
schedule of the respective quizzes and exams.
President University
Erwin Sitompul
NNFL 1/3
Introduction to Neural Networks and Fuzzy Logic
Grade Policy
 The score of a make up quiz or exam, upon discretion, can be
multiplied by 0.9 (the maximum score for a make up is then 90).
 Extra points will be given if you solve a problem in front of the
class. You will earn 1, 2, or 3 points.
 Lecture slides can be copied during class session. It is also
available on internet. Please check the course homepage regularly.
http://zitompul.wordpress.com
• Heading of Written Homework Papers (Required)
President University
Erwin Sitompul
NNFL 1/4
Neural Networks
Introduction
Introduction to Neural Networks
Empirical
Phenomenon
measurement
Data
Experimental
modeling
Theoretical
modeling
validation
Mathematical
Model
Validation:
• Generally, means confirming that a
product or service meets the needs of
its users.
• Testing whether the mathematical
model is good enough or not to
describe the empirical phenomenon.
President University
Erwin Sitompul
NNFL 1/5
Neural Networks
Introduction
Experimental Modeling
 Experimental modeling consists of three steps:
1. The choice of model class
2. The choice of model structures (number of parameters, model
order, time delay)
3. The calculation of the parameters and time delay.
 The model may be chosen to be linear, nonlinear, or multi locallylinear.
 A-priori (prior, previous) knowledge of the system to be modeled is
required in most cases.
 Artificial Neural Networks (or simply Neural Networks) offers
a general solution for experimental modeling.
President University
Erwin Sitompul
NNFL 1/6
Neural Networks
Introduction
Experimental Modeling Using Neural Networks
 A neural network is a massively-parallel distributed processor
made up of simple processing unit, which has natural propensity
for storing experiential knowledge and making it available for use.
 It resembles the brain in two respects:
1. Knowledge is acquired by the network from its environment
through a learning process.
2. Interneuron connection strengths, known as synaptic weights,
are used to store the acquired knowledge.
President University
Erwin Sitompul
NNFL 1/7
Neural Networks
Introduction
Biological and Artificial Neuron
dendrite
Structure of
Biological neuron
soma
axon
synapse
x1
Structure of
Artificial neuron
Activation
function
wk1
x2
wk 2
wkm

bk
xm
1
yk
f()
net
m
net   wki xi  bk
i 1
yk  f (net )
President University
Erwin Sitompul
NNFL 1/8
Neural Networks
Introduction
Activation Function
 Any continuous (differentiable) function can be used as an
activation function in a neural network.
 The nonlinear behavior of the neural networks is inherited from the
used nonlinear activation functions.
y
y
y
1
1
x
x
y  f ( x)  x
y
y  f ( x) 
Linear
function
President University
2 1
1 e2 x
Tangent
sigmoid
function
x
x
y  f ( x) 
1
1  e x
Logarithmic
sigmoid
function
Erwin Sitompul
y  f ( x)  eax
2
Radial basis
function
NNFL 1/9
Neural Networks
Introduction
Network Architectures
Single layer feedforward network
(Single layer perceptron)
Input
layer
Output
layer
Multilayer feedforward network
(Multilayer perceptron)
Input
layer
President University
Erwin Sitompul
Hidden
layer
Output
layer
NNFL 1/10
Neural Networks
Introduction
Network Architectures
Diagonal recurrent networks
Input
layer
Hidden
layer
Output
layer
Fully recurrent networks
Input
layer
Hidden
layer
Output
layer
z 1
z 1
Delay element in a
recurrent network
President University
Erwin Sitompul
z 1
z 1
NNFL 1/11
Neural Networks
Introduction
Network Architectures
Elman’s recurrent networks

z

Jordan’s recurrent networks



z 1
1
z 1

z

 
1
z 1
z 1
President University
Erwin Sitompul
NNFL 1/12
Neural Networks
Introduction
Preparation Assignment
 Ensure yourself to install Matlab 7 or newer in your computer,
along with Matlab Simulink, Control System Toolbox, and Fuzzy
Logic Toolbox.
 Quizzes, Midterm Exam, and Final Exam will be computer-based.
President University
Erwin Sitompul
NNFL 1/13
Neural Networks
Introduction
Homework 1A
 Make 3 groups. Then, conduct an internet research and prepare a
short PowerPoint presentation about the following topics:
1. The timeline/milestones in neural networks development.
2. The applications and implementations of neural networks.
3. The current trends/methods of neural networks research.
 Each group will be given 15 minutes time for presentation on
Tuesday, 15.01.2012.
President University
Erwin Sitompul
NNFL 1/14