FUNDAMENTALS OF NEURAL
NETWORKS
The Biological Neural Network
Characteristics of Human Brain
• Ability to learn from experience
• Ability to generalize the knowledge it possess
• Ability to perform abstraction
• To make errors.
The Biological Neural Network
Objective
• To emulate or simulate the human brain.
Organization of Human Brain
• Over one hundred billion neurons.
• Over one hundred trillion connections called
synapses.
• Neurons are responsible for thought
emotion, cognition etc.
• Consists of a dense network blood vessels.
Organization of Human Brain
• A highly effective filtration system called blood
brain barrier.
• A tight covering of glial cells around the
neurons.
• The glial cells provide structural scaffolding for
the brain.
• Brain is the most concentrated consumer of
energy in the body.
The Neuron
• Fundamental building
block of the nervous
system
• Performs all the
computational and
communication
functions within the
brain
• A many inputs/ one
output unit
The Neuron
Consists of three
sections
❖ cell body
❖ dendrites
❖ axon
Cell body
• manufactures a wide variety of complex
molecules, to keep it renewed for a life time
• manages the energy economy of the neuron
• the outer membrane of the cell body
generates nerve impulses.
• Cell body is 5 to 100 microns in dia
Dendrites
• bushy branching structure emanating from the cell body.
• Receive the signals from other cells at connection points called
synapses.
• Usually no physical or electrical connection made at the
synapse.
Dendrites
• Neurotransmitters which are specialized
chemicals are released by the axon, into the
synaptic cleft, diffuse across to the dendrite.
• Neurotransmitters are excitatory, which tend
to produce an output pulse.
• Some are inhibitory, which tend to suppress
such a pulse
• More than thirty neurotransmitters.
The Axon
• may be as short as 0.1mm or it is 1m in length.
• has multiple branches each terminating in a
synapse.
• Axons are wrapped in Schwann cells, forming an
insulating sheath known as myelin.
• This myelin sheath is interrupted every
millimeter or so, at narrow gaps called the
nodes of Ranveir.
The Axon
• Nerve impulses which pass down the axon, jump
from node to node, thus saving energy.
Computers and Human Brain
Similarities
• both operate on
electrical signals
• both are a
composition of a
large number of
simple elements.
• both perform
functions that are
computational.
Computers and Human Brain
Differences
• compared to μs or ns time scales of digital computation,
nerve impulses are astoundingly slow.
• The brain’s huge computation rate is achieved by a
tremendous number of parallel computational units, far
beyond any proposed for a computer system.
• A digital computer is inherently error free, but brain
often produces best guesses and approximations from
partially incomplete and incorrect inputs, which may be
wrong.
Simple Neural Network
X = I1W1+ I2W2+ ----- + INWN
Activation Function
S = K(X)
K is a threshold function
ie. S = 1
if X > T
S = O otherwise
T is a constant
threshold value.
Fundamentals of ANN
Squashing Function or Logistic Function or Sigmoid Function.
Y = 1/1+e-f
f=0
f>0
f<0
Y = 0.5
Y=1
Y=0
Fundamentals of ANN
Hyperbolic Tangent Function.
Y = tanh(f)
f=0
f>0
f<0
Y=0
Y=1
Y = -1
Single Layer Artificial Neural Networks
Multilayer Artificial Neural Networks
Multilayer Artificial Neural Networks
• Output of the first layer is obtained by multiplying
the input vector by the first weight matrix.
• If there is no nonlinear activation function multiply
the resulting vector by the second weight matrix.
• Y = (XW)K.
Training
Training is accomplished by sequentially applying input
vectors while adjusting network weights according to a
predetermined procedures.
Supervised Training
requires the pairing of each input vector with a target
vector representing the desired output.
Unsupervised Training
requires no target vector for the output and no
comparisons to predetermined ideal responses.
The
training algorithm modifies network weights to produce
output vectors that are consistent