An Introduction to Neural Networks
Presented by Scott Lichtor
Motivation I found for Neural
Networks
 Pavlov’s dog
 Simple->Complex
 Learning
Overview
• Basics of the Nervous System
– Neurons
– Synapses
– Action Potentials
• Neural Networks
– Abstract Neurons
– More Complicated Neurons
– Learning
– Supervised
– Unsupervised
– Reinforcement
• Conclusion
Basics of the Nervous System
 The nervous system coordinates the actions of an animal
 Body parts send messages to the brain
 Brain sends messages to body parts
 The basic unit of the nervous system is the neuron
Neurons
 Receive messages at the dendrites
 Message is sent quickly down the axon using electrical impulses
 What happens when the signal reaches the end of the axon?
Image taken from img460.imageshack.us
Synapses
 Chemical Synapses
 Slow
 Strong
 Can be transmitted over long distances
Image taken from http://www.airlinesafety.com/editorials
Synapses
 Electrical Synapses
 Very fast
 Fade quickly
Image taken from wikipedia.org
Action Potentials
 Action potentials are shocks to a particular neuron
 The shock travels along the affected neuron
 Then, the action potential is transmitted from the affected
neuron to the neurons connected to it
 The shock is transmitted to its destination in the same fashion
Abstract Neurons
 So biological neurons can be used to send modified messages
from place to place
 Can be used to accomplish very complex tasks using
relatively simple parts
 Can neurons represent other things/be used for other
objectives?
Abstract Neurons
 Neurons can represent neuron-like things
 Inputs -> Processes -> Outputs
Image taken from http://3.bp.blogspot.com/
Abstract Neurons
 Can “train” the neurons
 Neurons fire (output 1) under certain patterns
 Don’t fire (output 0) under other patterns
 Firing rule: if an outcome doesn’t fit in either pattern, it fires if
it has more in common with the first set, and doesn’t fire if it
has more in common with the second set.
 If there’s a tie, the neuron may fire, or it may not
Abstract Neurons
 Example
 A neuron takes three inputs (X1, X2, X3)
 The neuron is trained to output 1 if the inputs are 111 or 101
 Trained to output 0 if the inputs are 000 or 001
 Before firing rule:
X1
0
0
0
0
1
1
1
1
X2
0
0
1
1
0
0
1
1
X3
0
1
0
1
0
1
0
1
Out
0
0
0/1
0/1
0/1
1
0/1
1
 After firing rule:
X1
0
0
0
0
1
1
1
1
X2
0
0
1
1
0
0
1
1
X3
0
1
0
1
0
1
0
1
Out
0
0
0
0/1
0/1
1
1
1
Abstract Neurons
 The abstract neuron model can be used for pattern
recognition
 Example: determine whether a ‘T’ or ‘H’ is displayed
 Can we model more complicated processes with neurons?
More Complicated Neurons
 McCulloch and Pitts model
 Difference from previous model: inputs are weighted.
 Add weighted inputs together: if the sum is greater than a
threshold, then the neuron fires
Image taken from
http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11
/report.html
More Complicated Neurons
 Mathematically: neuron fires if
 X1W1 + X2W2 + X3W3 + ... > T
Examples
AND Gate
XOR Gate
Image taken from http://www.heatonresearch.com
More Complicated Neurons
 New model is very adaptable/powerful
 Input weights and threshold can be changed so the neuron
responds differently/more accurately to a situation
 Pavlov’s dog
 Various algorithms adapt neurons and neural networks to
situations
 Delta rule (feed-forward networks)
 Back-error projection (feedback networks)
Learning
 For the network to adapt, it must learn.
 There are three types of learning used with neural networks:
 Supervised learning
 Unsupervised learning
 Reinforcement learning
Supervised Learning
 In supervised learning, the system learns using test data given
from an external teacher
 The test data tells the system what outputs result from certain
inputs
 The system tries to match the response of the test data, i.e.
minimize the error between the neural network outputs and the
test data outputs given the same inputs
Image taken from http://www.learnartificialneuralnetworks.com
Unsupervised Learning
 In unsupervised learning, the network is given no output data
 Instead, the network is given just input data
 The goal of the network, then, is to group the input data
 Example: mortgage requests
 The network is given credit ratings, size of mortgage, interest
rate, etc.
 The network groups the data; probably into accept and deny
Reinforcement Learning
 Network performs actions on the input data
 The environment grades the network (good or bad)
 The network makes adjustments accordingly
 Middle ground between supervised and unsupervised
learning
Conclusion
 The learning aspect of neural networks makes their
applications astounding
 For computers, one has to know how to solve a particular
problem
 Neural networks can solve problems that one doesn’t know
how to solve
Conclusion
 Just some of the uses: sales forecasting, stock market
prediction, customer research, modeling and diagnosing the
cardiovascular system, “Instant Physician”, interpretation of
multi-meaning Chinese words, facial recognition, etc. etc.
etc.
 Something I found interesting: the interconnectedness of
different subjects
Sources
 http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11





/report.html
http://www.learnartificialneuralnetworks.com/
http://www.ryerson.ca/~dgrimsha/courses/cps721/unsupervis
ed.html
http://www.willamette.edu/~gorr/classes/cs449/intro.html
http://www.statsoft.com/textbook/stneunet.html
http://www.wikipedia.org