Matakuliah
Tahun
: T0293/Neuro Computing
: 2005
Pendahuluan
Pertemuan 1
1
Basic Concept of Neural Networks
•
Traditional, sequential, logic-based digital computing
excels in many areas but has been less successful for
other types of problems.
•
Artificial Neural Networks (ANN) was motivated by a
desire to try both to understand the brain and to
emulate some of its strength.
•
The basic idea of ANN is to adopt the human brain-like
mechanism, i.e.:
1.
2.
3.
The processing element receives many signals
Fault tolerance
Parallel processing
2
•
A NN is characterised by
1.
2.
3.
•
its pattern of connections between the neurons (= its
architecture)
its method of determining the weights on the connections (=
learning / training algorithm)
its activation function
A NN consists of many processing elements called
neurons / units / cells / nodes
3
• Karakteristik Neural Networks
  Filosofis:
–
–
–
–
Learning
Generalization
Abstraction
Applicability
  Teknis:
– Node Characteristics
– Topology ( struktur rangkaian node-node )
– Learning Rules
4
Neural Networks Development
•
The 1940s: The Beginning of Neural Nets
– McCulloch-Pitts Neurons
•
•
•
•
Warren McCulloch & Walter Pitts (1943)
The neurons can be arranged into a net to produce any
output that can be represented as a combination of logic
functions
Feature: the idea of a threshold such that (s.t.) if the net
input to a neuron is greater than the threshold, then the
unit fires
Most likely used as logic circuits
5
– Hebb Learning
• Donald Hebb (1949)
• The first learning law for ANNs
• Premise: if two neurons were active simultaneously, then the
strength of the connection between (b/w) them should be
increased
• Feature: weight adjustment
6
• The 1950s & 1960s: The First Golden Age of Neural
Networks
– Perceptrons
• Group of researchers (Black, 1962; Minsky & Papert, 1969;
Rosenblatt, 1958, 59, 62)
• Typical perceptron consists of an input layer (the retina)
connected by paths with fixed weights to associator neurons;
the weights on the connection paths are were adjustable
• The learning rule uses an iterative weight adjustment that is
more powerful than the Hebb rule
7
– ADALINE
• Bernard Widrow & Marcian Hoff (1960)
• Closely related to the perceptron learning rule
• The perceptron learning rule adjusts the connection weight
to a unit whenever the response of the unit is incorrect
• The delta rule adjusts the weights to reduce the difference
b/w the net input to the output unit and the desired output
• The learning rule for a single layer network is a pre cursor of
the BP rule for multi-layer nets
8
• The 1970s: The Quiet Years
– Kohonen
• Teuvo Kohonen (1972) – Helsinsky Univ of Tech
• Dealt with Associative Memory Neural Nets
• Basic of the development of self-organising feature maps
that use a topological structure for the cluster units
• Application: speech recognition, solution of the TSP, etc
9
– Anderson
• James Anderson, Brown Univ ( 1968, 1972)
• Dealt with Assoc Mem NN
– Grossberg
• Stephen Grossberg (1967 – 1988)
– Carpenter
• Gail Carpenter together with Grossberg has developed a
theory of a self organizing neural networks called adaptive
resonance theory for binary input patterns (ART1 & ART2)
10
• The 1980s: Renewed Enthusiasm
– Backpropagation
• Overcome the problem discovered in the previous decade,
which caused by two reasons: the failure of single-layer
perceptrons to be able to solve such simple problems
(mapping) as the XOR function and the lack of a general
method of training a multilayer net.
– Hopfield Nets
• David Tank, John Hopfield (1982)
• A number of NNs have been developed based on fixed
weights and adaptive activations
• These nets can serve as associative memory nets and can be
used to solve constraint satisfaction problem such as the TSP
11
– Neocognitron
• Kunihiko Fukusima et al (1975, 1988)
• A series of specialised NN for character recognition has been
developed
– Boltzmann Machine
• Nets in which weights or activations are changed on the
basis of a probability density function.
• These nets incorporate such classical ideas as simulated
annealing and Bayesian decision theory
12
• Prospek Masa Depan Neural Networks
– Model-model Neural Networks:
•
•
•
•
Hopfield Net
Multilayered-networks (Back Propagation)
Bidirectinal Associative Memory
Self-organizing Maps, dll
– Berbagai Macam Aplikasi
• Pattern Recognition
• Image Compression
• Optimization, dll
13