Tuesday September 24, 2002
DSES-6870-01 INTRO TO NEURAL NETWORKS
Instructor:
Office Hours:
Class Time:
Text:
Prof. Mark J. Embrechts (x 4009 or 371-4562) (embrem@rpi.edu)
Thursday 2-3 pm (CII 5217)
Or by appointment.
Tuesday 8:30-11:20 am (DCC232)
Simon Haykin: Neural Networks: A Comprehensive Foundation,
Prentice Hall 1998
LECTURES 9&10: RADIAL BASIS FUNCTIONS
Radial Basis Function (RBFs) neural networks are the most popular neural networks next
to backpropagation. RBFs have just one hidden layer which makes up a nonlinear kernel
transformation. RBFs generally have as many neurons (grandmother cells) in the hidden
layer as there are training patterns, and the outputs of the hidden layer indicate how
similar a particular pattern is to the different patterns in the learning set. The output of the
RBF neural network can then be obtained by making a weighted average of the response,
where the similarity indexes from the hidden layer make up the weighting factors.
Radial basis functions are easy to train (now usually a one step procedure) and just need
tuning for the sigma for the Gaussian kernel widtyh or Parzen window width. RBFs are a
crude form of support vector machines and also form the basis for PNNs (probabilistic
neural networks) and GRNNs (general regression neural networks).
The drawback of radial basis function is that the verification mode is slow and requires
that a copy of all the training patterns is kept on hand. For that reason RBFs are generally
not implemented in hardware.
Handout:
Philip D. Wasserman, 1993, Advanced Methods in Neural Computing, Chapter 8, pp.147176.
Tasks:
1. Explore options #4431, 4418 and 4433 of analyze for running RBFs. Experiment with
the proper value for the Gaussian kernel width by cross-validation.
Homework Project #2: (Due October 8)
Find a neural networks related Journal paper related to your project and summarize the
paper in 10-15 PowerPoint slides (on paper) as if you were going to explain the paper to
an audience. Also provide a 2 page summary of the paper in clear language with a
minimum of jargon. Include the paper (and important paper) with your project. Make
1
sure you provide full references to the paper(not just a web address) in the slides and
summary.
PROJECT DEADLINES:
September 3
September 24
September 31
October 8
Homework Project #0 (web page summary)
Homework Project #1 (MetaNeural™ exercises)
Project proposal (1 or 2 typed pages)
Homework Project #2 (paper)
2