ROLL NO.:
NAME:
CS 537 / CMPE 537 – Neural Networks
Midterm Exam
April 14, 2005
Duration: 80 minutes (14:00 to 15:20)
Instructions:
(1) Please write legibly. Unreadable answers will NOT be graded. Use a BALL-POINT
and write in a readily readable font size.
(2) Write in the space provided for each question only. If you need to continue a question
in a different space, have it signed by the instructor/invigilator. Otherwise, it will not
be graded.
(3) Distribute your time judiciously as some questions might be more involved than
others.
(4) Make all usual assumptions if something is not stated in the question explicitly. If you
feel your answer depends on an unstated assumption, then state it explicitly.
(5) You may use a double-sided help-sheet and a basic calculator.
(6) There are 18 questions in this exam.
Questions 1 to 15 require short answers or multiple choice selections. Each question
is worth 4 points.
1. If the Euclidean distance between an input vector x and the weight vector w of a
neuron is d, compute the dot-product between these two vectors. Take the Euclidean
norms of the vectors equal to 1.
2. By definition, a learning system (like a neural network) can learn and adapt to its
environment. It is therefore essential that such systems have a feedback mechanism.
Clearly and briefly explain this concept in the context of neural networks.
CS 537 / CMPE 537 (Sp 04/05) – Dr. Asim Karim
Page 1 of 7
3. Consider a Hebbian neuron with 3 inputs. If the current weight vector is [1 0 1]T
compute the new weight vector when presented with the input vector [2 -1 -1]T. Take
η = 1.
4. Mathematically define auto-associative memory. Draw a neural network architecture
for auto-association.
5. Consider a fully connected 3-layer feedforward network with 10 input nodes, 8
compute nodes each in the hidden layers, and 4 compute nodes in the output layer.
Each compute node has a bias associated with it. What is the number of free
parameters in the network?
CS 537 / CMPE 537 (Sp 04/05) – Dr. Asim Karim
Page 2 of 7
6. Show that a two-layer linear network is equivalent in (input-output mapping)
capability to a single-layer linear network. You need to show this mathematically and
preferably in vector/matrix math.
7. MATLAB provides several training rule functions. What is the difference between
‘trainb’ and ‘trainc’ Explain briefly and clearly.
8. The perceptron learning algorithm is guaranteed to stop (converge) for a linearly
separable input environment. True or false? Explain briefly.
9. The Wiener filter
a. Can be applied to unknown environments
b. Works when the correlation matrix of the input to be singular
c. Provides an approximate solution to the linear filtering problem in the meansquared error sense
d. Can be modeled by a single-layer feedforward network with linear activation
functions
e. None of the above
10. Find the weight vector that minimizes the following cost function:
1 N
C (w ) 
[ d ( n)  x T ( n) w ]2

2 N n1
(answer this on the back-side of the PREVIOUS page)
CS 537 / CMPE 537 (Sp 04/05) – Dr. Asim Karim
Page 3 of 7
11. The BP algorithm was proposed in the
a. 1950s
b. 1960s
c. 1970s
d. 1980s
e. 1990s
f. No idea!
12. Consider the following two-layer network with McCullogh-Pitts ([0 1]) neurons.
Show that this network can solve the XOR problem by drawing the decision
boundaries for various inputs vectors in the input space.
X1
+1
+1
-2
Y
+1
X2
+1
-1.5
-0.5
13. Invariance can be built into a neural network design by
a. Training with a uniformly distributed pattern set
b. Training with as few patterns as necessary
c. Training with frequently occurring patterns only
d. Training with infrequently occurring patterns only
14. Consider a feedforward network with 4 input nodes and 2 hidden layer neurons. The
input-to-hidden weight matrix is given by
2  1.5 0.5 1.5  1
W

1 1.5  1  0.5 2 
Compute the outputs of the hidden layer when the input is [2 1 1 2]T and the logistic
activation function is used.
(answer on the back-side of the PREVIOUS page)
CS 537 / CMPE 537 (Sp 04/05) – Dr. Asim Karim
Page 4 of 7
15. The following approach may be used to improve the rate and/or stability of
convergence of the BP algorithm:
a. Normalize the targets to lie within the limits of the activation function
b. Use an anti-symmetric nonlinear activation function
c. De-correlate the input
d. Initialize weights from a normal distribution
e. a and b
f. a, b, and c
g. a, b, c, and d
Questions 16 to 18 require brief answers. Points are indicated next to the question
number.
16. (a) (7 points) In the neuron models that we have studied so far, the inner (or dot)
product between the input and weight vectors is performed to obtain the net input to
the neuron. Draw the signal-flow graph of a neuron in which the Euclidean distance
operation is performed. That is, the net input to the neuron is given by v  x  w
where x is the input vector and w is the weight vector of the neuron. Show at least 2
input nodes.
(b) (3 points) Name a learning task/problem or learning rule in which such a neuron
model may be used.
CS 537 / CMPE 537 (Sp 04/05) – Dr. Asim Karim
Page 5 of 7
17. (10 points) Consider the following two cost functions for a (simple single neuron)
perceptron:
J 1 (w ) 
N


 (w T x) and J 2 (w)   d (n)  w T x(n) x(n)
xM ( w )
n 1
where M(x) is the set of samples misclassified by the choice of weight vector w and N is
the total number of input samples.
(a) (2 points) Write the instantaneous equivalents of these two cost functions
(b) (6 points) Using any one of these functions, derive the perceptron update rule.
Can be same rule be obtained by using the other function.
(c) (2 points) What is the interpretation of the cost function J1 (or J2)? Comment
briefly.
CS 537 / CMPE 537 (Sp 04/05) – Dr. Asim Karim
Page 6 of 7
18. Consider the following MLP:
1
1
w110
X1
X2
w210
w211
Y2 1
w122
w212
The superscript indicates the layer number. Assume logistic activation functions, and η =
0.1. (continue the following questions on the back-side of the PREVIOUS page)
(a) (10 points) Calculate the update to the weights in layer 2 given Y12 = 0.2, Y1 =
[0.4 0.8]T, and the desired output is 0.6. Show your workings.
(b) (10 points) Calculate the update to the weights in layer 1 given X = [0.5 0.5]T.
Show your workings
CS 537 / CMPE 537 (Sp 04/05) – Dr. Asim Karim
Page 7 of 7