CS 672 – Neural Networks – Fall 2010

Instructor: Marc Pomplun

Practice Midterm Exam – Sample Solutions

Duration: 75 minutes

No calculators, no books, and no notes are allowed (in the actual exam).

Question 1: ____ out of ____ points

Question 2: ____ out of ____ points

Question 3: ____ out of ____ points

Question 4: ____ out of ____ points

Question 5: ____ out of ____ points (bonus question)

Total Score:

Grade:
Question 1: Is it true?

Tell whether each of the following statements is true or false by checking the appropriate
box. Do not check any box if you do not know the right answer, because you will lose
points for incorrect answers.

Statement True False

a) A perceptron is guaranteed to perfectly learn a given linearly [X] [ ]

c) A single perceptron can compute the XOR function. [ ] [X]

d) The more hidden-layer units a BPN has, the better it can predict [ ] [X]
desired outputs for new inputs that it was not trained with.

e) In backpropagation learning, we should start with a small learning [ ] [X]

f) A three-layer BPN with 5 neurons in each layer has a total of 50 [X] [ ]

g) Supervised learning like in the BPN is biologically plausible. [ ] [X]

h) The backpropagation learning algorithm is based on the gradient- [X] [ ]

i) Some conflicts among training exemplars in a BPN can be [X] [ ]

j) Typically, Adalines produce better results for new (untrained) inputs [X] [ ]
than do perceptrons.
Question 2: The Soccer Network

Once again, here is an idea for an ANN that would make you rich if it performed well.
This ANN predicts the results of soccer matches. The network receives information about
the two competing teams and the conditions of the match and is supposed to predict how
many goals each team will score. With this knowledge, you could bet on the projected
winner team and gain a lot of money.

Let us say that every team consists of 20 players. You are providing the following input
data to the network:

• The skill level of every player on each of the two teams. Skill is rated by a group
of soccer reporters on a scale from 0 (“is unable to kick the ball”) to 10 (“world
class player”).

Represent each player’s skill level by a single input (neuron). Divide the skill
level by 10 so that the input values range from 0 to 1. Total input neurons
required: 40

• The number of matches that each team has played during the last two weeks.
There are never more than seven matches in that period of time.

Represent each team’s value by a single neuron. Divide the value by 7 so that the
input is scaled from 0 to 1. Total input neurons required: 2

• The statistics of former matches between the same two teams within the past 10
years (e.g., Team A won 30% of the matches, Team B 45%, and 25% of the
matches were tied).

Represent the proportion of wins by Team A with one neuron and the one for
Team B by another one. You could also use a third neuron for the proportion of
ties, but this redundant information may not help with the learning. Total input
neurons required: 2

• The continent that each team comes from (North America, South America,
Europe, Africa, Asia, or Australia).

These values do not represent a scale, so use six neurons for representation by
orthogonal vectors.

• Where the match takes place (Team A’s stadium, Team B’s stadium, or neutral
place).
 You could use 3 neurons and orthogonal input vectors or just one neuron, using
a scale like 0 = Team A’s stadium, 0.5 = neutral, 1 = Team B’ stadium. Let’s say
you are using 3 neurons.

• The phase of the soccer season (early season vs. late season).

Since this is only a binary variable, one neuron (value 0 or 1) is sufficient.

Total number of input neurons is 54. The best number of hidden-layer neurons is
difficult to estimate; let’s say 10. You could use 2 output neurons, one for each
team’s number of goals scored. You would have to estimate the maximum number
of goals ever to be scored, and then scale the desired output values so that output ε
represents 0 goals, and output (1-ε) represents that maximum number of goals.

As usual, you should use a completely connected network, which means that it has a
total of 540 + 20 = 560 weights, so ideally you should collect at least 5600 training
samples. You could maybe find them in online databases of soccer matches during
the past years. Make sure that your collection of exemplars samples the entire input
space; for example, include matches from both the early season and the late season.

For training, use the samples as network input in random order and apply the
backpropagation learning rule each time. After the network error is below a certain
threshold, test the network’s performance using another, possibly smaller set of
exemplars, called the test set.
Question 3: Linear Neurons

The following is a network of linear neurons, that is, neurons whose output is identical to
their net input, oi = neti. The numbers in the circles indicate the output of a neuron, and
the numbers at connections indicate the value of the corresponding weight.

-2 3
1
-2
0.5
2 -0.5 1

-0.5 2 0
0 -2
2 1
-1
1

0.5 1

a) Compute the output of the hidden-layer and the output-layer neurons for the given
input (0.5, 1) and enter those values into the corresponding circles.

Result: (-2, 3).

b) What is the output of the network for the input (1, 2), i.e. the left input neuron having
the value 1 and the right one having the value 2? Do you have to do all the network
computations once again in order to answer this question? Explain why you do or do
not have to do this.

It’s (-4, 6). We do not have to do all the computations, because every neuron
computes a linear function on its inputs, which means that the entire network
computes a linear function. For such a function, if we double the input, we
simply double the output as well.
Question 4: Adalines

(a) Explain how an Adaline works, i.e., describe the function that it computes and its
learning algorithm. If you do not remember how the weights are modified during
training, just describe what exactly the purpose of these changes is and how their
magnitude can be derived (no equations necessary).

Please see textbook (sorry for the laziness).

(b) What is the advantage of Adalines over perceptrons? How is it achieved?

The advantage of Adalines is that they do not simply find any solution that leads to
perfect classification of the training set, but they try to optimize their computation
so that it works as best as possible with new (untrained) inputs. This is achieved by
using a continuous, differentiable error function and minimizing this error using
gradient descent. This error minimum is the best estimate for the optimal
classification function with regard to the entire data set (of which the training data
are usually only a small subset).

(c) Explain how we can use a layer of Adalines to perform classification for more than
two classes. How are the units trained, and how do we interpret the units’ output in
production mode to determine the class of the current input?

If there are K classes, then use K Adalines and train each one individually so that
unit k outputs 1 if the input is from class k, and 0 (or -1 if you prefer) otherwise. In
production mode, if exactly one unit outputs 1, then the input is estimated to be of
class k. If no unit or multiple units output 1, there is a misclassification (the input is
estimated to belong to none of the classes).

Question 5 (Bonus Question): Brains and Computers

Please describe the main differences between the human brain and today’s computers
(such as your desktop PC) in terms of information processing.

• The brain works in a highly parallel fashion, but in the PC, everything has to
go through one or several processors.
• Neurons compute slowly (several ms per computation), electronic elements
compute fast (<1 nanosecond per computation).
• The brain represents information in a distributed way, because neurons are
unreliable and could die any time. Computer programs rely on every single
bit to function properly, otherwise these programs would crash.
• Our brains change their connectivity over time to represent new information
and requirements imposed on us. The connectivity between the electronic
elements in a computer never changes unless we replace its components.