AIspace

Practice Exercise 7.B

Neural Networks

Back to practice exercises.

1: Background Reading

7.4.1 Neural Networks

2: Learning Goals

Calculate the output units of a neural network given the inputs.
Construct feed-forward neural networks.
Train feed-forward neural networks using back propagation.

3: Directed Questions

Describe at least two reasonable stopping criteria that can be used when training a neural network. [solution]
How many arcs would a fully-connected feed-forward neural network have if it has one hidden layer, three input units, two hidden units, and two output units? [solution]
The simplest type of feed-forward neural network is called a perceptron. It consists of a single layer of output nodes (the inputs are fed directly to the outputs via weights). What type of data are perceptrons capable of learning? [solution] Can you give an example of a function which would be impossible to learn using a perceptron network? [solution]
Describe at least one way to overcome the problem of overfitting when constructing neural networks. [solution]

4: Exercise: Jane's Pizza Party

Jane is very picky about the toppings she likes on her pizza. Her favorite toppings are ham and pineapple. She only likes these toppings when they are on separate pizzas and hates it when they are mixed! She is indifferent about whether her pizza contains mushrooms. Here is a table summarizing her topping preferences:

ham?	pineapple?	mushroom?	good pizza?
no	no	no	no
no	no	yes	no
no	yes	no	yes
no	yes	yes	yes
yes	no	no	yes
yes	no	yes	yes
yes	yes	no	no
yes	yes	yes	no

Consider the following perceptron network:

AIspace Applet failed to load. Is Java enabled in your browser?

The inputs are set to 1 if the topping is present and -1 if not present. The output node uses a logistic activation function and has a bias term of 1.0. An output below 0.5 means Jane does not like the pizza and above 0.5 means Jane likes the pizza. You can left click on an arc to set its weight. Right click on the canvas and click on "Summary Statistics" to see how the network classifies Jane's pizza preferences. Leave the "Input range threshold of classification" parameter at 0.5.

Manually modify the network weights to try to maximize the number of correct classifications. What is the best number of correct predictions you can get when the weight from the mushroom topping is set to zero? [solution] Can the mushroom topping be used to improve this classification rate? [solution] Why can't this network correctly predict all of Jane's pizza preferences? [solution]

Jane is planning a surprise pizza party for her friends. She doesn't know what pizza toppings her friends like and she doesn't want to ruin the surprise by asking them. Instead she decides to survey random people about their pizza preferences. After examining the survey results, she manually decides on what pizzas to order. Unfortunately some of the people at the party complained about her pizza choices. She recorded all of the complaints and compliments about the pizza.

The following file contains all the data Jane recorded: survey.txt. The training data contains Jane's survey results. The test data contains the pizza complaints and compliments from the party. Use the neural network tool to create a network predicting people's pizza preference based on the data Jane collected.

How many errors did Jane make on the test set? [solution] You should be able to create a neural network which does better. Try adjusting the network structure and the learning rate to minimize the test error. You can also try solving this problem using the decision tree tool (as seen in Exercise 7.A).

5: Learning Goals Revisited

Calculate the output units of a neural network given the inputs.
Construct feed-forward neural networks.
Train feed-forward neural networks using back propagation.