You can work on this assignment individually or in a group of 2. If you are working in a group please establish a group in Assignment 2 Group on Canvas

In this assignment you are asked to explore the use of neural networks for classifica- tion and numeric prediction. You are also asked to carry out a data mining investiga- tion on a real-world data file. You are required to write a report on your findings. You will be assessed on methodology, analysis of results and conclusions.

PART 1: CLASSIFICATION WITH NEURAL NETWORKS

This part involves the following file:

heart-v1.arff

in the directory:

For the neural network training runs build a table with the following headings:

1. Describe the data encoding that is required for this task. How many outputs and how many inputs will there be?

2. Develop a script to generate the necessary training, validation and test files. You might want to normalize the numeric attributes with Weka beforehand. Include your data preparation script as an appendix (not part of the page count).

3. Determine the "analyze" strategy that you will use.

4. Using Javanns carry out 5 train and rest runs for a network with 10 hidden nodes. Comment on the variation in the training runs and the degree of overfitting.

5. Experiment with different numbers of hidden nodes. What seems to be the right number of hidden nodes for this problem?

6. For 10 hidden nodes, explore different values of the learning rate. What do you conclude?

7. [Optional] Change the learning function to backprop-momentum. Explore dif- ferent combinations of learning rate and momentum. What do you conclude?

8. Perform a run with 10 hidden nodes and no validation data. Stop training when the MSE is no longer changing. Get the classification error on the training and test data. Comment on the degree of overfitting.

9. Compare the classification accuracy of the neural classifiers with the classifica- tion accuracy of Weka J48 and MultilayerPerceptron.
Report Length Up to two pages.

PART 2: NUMERIC PREDICTION WITH NEURAL NETWORKS

This part involves the following file:

heart-v1.arff

in the directory:

The task is to predict the value of the Weight variable. Build a similar table of runs to the one in the previous question.

1. Describe the data encoding that is required for this task. How many outputs and how many inputs will there be? What scaling or normalization is required?

2. Modify your script from part 1 to generate the necessary training, validation and test files. You can use Weka to normalize all of the numeric attributes except for the class, ie weight attribute. You will need to write a suitable program to scale the weight to the range [0,1] and another one to reverse scale the neural net outputs to get the mean absolute error. Include your data preparation script as an appendix (not part of the page count).

3. Using Javanns carry out 5 train and test runs for a network with 5 hidden nodes. Comment on the variation in the training runs and the degree of overfitting.
[Hint: When you are comparing the predictive accuracy of different models you don't have to reverse scale the output.]

4. Experiment with different numbers of hidden nodes. What seems to be the right number of hidden nodes for this problem?

5. For 5 hidden nodes, explore different values of the learning rate. What do you conclude?

6. [Optional] Change the learning function to backprop-momentum. Explore dif- ferent combinations of learning rate and momentum. What do you conclude?

7. Perform a run with 5 hidden nodes and no validation data. Stop training when the MSE is no longer changing. Get the error on the training and test data. Com- ment on the degree of overfitting.

8. Compare the mean absolute error of the neural classifiers with the mean absolute error of Weka M5P and MultiLayerPerceptron.
Report Length Up to one page.

PART 3: DATA MINING

Choose EITHER the census data OR the movies data. The relevant files are can be found in the directory :

The file adult.arff contains American census data collected several years ago. The file adult.names describes the data items. There is some further information about the data at the website.

The movie data is was collected from the IMDb web site which claims to be "the world's most popular and authoritative source for movie, TV and celebrity content". It was collected to answer the question " How can we tell the greatness of a movie before it is released in cinema?"

IMDB-movie-data.csv has some changes from the kaggle file, mostly to make the genre information more usable.

Your task is to analyze the data with appropriate data mining techniques and identify any "golden nuggets" in the data. You are expected to use classification, clustering, association finding, attribute selection and visualization in your analysis, or to explain why a particular technique is not relevant. Be sure to give the rationale for each ex- periment.