Portfolio 1 - Classification and partitioning

In addition to the combined marks for each of the portfolio tasks, you will also be graded on the structure, presentation and clarity of the portfolio as a whole. So your work should be professionally presented, with good use of English.

In the real world, you will be expected to communicate the results from any analysis you perform to non-specialists, so you should conclude each task with a brief explanation of your results, presented in terms a lay person would understand.

Deadline

The final submission for this task is week 24. However, as you will be set further tasks over the coming weeks, I recommend that you try to complete this by the end of week 22.

Description

This coursework requires you to use a variety of data mining techniques to explore the structure of your data, and use them to build a throrough description of the relationships and differences within the observations.

You may use any software you feel is appropriate for initial exploratory analysis of the data. However, you must use appropriate data mining software as your main tools. This should include both RapidMiner and R, although you may work with other packages or programming languages if you wish, and you may add extensions to RapidMiner, such as Weka and R.

(In particular, before using R or RapidMiner for discriminant analysis, you may wish to begin with performing an analysis with SPSS and/or Minitab. These will provide you with more detailed output which you can use to compare against the results you obtain from the data mining tools.)

N.B. It is possible to pass this assignment by just using the methods described in the lecture notes and course books (see below). However, additional marks are available for demonstrating research and exploration which leads to improving your models or overcoming initial limitations of the methods described.

Classification This task uses the well-known Iris data set. task 1

The data were first collected by American botanist Edgar Anderson, but became a popular method of exploring various multivariate statistical methods when it was used by Ronald Fisher to explore discriminant analysis in 1936. This version is from the UCI's Machine Learning Repository . http://archive.ics.uci.edu/ml/datasets/Iris

The data consists of four different measurements taken from 50 irises each of three different species. The original data set does not include any identification label for the observations, but I have added one - you may find it useful when assessing your results (don't forget that this should not be included in any analysis).

For some of the tasks, you will need to separate the data into training and testing data sets. As the data is ordered, you will need to use some method of randomisation or randomised sampling, which you should do using the appropriate software.

You should employ the sampling functions of the data mining software you use. For consistency, and to assess the relative strengths of the software and algorithms used, you may use the sets from one package in another. But I want to see evidence that you are using as much of the relevant functionality in your software as possible.

In each case, consider whether the strength of your models can be improved by restricting the variables used.

Compare the R and RapidMiner results, giving an account of their similarities and differences, and assesing their relative strengths and weaknesses.

a) Perform suitable exploratory analyses to examine the data, in particular how the values of the variables change with the species.

Use your results to decide whether you need to standardise the data in any way for the models you will build.

b) Use the k-NN algorithm to produce an assignment model for the data, using R and RapidMiner. In both cases, check the accuracy of the predictions, and use appropriate methods to try to improve it if necessary.

c) Perform a k-means cluster analysis on the data. Explain your choice of value for k and assess the strength of your results in terms of accuracy of partitioning.

Can you learn anything from changing the value of k?

Use hierarchical cluster anlaysis to justify (or otherwise) your value for k.

d) Build a decision tree for the data using RapidMiner and R.

Use appropriate methods to refine the tree to try to achieve maximum leaf purity based on the outcome variable species.

e) Use RapidMiner and R to produce a discriminant analysis of the data, with the goal of finding a set of discriminant equations which will best assign observations to their actual species.

f) Give an overall summary of your results above to give a description of how the combination of classification techniques builds a picture of the data set.

Identify which methods, algorithms, software etc. do the best job of explaining the data, and in particular, if the results from one method helped you refine another.

Are there any observations which cause problems for the different methods?

Classification These data are the results of a chemical analysis of wines grown in the same region task 2 in Italy but derived from three different cultivars. (A cultivar is a grouping of plants which which have similar, usually sought-after properties.) The analysis determined the quantities of 13 constituents found in each of the three types of wines.

The data is originally attributed to M. Forina, and may have been much larger. This version was donated to the UCI Machine Learning repository by Stephan Aeberhard.

See http://archive.ics.uci.edu/ml/datasets/Wine

(A slightly reduced version is available within your R installation, but this is the most complete version I could find.)

Note that this is a larger and more complex data set than was used in section A, and is therefore more like the data typically encountered.

a) Perform suitable exploratory analyses to examine the data, in particular how the values of the variables change with the three different cultivars.

Note that as you have 13 numeric variables in this data set variables, you may find that you can reduce the size of your models based on your EDA observations.

b) Use the k-NN algorithm to produce an assignment model for the data, using R and RapidMiner. In both cases, check the accuracy of the predictions, and use appropriate methods to try to improve it if necessary.

c) Perform a k-means cluster analysis on the data. Explain your choice of value for k and assess the strength of your results in terms of accuracy of partitioning.

Can you learn anything from changing the value of k?

Use hierarchical cluster anlaysis to justify (or otherwise) your value for k.

d) Build a decision tree for the data using RapidMiner and R.

Use appropriate methods to refine the tree to try to achieve maximum leaf purity based on the outcome variable cultivars.

e) Use RapidMiner and R to produce a discriminant analysis of the data, with the goal of finding a set of discriminant equations which will best assign observations to their actual cultivars.

f) In the above sections you built your models based on classifying wines according to the cultivar from which they were made.

One could quite reasonably explore some other way of classifying wines - alcohol content, for example.

Using the results of your exploratory data analysis, find a suitable method of classifying wines by their alcohol content and re-run your data mining modules to reflect this.

How do your results compare to the first set of models?

g) Give an overall summary of your results above to give a description of how the combination of classification techniques builds a picture of the data set.

Identify which methods, algorithms, software etc. do the best job of explaining the data.

Are there any observations which cause problems for the different methods?

Basic source material:

Lantz, B (2013) Machine Learning with R. Packt Publishing Ltd.

North, M (2012) Data Mining for the Masses. Global Text Project

Zumel, N Mount, J (2014) Practical Data Science with R. Manning Publications Co.