Project
We examine handwritten digits scanned from the ZIP codes written on samples of U.S. postal envelopes. In this project, we focus on digits "7" and "9" only. Each digit image (resulting from a 16 16 grayscale matrix) is represented by a 256 dimensional vector of normalized intensity values. All the files relevant to the final project are available in Final_Project folder on Carmen. The computer code, final.R, will help you read data files (train.7.txt and train.9.txt), visualize individual digit images, and split the data into a training set of 800 observations (400 for each digit) and a test set of 489 (245 "7"s and 244 "9"s).
The main focus of this project is how to represent the data in much fewer dimensions and how to use such representations for classification of the two digits. Your analysis should address the following suggested questions. Feel free to explore other aspects of analysis that are not instructed below but of your own interest!
1. Dimension reduction
(a) Carry out a principal component analysis on the training data of 400 "7"s in data.7.train. To understand the nature of variations that the first few principal component directions capture, visualize the average image of "7"s first and then visualize the synthetic digit images that you would get along the principal component directions by considering x¯ + c.e, in other words, "average image + (constant) eigen-image" for some appropriately chosen range of c values including zero. Comment on the nature of variations that they capture.
Using each of the first two principal components, identify and display typical images of "7" and atypical images among the observed images.
How many components might be adequate to represent the 256 dimensional data?
(b) Carry out a similar principal component analysis on the training data of 400 "9"s in data.9.train.
(c) Carry out a principal component analysis on the combined training data of 400 "7"s and 400 "9"s in data.train and address similar questions as in (a). Comment on differences in the results from (a)-(c), if any.
In addition, generate a plot of mean-centered principal component scores using the first two principal components and label each point as the corresponding digit. Are the principal components useful for discriminating the two digits?
2. Classification
(a) Perform Fisher's linear discriminant analysis on the combined training data (data.train), using all the variables with sufficient variation. See the R code for subsetting those variables with sufficiently large standard deviations. There are, in fact, 18 pixels (or variables) with zero variation in the data and we should not use those variables for discrimination.
Visualize the coefficient vector of the linear discriminant as an image and compare it with the principal component directions obtained in 1(c). Evaluate the linear discriminant rule in terms of the apparent error rate and test error rate (error rate over the test data, data.test).
(b) Alternatively, perform Fisher's linear discriminant analysis on the combined training data, using the first two principal components from 1(c) as predictors. Compare the two classification rules in parts (a) and (b) in terms of the apparent error rate and test error rate.
Does the addition of subsequent principal components as predictors seem to improve classification accuracy?
Attachment:- final.rar