You must upload both your code (to Assignment 6 scripts/codes) and your report (to Assignment 6 computing report). The assignment is due at 11:00pm. I have set the due time in Canvas to 11:05pm and if Canvas indicates that you submitted late, you will be given 0 on the assignment. Your computing report must be exactly 1 page. There will be a penalty given if your report is longer than one page.

• Please read the Guidelines for Assignments first.

• Keep in mind that Canvas discussions are open forums.

• Acknowledge any collaborations and assistance from colleagues/TAs/instructor.

A. Computing Assignment - Machine Learning

Required submission: 1 page PDF document and scripts/codes uploaded to Canvas. The purpose of this assignment is to teach your computer to distinguish between red and blue points in 2D. This is an example of linear discriminant analysis. For further information about this topic, look at the book "Elements of Statistical Learning" by Hastie et. al which is available online for free. In class we have spent considerable time on polynomial interpolation where the interpolating polynomial exactly equals the underlying data at each of the nodes. In the case where the data is noisy or the system is overdetermined, the method of least squares is used instead. Here is a basic reference on least squares data fitting: https : //en.wikipedia.org/wiki/Least squares. You are recommended to perform the following tasks in the given order and generate the required plots or numbers as you go. The associated, partially completed Matlab script is on the computing assignment page and is called linear discriminant.m. There is also a data file called dataset.mat which contains two long matrices training set and test set. All of these files must be placed in the same folder. The idea is to train your computer using the training set and then use the test set to see how well it has learned its lesson.

Description of the Data Sets 1. Each dataset has three columns.

2. The training set has 2000 rows (i.e. 2000 red and blue points) and the test set has 200 points.

3. Let X = training set(:, [1 2]) (i.e. the first two columns of the training set) and let y = training set(:, [3]).

4. Each row of the matrix X is the location of a point in 2D where the first column is the x-coordinate and the second column is the y-coordinate.

5. Each entry in y corresponds to a row of X and is either 0 or 1. If the value is 1 then we say that point is blue and if it is 0 then we say it is red. 1

The provided script for this assignment loads both datasets and assembles the X and y matrices for you and also generates a plot of the coloured points in 2D. Run the script to see Figure 1.

Training dataset consisting of red and blue points

Figure 1: Visualizing the set of blue and red points in the training dataset. The Training of the computer 1. Now define a matrix A so that A = [ones(2000, 1), X].

2. Note that A is a 2000 × 3 matrix and y is a 2000 × 1 vector (check using the size function in MATLAB).

3. The objective is to find a 3 × 1 vector β, so that Aβ = y.

4. A unique solution is not possible since this system is overdetermined.

5. Therfore, find an approximate solution ˆβ so that RSS(ˆβ) := ky - Xˆβk 2 2 has the minimum value. To do this, modify the script to compute the vector ˆβ using the normal equations (AT A) ˆβ = AT y, and report the value of RSS( ˆβ) in your report.

6. The RSS stands for Residual Sum of Squares. It gives you an idea of how good ˆβ is at predicting a value in y given the corresponding row in A.

After you complete the previous step uncomment the section of the script titled "plot the discriminant". This generates a new plot of the dataset along with a black line given by the equation

βˆ 1 + βˆ 2x + βˆ 3y = 1/2, where ˆβ = [βˆ 1, βˆ 2, βˆ 3] T . (0.1)

Explain, just by looking at the figure, what this line is saying about the red and blue points. Include the figure and your explanation in your report.

2 The Testing of how well the computer learned

1. Use ˆβ to predict whether a certain point is red or blue from the test dataset.

2. Let z = test set(:, [3]) which is a vector of size 200 × 1 and B = [ones(200,1),test set(:,[1 2])] which is a 200 × 3 matrix similar to A. Now modify the script to compute v = Bˆβ and evaluate the vector zˆ so that zˆj = ( + 1 if vj ≥ 1/2 0 if vj < 1/2 for j = 1, · · · , 200.

3. In MATLAB, output zˆ and z side by side (do not include in your report). How is ˆz predicting the values in z?

4. Compute the error of the prediction using

Err(βˆ) = 1 200 X 200 j=1 |zj - zˆj |.

Report this value in your report. Also, generate a plot of the test dataset where the points are colored according to the prediction z along with the discriminant ˆ line defined in equation (0.1) above. 

Attachment:- dataset.mat

Attachment:- linear_discriminant.m