Machine Learning Program: Linear Regression

1 - Program Description

In this programming assignment, you are asked to write python code to implement linear regression model. To test the correctness of your model, you need to apply your linear regression model on two linear regression problems.

The first problem is to predict rental price based on descriptive features of size, floor and, broadband rate. The dataset can be found in prog3.txt, and textbook Table 7.1 (Page 325). You need to do the followings:

1. Using the initial weights given from textbook Table 7.3 (Page 345) to calculate the prediction, error, squared error, erroDelta(D, w[0]), erroDelta(D, w[1]), erroDelta(D, w[2]), erroDelta(D, w[3]). Output the calculations similar to program screenshot in Figure 1, and textbook Table 7.3.

2. Print out the new weights after the _rst iteration of gradient descent algorithm.

3. Using the new weights generated from your algorithm to calculate the prediction, error, squared error, erroDelta(D, w[0]), erroDelta(D, w[1]), erroDelta(D, w[2]), erroDelta(D, w[3]). Output the calculations similar to program screenshot in Figure 1, and textbook Table 7.3.

4. Print out the new weights after the _rst iteration of gradient descent algorithm.

5. Print out the new weights after 100 iterations, and the _nal sum of squared errors.

6. Do the plot between cost function and iterations.

The second problem is to predict oxyen consumption (Column 2) based on descriptive features of age and, heart rate. The dataset can be found in prog3 2.txt, and textbook Page 389. Do the same work as described above, using the initial weights and learning rate from Page 389. You should check your results with the ones provided from the solution manual.

2 - Useful Help

You should not use scikit-learner for this program, but you are allowed to use a slightly modified version of linear regression model (lr house.py) I provide to you, based on online source code.

The online Machine Learning with Python - Linear Regression can be found at http://aimotion.blogspot.com/2011/10/machine-learning-with-python-linear.html.

Read the tutorial and understand how linear regression can be used for predicting house prediction using dataset ex1data2.txt.

You can use the majority of source code for your program, and modify based on that.

The main goal of this program is to understand how the gradient descent algorithm is implemented, and dig into the details of the gradient descent algorithm by printing out errors, deltaErrors for each iteration. You need to pay attention to the usage of dot(), transpose(), matrix addition, matrix multiplication, array cancatenation with numpy.

Text Book - Fundamentals of Machine Learning For Predictive Data Analytics Algorithm, Worked Examples, and Case Studies by John D. Kelleher, Brain Mac Namee and Aoife D'Arcy

Attachment:- Assignment Files.zip