Suppose p  N, you have a data matrix X and a quantitative response vector y, and you plan to t a linear regression model.

(a) Explain why the ordinary least squares solution is not unique. What can you say about the residuals of any of the solutions.

(b) Is the ridge regression solution unique? why?

(c) Suppose you compute a series of ridge solutions, letting  get successively smaller. What can you say about the limiting ridge solution in this case, as  # 0.

(d) Using the SVD of X, write a closed form expression for this limiting solution.