In this problem you will tackle a challenging case when the parameters are degenerate: that is, there is a special combination if x1 and x2 where the model makes the same prediction while simultaneously varying the values of both x1 and x2. In such situations, neural networks trying to learn the likelihood surface may also experience significant difficulties. Obviously, the problem gets even worst when the dimensionality of the parameter space grows beyond two.
In the presence of a model degeneracy, one may encounter a likelihood function such as
L1(X1, X2) = exp (- (1 - x1)2 - 100 (x2 - x12)2) .
(a) Plot In (L1(x1, x2)) over the region x1, x2 ∈ [-5, 5]. The model's degeneracy will show up as lines in the x1-x2 plane where the value of ln L1 does not change.
(b) Modify the code you wrote to solve problem 2 to compute the value of
Z1 = -5∫5 -5∫5 L1(X1, X2) dX1 dX2
(c) Let N = 10k for k = 2,3,4, ... and compute In (Z1) by continually increasing N. Continue increasing N until you can convincingly get at least 2 digits of precision. Quote some values of N and In (Z1).
(d) By any means possible (using your code or some other way entirely), compute Z2 for likelihood function
L2(X1, X2) = 1
What fits the data better, model 1 or model 2?