Reconsider Prob. 21.5-1.

(a) Formulate a linear programming model for finding an optimal policy.

(b) Use the simplex method to solve this model. Use the resulting optimal solution to identify an optimal policy.

Prob. 21.5-1

Joe wants to sell his car. He receives one offer each month and must decide immediately whether to accept the offer. Once rejected, the offer is lost. The possible offers are $600, $800, and $1,000, made with probabilities 5/8, ¼ and 1/8 respectively (where successive offers are independent of each other). There is a maintenance cost of $60 per month for the car. Joe is anxious to sell the car and so has chosen a discount factor of α = 0.95. Using the policy improvement algorithm, find a policy that minimizes the expected total discounted cost.