A soap company specializes in a luxury type of bath soap. The sales of this soap fluctuate between two levels-low and high-depending upon two factors: (1) whether they advertise and (2) the advertising and marketing of new products by competitors. The second factor is out of the company's control, but it is trying to determine what its own advertising policy should be. For example, the marketing manager's proposal is to advertise when sales are low but not to advertise when sales are high (a particular policy). Advertising in any quarter of a year has primary impact on sales in the following quarter. At the beginning of each quarter, the needed information is available to forecast accurately whether sales will be low or high that quarter and to decide whether to advertise that quarter. The cost of advertising is $1 million for each quarter of a year in which it is done. When advertising is done during a quarter, the probability of having high sales the next quarter is ½ or ¾ depending upon whether the current quarter's sales are low or high. These probabilities go down to ¼ or ½ when advertising is not done during the current quarter. The company's quarterly profits (excluding advertising costs) are $4 million when sales are high but only $2 million when sales are low. Management now wants to determine the advertising policy that will maximize the company's (long-run) expected average net profit (profit minus advertising costs) per quarter.
(a) Formulate this problem as a Markov decision process by identifying the states and decisions and then finding the Cik.
(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrix and write an expression for the (longrun) expected average net profit per period in terms of the unknown steady-state probabilities
(c) Use your OR Courseware to find these steady-state probabilities for each policy. Then evaluate the expression obtained in part (b) to find the optimal policy by exhaustive enumeration.