We consider the particular case of the gambler's ruin problem for which fc = 4 and p = 1/2. Suppose that the length T (in minutes) of a play (the outcome of which is the player's winning or losing $1) has an exponential distribution with mean equal to 1/2. Moreover, when the player's fortune reaches $0 or $4, he waits for an exponential time S (in hours) with mean equal to 2 before starting to play again, and this time is independent of what happened before. Finally, suppose that the player has $1 when he starts to play again if he was ruined on the last play of the preceding game and that he has $3 if his fortune reached $4 on this last play.
Let X{t), for t ≥ 0, be the player's fortune at time t. The stochastic process {X(t),t ≥ 0} is a continuous-time Markov chain.
(a) (i) Is the process {X(t),t ≥ 0} a birth and death process? If it is, give its birth and death rates. If it's not, justify.
(ii) Answer the same question if the player always starts to play again with $1, whether his fortune reached $0 or $4 on the last play of the previous game
(b) (i) Write, for each state j , the Kolmogorov backward equation satisfied by the function Po,j(t).
