At all times, the urn comprises N balls-some white balls and some black balls. At each stage, coin having probability p, 0 < p < 1, of landing heads is flipped. If heads appears, then ball is selected at random from urn and is replaced by white ball. Let Xn denote number of white balls in urn after nth stage.

(a) Is {Xn , n ≥ 0} a Markov chain? If so, describe why?
(b) What are its classes? Write down their periods? Are they transient or recurrent?
(c) Compute transition probabilities Pi,j .
(d) Let N = 2. Determine the proportion of time in each state.
(e) Based on answer in part (d) and your intuition, guess answer for limiting probability in general case.
(f) Prove your guess in part (c)
(g) If p = 1, determine expected time until there are only white balls in urn if initially there are i white and N - i black?