Let {Xn, n ≥ 1} denote a positive recurrent Markov chain having a countable-state space. Now consider a new stochastic process {Yn, n ≥ 0} that only accepts values of the Markov chain that are between 0 and some integer m. For instance, if m = 3 and X1 = 1, X2 = 3, X3 = 5, X4 = 6, X5 = 2, then Y1 = 1, Y2 = 3, Y3 =  2.

(a) Is {Yn, n ≥ 0} a Markov chain? Explain briefly.

(b) Let pj denote the proportion of time that {Xn, n ≥ 1} is in state j. If pj > 0 for all j, what proportion of time is {Yn, n ≥ 0} in each of the states 0, 1, ... , m?

(c) Suppose {Xn} is null recurrent and let pi(m), i = 0, 1, ... , m denote the long-run proportions for {Yn, n  ≥ 0}. Show that for j  /= i, pj(m)  = pi(m) E[time the  X process spends in j between returns to i].

Text Book: Stochastic Processes: Theory for Applications By Robert G. Gallager.