Consider an E = {1, 2, 3, 4, 5, 6}-valued Markov chain {Xn; n ∈ N} with transition matrix P, whose off-diagonal entries are specified by

1. Find the diagonal terms of P.

2. Find the equivalence classes of the chain

3. Show that 4 and 6 are transient states, and that the other states can be grouped into two recurrent classes to be specified. In the sequel, we let T = {4, 6}, C be the recurrent class containing 1, and C the other recurrent class. For all x, y ∈ E, define ρx := Px (T ), where T := inf{n ≥ 0; Xn ∈ C}