Let {Xn T1 = 0, 1, . . . } be a (discrete-time) Markov chain whose state space is the set Z of all integers. Suppose that the process spends an exponential time τ (in seconds) with parameter λ = 1 in each state before making a transition and that the next state visited is independent of r. Let N(f), for t > 0, be the number of transitions made in the interval [0, t].
(a) What is the probability that the third transition took place before the fifth second, given that five transitions occurred during the first 10 seconds?
