Section 4.5.1 showed how to find the expected first-passage times to a fixed state, say 1, from all other states. It is often desirable to include the expected first recurrence time from state 1 to return to state 1. This can be done by splitting state 1 into two states, first an initial state with no transitions coming into it but the original transitions going out, and, second, a final trapping state with the original transitions coming in.

(a) For the chain on the left side of Figure 4.6, draw the graph for the modified chain with five states where state 1 has been split into two states.

(b) Suppose one has found the expected first-passage times vj for states j = 2, ... ,4

(or in general from 2 to M). Find an expression for v1, the expected first recurrence time for state1 in terms of v2,v3, ... , vM and P12, ... , P1M.

Text Book: Stochastic Processes: Theory for Applications By Robert G. Gallager.