An individual has r umbrellas which he uses in going from his home to his office and vice versa. If he is at home (at the office) at the beginning (end) of the day and it is raining, then he will take an umbrella with him to the office (home), provided there is one to be taken. If it not raining then he never takes an umbrella. Assume that, independent of the past, it rains at the beginning (end) of a day with probability p. Let Xn denotes the number of umbrellas the individual has at his disposal at the moment he begins his nth trip.
i) Define a Markov chain with r+1 classes and find its transition probability matrix.
ii) Find the limiting probabilities.
iii) What fraction of time does the individual get wet?