This is not a wikipedia question, because I cannot find it there. How does one compute the first return time of a time-homogeneous Markov chain. There are a lot of neat notions such as recurrence and expected return time, but I am not sure how one can calculate the first return time.
For simple example, if we have $p_{11} =1/3$, $p_{12}=2/3$, $p_{22}=1/6$ and $p_{21}=5/6$, then what is the first return time to state 1?
Best Answer
The expected time to return to a given state $i$ is $1/\pi(i)$, where $\pi$ is the invariant probability vector. In your problem, the transition matrix is $P=\pmatrix{1/3&2/3\cr 5/6&1/6}$ and the invariant probabilities are the solution to $\pi =\pi P$. This works out to $\pi=(5/9,4/9)$, so the answer is $9/5$.