I've been looking for a proof for the variance on the number of steps before being absorbed in an absorbing Markov chain. The theorem is given on Wikipedia without citation.
Following the references given at the bottom of the page, in particular Kemeny & Snell's "Absorbing Markov Chains" p. 51, gave a proof but I'm struggling to follow it. I'd appreciate any help you can give in going through it, or in pointing me in the direction of another proof/resource.
Thanks.
Partial Kemeny & Snell's Proof:
\begin{align}
\{\mathbf{M}_i[t^2]\} &= \left\{ \sum\limits_{s_k \in T} p_{ik}\cdot 1 + \sum\limits_{s_k \in T} p_{ik}\mathbf{M}_k[(t+1)^2] \right\}\tag{1}\\
&= \left\{\sum\limits_{s_k \in T} p_{ik}(\mathbf{M}_k[t^2]+2\mathbf{M}_k[t]) +1 \right\}\tag{2}\\
&=\mathbf{Q}\{\mathbf{M}_i[t^2]\} +2\mathbf{Q}\mathbf{\tau} + \mathbf{\xi}\\
\end{align}
etc
Note, here $\mathbf{M}[X]$ is the expected value of $X$, $t$ is the number of steps until absorption, $s_k$ is an arbitrary state, $p_{ik}$ are the elements in the transition matrix, $T$ is the set of all transient states (i.e. non-absorbing), $Q$ is a matrix defined in the canonical form of the transition matrix, $\mathbf{\tau}$ is a column vector of the expected number of steps, and $\xi$ is a column vector of ones.
My issue lies between lines one and two. I don't really understand why the two can be removed from the sum. In my mind, the second part of line one should expand to
\begin{align*}
p_{ik}\mathbf{M}_k[(t+1)^2] &= p_{ik}(\mathbf{M}_k[t^2]+\mathbf{M}_k[2t]+\mathbf{M}_k[1])\\
&=p_{ik}(\mathbf{M}_k[t^2]+2\mathbf{M}_k[t]+1)
\end{align*}
and then I don't see how the two $p_{ik}$'s can result in a one outside of the sum.
Best Answer
I suspect that there is a typo somewhere in (1) and you should have something like $$\sum\limits_{s_k \in A} p_{ik}\cdot 1 + \sum\limits_{s_k \in T} p_{ik}\mathbf{M}_k[(t+1)^2] $$ where $A$ is the set of absorbing states, i.e. $T^c$.
You will then later find yourself adding $\sum\limits_{s_k \in A} p_{ik}\cdot 1 +\sum\limits_{s_k \in T} p_{ik}\cdot 1$, i.e. $\sum\limits_{s_k \in {A \cup T}} p_{ik}\cdot 1$ which is indeed $1$.