Harry's restaurant changes year after year between the states $0$
(bankrupt), $1$ (almost bankrupt) and $2$ (solvent).The transition matrix is
$P= \begin{pmatrix}
1 & 0 & 0 \\
1/2 & 1/4 & 1/4 \\
1/2 & 1/4 & 1/4
\end{pmatrix}$Calculate the expected value for the amount of years till state $0$ is
reached, if we started from state $2$.
I took this question from an exam and try to solve it but I'm not sure how to do this correct? I'm a bit confused we need to work with expected value to calculate the required steps / years to get from state $2$ to state $0$. At least that's how I understood this so far.
It all sounds like I need to solve some recursive relations. Let $h(k)$ be the expected number of steps / years in this example until we reach the state $0$ when you are in state $2$. So we have that
$$h(2)=0$$
because when you are in state $2$, you need $0$ steps / years to reach $2$. Then for $k=1$
$$h(1) = 1+0.25h(1)+0.25h(2)+0.5h(0)$$
because when you are in state $1$ you will need a step ($+1$) so you will reach with probability $0.25$ state $1$ again and with probability $0.25$ state $2$ and with probability $0.5$ state $0$.
Similarly we do this for $h(0):$
$$h(0) = 1+1h(0)$$
But from here I don't really know how to continue to get the system and calculate the expected number pf steps with that? : /
Best Answer
Let $h(k)$ be the expected time to reach state $0$ if we started from state $\color{blue}k$.
Then $h(0)=0$.
And if we start with state $1$, with probability $\frac12$ we reach state $0$, with probability $\frac14$ we reach state $1$, and with probability $\frac14$ we reach state $2$.
Hence $$h(1)=1+\frac12h(0)+\frac14h(1)+\frac14h(2)$$
Similarly,
$$h(2)=1+\frac12h(0)+\frac14h(1)+\frac14h(2)$$
Substituting $h(0)=0$, we have
$$h(1)=1+\frac14h(1)+\frac14h(2)\tag{1}$$
$$h(2)=1+\frac14h(1)+\frac14h(2)\tag{2}$$
Subtracting both equation we have $$h(1)=h(2)\tag{3}$$
Use equation $(3)$ and $(2)$ to solve for $h(2)$.