Solved – Markov Chain for system with failure and repair

markov-processself-studytransition matrix

I am solving the below problem, and facing a few doubts regarding my approach.

Consider a system with two components. We observe the state of the system every hour. A given component operating at time n has probability p of failing before the next observation at time n + 1. A component that was in a failed condition at time n has a probability r of being repaired by time n + 1, independent of how long the component has been in a failed state. The component failures and repairs are mutually independent events. Let $X_n$ be the number of components in operation at time n. The process $\big\{$$X_n$, n = 0, 1, . . .$\big\}$ is a discrete time homogeneous Markov chain with state space $I$ = 0, 1, 2.

  1. Determine its transition probability matrix, and draw the state diagram.
  2. Obtain the steady state probability vector, if it exists.

From my understanding, there are 3 possible states of the system:

  1. All 2 components are working fine
  2. One component has failed and one is working fine
  3. Both components are in the failed state

So I have arrived to the following Markov Chain diagram:

enter image description here

The 2r is because any one system can be repaired if both are in failed state. Using this diagram, I arrived at the below transitional probability matrix:

\begin{bmatrix}
-2p & 2p & 0 \\
r & -(r+p) & p \\
0 & 2r & -2r
\end{bmatrix}

Am I thinking in the right direction? And can anybody help me with the second part of the question? To find the steady state probability vector?

Thanks!

Best Answer

I think that, your are messing up with the state transition probabilities.

I recommend to you to draw the Markov chain with the probabilities of both jumping to a new state and to staying in the current state. Remember that for each state the probability of staying a leaving the state should sum up to $1$.

For example, in the state $0$ (i.e., no component is working):

  • The probability of staying is $p_{00}=(1-r)^2$, that is both components failed to be repaired.
  • The probability of jumping to state $1$ (one components is working) is $p_{01}=2\cdot r(1-r)$, that is, one of the components is repaired and the other is not (note the 2, that accounts for the fact that either of the components can be repaired/no repaired).
  • The probability of jumping to state $2$ (two components are working) from state $0$ is $p_{02}=r^2$ that is, both components are repaired.
  • If you sum up those probabilities you get $r^2 +1+r^2 -2r +2r -2r^2=1$
Related Question