I am trying to solve a continuous-time Markov chain exercise. But I have been stuck for a few days and I don't know how to continue or even If what I have done is correct.
Consider a system having five identical independent components.
Each component works for an exponential time, measured in days, and
parameter 4.When a component breaks down, it is replaced immediately as sufficient
technicians are available to address any failure at any time. The
duration of a repair is also exponential and on average lasts half a
day.Determines, in the long run, how long the system is out of service
because it has all its components damaged.
I consider state n as the state where exactly n components are under repair. So I built the transition matrix Q as below:
$$
Q=\begin{pmatrix}
-20 & 20 & 0 & 0 & 0 & 0\\
2 & -18 & 16 & 0 & 0 & 0\\
2 & 2 & -16 & 12 & 0 & 0\\
2 & 2 & 2 & -14 & 8 & 0\\
2 & 2 & 2 & 2 & -12 & 4\\
2 & 2 & 2 & 2 & 2 & -10\\
\end{pmatrix}
$$
I'm not totally sure that the half under the diagonal is totally correct.
If it is, any hint how to calculate the system availability in the long run?
Best Answer
Let's first assume $j$ components work and $i$ components are being repaired.
The probability that a broken component is fixed before a working component breaks down is $$\mathbb{P}\Big(\min\{T_1,...,T_j\}>\min\{S_1,...,S_i\}\Big)$$ Here $T_1,...,T_j\sim \text{Exponential}(4)$ represent the times each of the $j$ components function properly while $S_1,...,S_i\sim \text{Exponential}(2)$ denote the repair times for each of the i broken components.
To calculate the aforementioned probability, set $T=\min\{T_1,...,T_j\}$ and $S=\min\{S_1,...,S_i\}$. It's easy to show that $T\sim \text{Exponential}(4j)$ and $S\sim \text{Exponential}(2i)$. Since $S,T$ are independent, the joint distribution of $(S,T)$ factors: $$f_{S,T}(x,y)=f_{S}(x)f_{T}(y)=8ij\exp\big\{-(2ix+4jy)\big\}$$ Then $$\mathbb{P}(T>S)=\int_0^{\infty}\int_x^{\infty}f_{S,T}(x,y)\mathrm{d}y\mathrm{d}x=\frac{i}{i+2j}$$
If $X_n$ is the number of components that function properly at time $n$, we see that conditional distribution $X_{n+1}|X_n=j$ for $1\leq j \leq 4$ is supported on $\{j-1,j+1\}$ and has pdf $$\mathbb{P}\left(X_{n+1}=j+1|X_{n}=j\right)=\frac{5-j}{5+j}$$ $$\mathbb{P}\left(X_{n+1}=j-1|X_n=j \right)=\frac{2j}{5+j}$$ Clearly $\mathbb{P}\left(X_{n+1}=1|X_n=0\right)=\mathbb{P}\left(X_{n+1}=4|X_n=5\right)=1$. Since the exponential distribution is memoryless, $\{X_n\}_{n\geq 0}$ is a Markov chain with state transition matrix $$P=\begin{pmatrix}0&1&0&0&0&0\\ \frac{1}{3}&0&\frac{2}{3}&0&0&0\\ 0&\frac{4}{7}&0&\frac{3}{7}&0&0\\ 0&0&\frac{3}{4}&0&\frac{1}{4}&0\\ 0&0&0&\frac{8}{9}&0&\frac{1}{9}\\ 0&0&0&0&1&0\end{pmatrix}$$ The unique stationary distribution of this Markov chain is $$\vec{\pi}=\begin{pmatrix}\frac{8}{81}\\ \frac{8}{27}\\ \frac{28}{81}\\ \frac{16}{81}\\ \frac{1}{18}\\ \frac{1}{162}\end{pmatrix}$$ The expected dwelling time in state $0$ is precisely equal to the expected value of $\min\{S_1,...,S_5\}\sim \text{Exponential}(10)$ which is $\frac{1}{10}$. So from this we see limiting expected holding time in state $0$ is $\frac{1}{10}\pi_0=\frac{4}{405}$.