markov-process
I have a transition matrix P,i don't understand how my teacher explains properties of the states without any calculation.
In example :
$$P= \begin{array}{ccc}
\frac12 & \frac14 & \frac14 \\
0 & \frac12 & \frac12 \\
0 & 0 & 1 \end{array}$$
How can i say that the first and second states are recurrent?
Is this chain irreducible because the third state doesn't communicates with other states?
And how can i understand the period of a state ?
$T = \{1,2\}$ almost certainly means that the set of transient states has as elements the states $1$ and $2$.
$C_1 = \{3\}$ probably means that one of irreducible closed sets has as its only element elements the state $3$. If there were more than one irreducible closed set (not this example) then you might see $C_2$ or others.
Let the state space of the Markov Chain be $S=\{1,2,3,4,5,6\}$. Now draw the state transition diagram.
(a). From the figure, we observe that $\{4\}$, and $\{6\}$ form non-closed communicating classes. State $2$ does not communicate even with itself and such a state is called a non-return state. Hence, the states 2, 4 and 6 are transient.
(b)&(c). The class $\{1,3,5\}$ is a closed-communicating class. Hence, states 1, 3 and 5 are recurrent states.
(c). There is only one closed-communicating class, $\{1,3,5\}$.
(d). As the chain is not an irreducible Markov Chain, it is not an ergodic chain.
Best Answer
In order to understand recurrence, transience, non-return state, and absorbing state, we don't require the actual transition probabilities of a Markov Chain, if the states of the chain can be accommodated in state transition diagram. For example, to understand the nature of the states of above Markov Chain, the given transition matrix can be equivalently be represented as
\begin{equation*} P = \left(\begin{array}{ccc} * & * & *\\ 0 & * & *\\ 0 & 0 & *\\ \end{array}\right) \end{equation*}
where a * stands for positive probability for that transition.
Now, draw the state transition diagram of the Markov Chain.
There are 3 communicating classes, here: {1}, {2} and {3}. Now identify which of these classes are closed communicating classes and non-closed communicating classes.
Consider class {1}. State 1 communicates with itself. However, an escape is possible to state 2 or state 3. Hence, it is a non-closed communicating class. States in a non-closed communicating classes become transient states.
Class {2} can be interpreted in a similar manner.
State 3 communicates with itself and all the edges are into the state 3. Hence, state 3 itself forms a closed-communicating class. States in a closed communicating classes become recurrent states. As there is only one state in this communicating class, the state is called an absorbing state. In a finite Markov Chain, there must be at least one recurrent state. As all the states do not belong to a single communicating class, the given chain is not irreducible.