We covered Markov chains in class and after going through the details, I still have a few questions. (I encourage you to give short answers to the question, as this may become very cumbersome otherwise)
1.) There are many theorems like: If $(X_n)$ is an irreducible( with a state space $S$), positive recurrent markov chain, then…(for example there is a unique stationary distribution $\pi>0$). Now, assuming that $(X_n)$ is not irreducible, but contains a positive recurrent class $A \subset S$. Does this mean that we can say that there is a unique stationary distribution $\pi|_A>0$ and $\pi=0$ elsewhere? So my question is: If we want to talk about one positive recurrent class instead of one irreducible chain, does this method of transferring the results ( by restricting the result to the subset $A$) always work? When I thought about it, I considered this to be a trivial conclusion, but most books don't even talk about this simple generalisation, so I wanted to ask this here.
2.) When you want to identify communication classes of a Markov chain on a finite set you can do this by drawing a picture and look whether there is a pathway between two states. But is there also a way of calculating the classes directly from the transition matrix?
If my guess in question (1) holds, then there should be a one-one correspondence between eigenvectors $\pi^T = \pi^T P $ with $\pi_A >0$ (and $\pi_{A^C}=0$) and positive recurrent classes $A$. In that case the only question would be how to find the remaining classes. But probably, the remaining transient states are just the states that always have a zero component in every stationary distirbution we get from the eigenvalue problem $P^T \pi = \pi$.
3.) The same as question 2) for the period. Is there a faster method of calculating the period of a state than calculating the powers of the transition matrix? Or at least a fast way to determine whether a state is periodic or aperiodic?
4.) I am struggeling with closed states. My intuition tells me that:
Every finite closed class is recurrent and every recurrent class is closed. So for finite chains the terms closed and (positive) recurrent should be $1:1$. Is this true?
It obviously diverges over infinite sets as the randomn walk in $\mathbb{Z}^3$ is transient, but the whole state space is closed.
5.) In case that we regard the continuous case of a homogenous markov chain and $P(X_{n+1} \in A |X_n =x) = \int_A q(x,y) dy$. How do we get $P(X_{n+k} \in A |X_n = x)
$?
6.) For a transient states $i,j$ in one class we have: $\sum_{n = 0}^{\infty} p^{(n)}_{ij} < \infty$. For a (positive) recurrent states $i,j$ in one class we have: $\sum_{n = 0}^{\infty} p^{(n)}_{ij} = \infty$. This is in my opinion a consequence of the fact that transience/recurrence is a class property, but I just wanted to be sure about this.
Unfortunately, this questions seems to be very unpopular. Any suggestion how I could improve my question is highly appreciated.
Best Answer
A) You did not consider the case say (12) and (34) are in two seperate communicating class such that
$p_{12}=p_{21}=p_{34}=p_{43}=1/2$, then there is no unique stationary distribution. If there is an only 1 closed communicating class, $A$, and all other states are transient, what you said is true.
If we want to talk about one positive recurrent class instead of one irreducible chain, does this method of transferring the results ( by restricting the result to the subset A) always work?
Yes.
B) I do not really understand you are trying to pull here? Take an example of a chain with two communicating classes. A possible solution to $\pi = \pi^T P$ is setting 0 for all states in one class and find the unique distribution for the other class.
Then the set of equilibrium measures are linear combinations of these vectors. You cannot use $\pi = \pi^T P$ unless you know that there is only $1$ closed communicating class (by same reason as question 1)
A finite state Markov Chain has a unique stationary distribution iff it contains only one closed communicating class. (so what you said is true, if you assume there is just 1 closed communicating class, everything else is open)
C) Not as far as I know. To see a state is periodic, find two path of different length on which it returns itself such that the highest greatest common factor. This is normally quite easy for a simple Markov Chain.
D) consider something like
1 -> 2 <-> 3
then 2 and 3 are closed. 1 is open. You know that open and closed are class properties, so are transient and recurrence. In a finite chain, recurrence and closed are the same thing.
The only difference occurs in infinite state Chains.
E) For a uncountable states, you can just integrate $k$ times
For example
$$P(X_2\in A|X_0=x) = \int_{y\in\Omega}\int_{z\in A} q(x,y)q(y,z)dydz$$
where $\Omega$ is the entire state space. For $X_3$, you just need to integrate 3 times etc. Notice that summation (used for countable state) is just integral with respect to counting measure.
F) you stated the sufficient and necessary condition for recurrence. There is a standard textbook proof, and yes verifying this for 1 pair of $i, j$ is enough because recurrence and transience are class properties. In fact often it is verified for $i=j$.