A) You did not consider the case say (12) and (34) are in two seperate communicating class such that
$p_{12}=p_{21}=p_{34}=p_{43}=1/2$, then there is no unique stationary distribution. If there is an only 1 closed communicating class, $A$, and all other states are transient, what you said is true.
If we want to talk about one positive recurrent class instead of one irreducible chain, does this method of transferring the results ( by restricting the result to the subset A) always work?
Yes.
B) I do not really understand you are trying to pull here? Take an example of a chain with two communicating classes. A possible solution to $\pi = \pi^T P$ is setting 0 for all states in one class and find the unique distribution for the other class.
Then the set of equilibrium measures are linear combinations of these vectors. You cannot use $\pi = \pi^T P$ unless you know that there is only $1$ closed communicating class (by same reason as question 1)
A finite state Markov Chain has a unique stationary distribution iff it contains only one closed communicating class. (so what you said is true, if you assume there is just 1 closed communicating class, everything else is open)
C) Not as far as I know. To see a state is periodic, find two path of different length on which it returns itself such that the highest greatest common factor. This is normally quite easy for a simple Markov Chain.
D) consider something like
1 -> 2 <-> 3
then 2 and 3 are closed. 1 is open. You know that open and closed are class properties, so are transient and recurrence. In a finite chain, recurrence and closed are the same thing.
The only difference occurs in infinite state Chains.
E) For a uncountable states, you can just integrate $k$ times
For example
$$P(X_2\in A|X_0=x) = \int_{y\in\Omega}\int_{z\in A} q(x,y)q(y,z)dydz$$
where $\Omega$ is the entire state space. For $X_3$, you just need to integrate 3 times etc. Notice that summation (used for countable state) is just integral with respect to counting measure.
F) you stated the sufficient and necessary condition for recurrence. There is a standard textbook proof, and yes verifying this for 1 pair of $i, j$ is enough because recurrence and transience are class properties. In fact often it is verified for $i=j$.
I think you need the condition that the Markov Chain is reversible.
Def: Let $X$ be an irreducible Markov chain such that $X_n$ has the stationary distribution $\mathbb{\pi}$ for all $n$. The chain is called reversible if the transition matrices of $X$ and its time-reversal $Y$ are the same, which is to say that $$\pi_iPij = \pi_jPji \:\:\:\: \forall i,j$$
Thm: Let $P$ be the transition matrix of an irreducible chain $X$, and suppose that there exists a distribution $\mathbb{\pi}$ such that
$$\pi_iPij = \pi_jPji \:\:\:\: \forall i,j$$
Then $\mathbb{\pi}$ is a stationary distribution of the chain. Furthermore, $X$ is reversible in equilibrium.
Suppose that $\mathbb{\pi}$ satisfies these conditions, then
$$\sum_{i} \pi_iPij = \sum_{i} \pi_jPji = \pi_j \sum_{i} Pji = \pi_j $$
Best Answer
The recurrent classes do not affect each other. Once we get into one recurrent class, we never leave, and the structure of the other recurrent class is irrelevant.
Just take an example, say with 4 states and transition probability matrix $(P_{ij})$ given by:
$$ (P_{ij}) = \left[ \begin{array}{cccc} 0 & 1/2 & 1/2 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 1 & 0 \end{array} \right] $$
State 1 is transient. State 2 forms an aperiodic recurrent class: If we get to state 2 then we always stay there. States 3 and 4 form a periodic recurrent class: If we get to state 3, we bounce around (periodically) between 3 and 4.
So:
-Given we start in state 2: We have a steady state distribution of $\pi=[0,1,0,0]$ (the limiting probabilities converge to this, and the time average fractions of time also converge to this with probability 1).
-Given we start in state 3: The limiting probabilities do not converge (they oscillate depending on even or odd slots), but the time averages converge to $p = [0,0,1/2,1/2]$ with probability 1.
-Given we start in state 1: Then the time averages will certainly converge, but they will not converge to a constant vector with probability 1. Rather, they will converge to a random vector. What they converge to will depend on the outcome of the first transition. If $p=[p_1,p_2,p_3,p_4]$ are the time averages, then given we start in state 1 we get: $$ p = \left\{ \begin{array}{ll} [0, 1, 0, 0] &\mbox{ with prob $1/2$} \\ [0, 0, 1/2, 1/2] & \mbox{ with prob $1/2$} \end{array} \right. $$
In general, for a finite state discrete time Markov chain with $K$ recurrent classes, each recurrent class $k \in \{1, \ldots, K\}$ has a unique probability distribution $\pi_k$ that satisfies $\pi_k = \pi_k P$ (where $P$ is the transition probability matrix) and such that $\pi_k$ has support only on the states of recurrence class $k$. If we start at a state in recurrence class $k$, then with probability 1 the time averages converge to $\pi_k$. If we start in a transient state, then the time averages will converge to a random vector $p$. We will eventually end up in one of the recurrent states (being the one we first visit). We can define $\theta_k$ as the probability we first visit recurrence class $k$ (defined for each $k \in \{1, \ldots, K\}$). Then $p$ is a random vector with $p = \pi_k$ with probability $\theta_k$, for $k \in \{1, \ldots, K\}$.