[Math] Estimating the transition matrix given the stationary distribution

linear algebramarkov chainsmarkov-processprobability

Let's say we are given a Markov chain for variable $X = [x_1, …, x_n]$; also we are given a desired stationary distribution for this graph $P_\infty = [p_1, …, p_n]^\top$. How can we design an initial distribution and a transition matrix such that in the limit gives us to the stationary distribution ? Note that the graph and the connection between the variables are given and we cannot change them. We can only put probabilities on the edges. Assume that the graph is directed.

More formally, we are given $P_\infty$ (the stationary distribution), and the graph of variables and their connections. These connections can be explained by a transition matrix in which some of the elements are forced to be zero:
$$
T = \begin{bmatrix}
p_{11} &… & p_{nn} \\
\vdots & \ddots & \vdots \\
p_{11} & … & p_{nn}
\end{bmatrix}
$$
some of which are forced to be zero and the rest are to be estimated (unknown). We are looking for some $T$ and $P_0 = [p_1, …, p_n]^\top$ (the initial distribution) such that,
$$
\lim_{n\rightarrow \infty} P_0^\top T^n =P_\infty^\top
$$
such that $T$ is a valid transition matrix, i.e. sum of elements in each row is one; and all the values are greater than one, or equal to zero.

Best Answer

First consider that the limit means something more than just a formalization so you have a set of equations on it, meaning: $$ P_\infty^T T = P_\infty^T $$ ans T must be a valid transition probability matrix (each column must sum to 1).

so these are the required conditions that I know of. Plus the known form one can solve the problem with some degree of freedom or declare that there is no solution.

Related Question