markov-processself-study
A Markov chain {Xn, n ≥ 0} with states 1, 2,3 has the transition probability matrix P
\begin{bmatrix}0&0.4&0.6\\1&0&0\\0.3&0.3&0.4\end{bmatrix}
with an initial distribution A (0.5,0,0.5), what is $$P(X_1=3|X_2=1)$$?
(I know a Markov chain property is the future, given the present, is independent of the past. the question here look like given future, what is the probability of the past? I am wondering
$$P(X_1=3|X_2=1)=P(X_1=3)=A_3=0.5$$
or
$$P(X_1=3|X_2=1)=P_{13}=0.6$$
or else?
[...] but by having the initial Prob. of each state (P(t=1)), and Transition prob. i can calculate prob. of each state in any given time [...]
That sounds correct. You should be able to use induction and conditioning to deduce this. It has been a while since I've done anything with Markov Chains, so apologies in advance for any poor notation (also my first post ever :D ). The base case for induction should look something like this.
Suppose that we know the initial probabilities as above. Then we have :
$P_1[2] = P(X_2=1|X_1=1)P(X_1=1)+P(X_2=1|X_1=2)P(X_1=2)= S_1\epsilon_{11}[1]+S_2\epsilon_{21}[1],$
which are all known quantities.
I think a simple induction argument should finish it from there.
Here is a graphical representation of the chain, with the vertices $2$ and $5$ highlighted:
The initial state $a$ can be indicated by labeling the vertices with their values, highlighting the nonzero values:
Two transitions from $a$, as computed by the matrix product $a\mathbb{P}\mathbb{P} = a\mathbb{P}^2$, is this distribution:
The weight on vertex $2$ is precisely the chance of reaching $2$ after two steps; that is, it is $\Pr(X_2=2)$. To represent this event, we now zero out all other weights, leaving the distribution $b = (0, 2/25, 0, 0, 0)$. The question asks us to take two more steps, beginning at $b$, computing $b\mathbb{P}\mathbb{P} = b\mathbb{P}^2$:
The labels give the new distribution. All two-step paths from vertex $2$ to vertex $4$ are highlighted. (There is just one, making it easy to compute the new distribution: $2/25$ is multiplied by $p_{2,4}=2/5$, giving $4/125$ for the transition from $2$ to $4$, then that is multiplied by $p_{4,5}=1/10$, yielding $4/1250=2/625$ for the double transition $2\to 4\to 5$. In more complicated situations we would have to examine all possible paths from $2$ to $5$ and add their contributions.)
Evidently, the chance of reaching vertex $2$ at step $2$ and then arriving at vertex $5$ at step $4$ is the final value at vertex $5$, $2/625 = 0.0032$.
Best Answer
Rather than trying to guess the answer, one can apply probability laws. Since
the transition matrix $\mathsf P$ is made of the probabilities $\mathbb P(X_t=j|X_{t-1}=i)$ as $(i,j)$ entries,
the probability $\mathbb P(X_{t-1}=j|X_{t}=i)$ can be written as$$\mathbb P(X_{t-1}=j|X_{t}=i)=\dfrac{\mathbb P(X_t=i|X_{t-1}=j)\mathbb P(X_{t-1}=j)}{\mathbb P(X_t=i)}$$by Bayes' theorem,
the probability $\mathbb P(X_t=i)$ can be written as$$\mathbb P(X_t=i)=\sum_{j=1}^3 \mathbb P(X_t=i|X_{t-1}=j)\mathbb P(X_{t-1}=j)$$by the law of total probability,
the reverse probabilities $\mathbb P(X_{1}=j|X_{2}=i)$ can be derived from the entries of the homework question.