First part:
Let $a,b,c$ represent $3$ consecutive days. Since we are in state $1$, that means we have the sequence $(a,b) = \text{(no rain, rain)}$. In order to jump onto state $0$, there must hold $(b,c) = \text{(rain, rain)}$. Then we have the sequence $(a,b,c) = \text{(no rain, rain, rain)}$. According to the assumptions, starting from $(a,b)$ we can reach $c$ with probability $p=0.5$.
Also, $P_{11} = 0$. Why? If we still have $3$ consecutive days $a,b,c$ then it must hold $(a,b) = \text{(no rain, rain)}$ and $(b,c) = \text{(no rain, rain)}$, which can't happen.
Second part:
Notice that we start from state $0$, thus $\pi(0) = \begin{bmatrix} 1& 0 & 0 & 0\end{bmatrix}$ and we are going to evaluate the probability:
$$\pi(0)\cdot P^2 = \begin{bmatrix} 0.49 & 0.12 & 0.21 & 0.18\end{bmatrix}. $$
Thus, the probability that it rains on Thursday is going to be $p=0.49+ 0.12 = 0.61$ (see part $3$).
Third part:
From part $2$ it is known that the initial state is the state $1$. Assuming that we have the sequence $(a,b,c,d)$ with $a$ corresponding to the first day (Monday) and $d$ correspond to the last day (Thursday). Thus, we want the following to hold:
$$(a,b,c,d) = \text{(rain, rain, x, rain)}.$$ $x$ could either represent a rainy day or a non - rainy day. Thus, the are $2$ paths.
1: $(a,b,c,d) = \text{(rain, rain, rain, rain)}$
2: $(a,b,c,d) = \text{(rain, rain, no rain, rain)}$
Τhus, $(c,d)$ is going to be either (rain, rain), which indeed corresponds to state $0$ or (no rain, rain), which corresponds to state $1$.
Speaking with term of states the first $4-tuple$ corresponds to the path $0\to 0\to 0$, thus we have $p_{00}\cdot p_{00}= 0.7^2=0.49$ and the second $4-tuple$ corresponds to the path $0\to 2\to 1$, thus $p_{02}\cdot p_{21} = 0.3 \cdot 0.4 = 0.12$. Adding the two probabilities, leads us to the answer of the second part.
Yes it is appropriate to consider the the state transition matrix
$$\begin{bmatrix}
\frac{2}{3} & \frac{1}{3} & 0 & 0 & \dots \\
\frac{2}{3} & 0 & \frac{1}{3} & 0 & \dots \\
\frac{2}{3} & 0 & 0 & \frac{1}{3} & \dots \\
\vdots & \vdots & \vdots & \vdots & \ddots
\end{bmatrix}
$$
since the original chain gets in this chain in two steps. So we can forget about the first two states, 1 and 2, $P_1$ and $P_2$ will be $0$ on the long run because the chain will never get (back) to these states.
In the case of the transition matrix above, it is easy to calculate the stationary probabilities:
So, we have the stationary probabilities in a row vector $\pi=[P_3\ P_4 \ P_5\cdots]$ and we have the equation for the stationary probabilities
$$[P_3\ P_4 \ P_5\cdots]\begin{bmatrix}
\frac{2}{3} & \frac{1}{3} & 0 & 0 & \dots \\
\frac{2}{3} & 0 & \frac{1}{3} & 0 & \dots \\
\frac{2}{3} & 0 & 0 & \frac{1}{3} & \dots \\
\vdots & \vdots & \vdots & \vdots & \ddots
\end{bmatrix}=[P_3\ P_4 \ P_5\cdots].$$
In the case of the row vector and the firs column we get
$$\frac23(P_3+P_4+\cdots)=\frac23=P_3.$$
Then in the case of the row vector and the second column we have
$$P_3\frac13=\frac29=P_4$$
and so on.
So we have for the row vector of the stationary probabilities
$$\left[0 \ 0\ \frac23\ \ \frac29\ \ \frac2{27}\ \cdots\right].$$
Here $P_1$ and $P_2$ were included.
If I understood the question well then
$$\lim_{n\to\infty} p^n(4,7)=\frac2{3^5}=\pi(7)$$
because
$$\lim_{n\to\infty} \begin{bmatrix}
0&1&0&0&\cdots\\
0&0&1&0&\cdots\\
\frac{2}{3} & \frac{1}{3} & 0 & 0 & \dots \\
\frac{2}{3} & 0 & \frac{1}{3} & 0 & \dots \\
\frac{2}{3} & 0 & 0 & \frac{1}{3} & \dots \\
\vdots & \vdots & \vdots & \vdots & \ddots
\end{bmatrix}^n
=\begin{bmatrix}
0&0&\frac23&\frac2{3^2}&\frac2{3^3}&\frac2{3^4}&\frac2{3^5}\cdots\\
0&0&\frac23&\frac2{3^2}&\frac2{3^3}&\frac2{3^4}&\frac2{3^5}\cdots\\
0&0&\frac23&\frac2{3^2}&\frac2{3^3}&\frac2{3^4}&\frac2{3^5}\cdots\\
0&0&\frac23&\frac2{3^2}&\frac2{3^3}&\frac2{3^4}&\boxed{\color{red}{\frac2{3^5}}}\cdots\\
\vdots & \vdots & \vdots & \vdots & \vdots& \vdots & \ddots
\end{bmatrix}.$$
Best Answer
If you are currently in state $i$, then the $i$'th row of the transition matrix tells you in which state you will be the next day. If $i=1$, then the next day will be either $i=1$ (remembering $0$, learning $1$ new) or $i=2$ (remembering $1$, learning $1$ new), both with probability $1/2$. There is zero probability to reach anything higher (in this one step). Therefore the first row of the matrix should be $$\begin{pmatrix}1/2 & 1/2 & 0 & \dots \end{pmatrix}.$$ Similarly, starting at $i=2$, the next day will be at $i=1,2,3$, all with equal probability $1/3$, so the second column is $$\begin{pmatrix}1/3 & 1/3 & 1/3 & 0 & \dots\end{pmatrix}.$$
Continuing this process yields: $$\mathbf{P} = \begin{pmatrix} 1/2 & 1/2 & 0 &0 & 0&\dots\\ 1/3 & 1/3 & 1/3 & 0 & 0 &\dots\\ 1/4 & 1/4 & 1/4 & 1/4 & 0&\dots\\ 1/5 & 1/5 & 1/5 & 1/5 & 1/5&\dots\\ \vdots &\vdots & \vdots &\vdots & \vdots &\\ \end{pmatrix}$$
Hope this helps.
Note: there are different conventions on how to write the transition matrix. Some literature will have rows/columns switched, which means the matrix is simply transposed. I have written this in a way that matches your formula for the stationary distribution.