Let $Y_{n}$, $n≥0$ be an i.i.d. sequence of Bernoulli $(p)$ random variables for some fixed $p ∈
(0, 1).$
Let $X_{n} = Y_{n−1} + 2Y_{n}$ for $n ≥ 1$. Is $X_{n}$, $n≥1$ a Markov chain? If no explain why not. If
yes then determine initial distribution and transition matrix.
I am able to figure out that this is indeed a Markov chain How I did that:
$X_{n-1} = Y_{n−2} + 2Y_{n-1}$
$X_{n} = Y_{n−1} + 2Y_{n}$
$X_{n+1} = Y_{n} + 2Y_{n+1}$
$X_{n+1}$ can only take values of 0,1,2 and 3.
If it is 0 that means $Y_{n}, Y_{n+1}$ are 0,0
If it is 1 that means $Y_{n}, Y_{n+1}$ are 1,0
If it is 2 that means $Y_{n}, Y_{n+1}$ are 0,1
If it is 3 that means $Y_{n}, Y_{n+1}$ are 1,1
If I know $X_{n}$ that means I also know $Y_{n-1}, Y_{n}$ and I don't need any information from $X_{n-1}$ to calculate $X_{n+1}$. Hence, it is a Markov chain.
State-space = ${0,1,2,3}$
I need help with the initial distribution. I kind of know-how the transition matrix will look like but any help is appreciable.
Best Answer
The distribution of $X_1$ is just the distribution of $Y_0+2Y_1$: $$ \mathbb P(X_1=i) = \begin{cases} (1-p)^2,& i=0\\ p(1-p),& i=1\\ p(1-p),& i=2\\ p^2,& i=3. \end{cases}. $$ For each positive integer $n$, conditioning on $\{X_n=0\}$ implies that $Y_{n-1}=Y_n=0$. Therefore the conditional distribution of $X_{n+1}$ is that of $2Y_{n+1}$ - zero with probability $1-p$ and two with probability $p$.
Conditioning on $\{X_n=1\}$ implies that $Y_{n-1}=1$ and $Y_n=0$, so the conditional distribution of $X_{n+1}$ is the same as when conditioning on $\{X_n=0\}$.
Conditioning on $\{X_n=2\}$ or $\{X_n=3\}$ implies that $Y_n=1$, so the conditional distribution of $X_{n+1}$ is that of $1+2Y_{n+1}$ - one with probability $1-p$ and three with probability $p$.
Therefore the transition matrix of this Markov chain is given by
$$ P = \left( \begin{array}{cccc} 1-p & 0 & p & 0 \\ 1-p & 0 & p & 0 \\ 0 & 1-p & 0 & p \\ 0 & 1-p & 0 & p \\ \end{array} \right). $$