In a solution to the problem below, the teaching assistant solves it by calculating $\mathbb{E}[X_t^2]$ and ends up with also having to calculate $\mathbb{E}[X_{t-1}Z_t]$ after expanding the square. To do this, he states that $\mathbb{E}[X_{t-1}Z_t]=0$ since $X_{t-1}$ is independent of $Z_t$ and $X_{t-1}$ is uncorrelated with $Z_t$.
Questions:
- Why is $X_{t-1}$ independent of $Z_t$? I don't see that we assume that the TS is casual.
- Why is $X_{t-1}$ uncorrelated with $Z_t$?
Problem:
Let a timeseries model $X:=(X_t, t\in\mathbb{Z})$ be given by
$$X_t=\phi X_{t-1}+Z_t, \quad \text{where} \quad Z_t\sim
\text{WN}(0,\sigma^2)\quad \text{and} \quad |\phi|\neq 1.$$Assume that the stochastic process satisfying this model is stationary. Compute the variance of $X.$
Note: Yes, I know that one simply can calculate $\text{Var}[X_t]$ directly in one line, but I'm trying to understand the motivations behind the instructors steps.
Best Answer
The general meaning of white noise (that's what that WN denotes) is that the random variables $Z_t$ are independent. Some statisticians claim that it suffices to assume that the $Z_t$'s are uncorrelated (zero-mean, finite variance $\sigma^2$) random variables, that is, $$\operatorname{cov}(Z_t, Z_{t^\prime}) = \begin{cases}\sigma^2, & t = t^\prime,\\0, & t \neq t^\prime\end{cases}\tag{1}$$ but some others (including your teaching assistant and non-statisticians such as myself) prefer the stronger requirement that the $Z_t$'s be independent (zero-mean, finite variance $\sigma^2$) random variables; i.i.d. random variables if you are familiar with the acronym. Of course, $(1)$ still holds when the $Z_t$'s are independent. As your TA says, $X_{t-1}$ is independent of $Z_t$.
Actually, if you iterate the defining equation to write \begin{align} X_t &= \phi X_{t-1} + Z_t\\ &= \phi \big(\phi X_{t-2} + Z_{t-1}\big) + Z_t\\ &= \phi^2 X_{t-2} + \phi Z_{t-1} + Z_t\\ &= \phi^2 \big(\phi X_{t-3} + Z_{t-2}\big)+ \phi Z_{t-1} + Z_t\\ &= \phi^3 X_{t-3} + \phi^2 Z_{t-2} + \phi Z_{t-1} + Z_t \end{align} and so on, you can use induction to deduce that $X_t = \sum_{n=0}^\infty \phi^n Z_{t-n}$ and so , since the $Z_{t-n}$'s are independent random variables, $$\operatorname{var}(Z_t) = \sum_{n=0}^\infty \operatorname{var}(\phi^nZ_{t-n}) = \sum_{n=0}^\infty \phi^{2n} \operatorname{var}(Z_{t-n}) = \sum_{n=0}^\infty \phi^{2n} \sigma^2 =\frac{\sigma^2}{1-\phi^2} \tag{2}$$ provided that $|\phi|<1$. If $|\phi| > 1$, that series in $(2)$ diverges and it is not possible to write the value of $\operatorname{var}(Z_t)$ in the form shown on the right side of $(2)$. To answer a secondary question raised by the OP in a comment, it is not necessary to define a $X_0$ separately; the problem statement says that $t \in \mathbb Z$ and so $Z_t$ is defined for all integers $t$. Note that your TA's claim that $X_{t-1}$ is independent of $Z_t$ is perfectly valid: $X_{t-1}$ is a weighted sum of $Z_{t-1}, Z_{t-2}, \ldots$ and is thus independent of $Z_t$ by definition (uncorrelated with $Z_t$ for the naysayers).