I am very new to time series analysis.
A random walk is defined as $Y_t=\phi Y_{t-1}+\varepsilon_t$, where $\phi=1$ and $\varepsilon_t$ is white noise. It is said that process is non-stationary for its variance is not constant. However, the mean is constant.
I am having a hard time understanding the "mean is constant" because when I plot a random walk process in R, I can see the variance is clearly changing, but to me the mean is also changing because there is a trend.
What does it exactly mean when they say random walk has a constant mean?
There are mathematical function that proves it but I want to understand more intuitively.
Really appreciate your help.
set.seed(1)
TT <- 100
y <- ww <- rnorm(n = TT, mean = 0, sd = 1)
for (t in 2:TT)
{
y[t] <- y[t - 1] + ww[t]
}
Best Answer
There is a difference between unconditional mean and conditional mean, as there is between unconditional variance and conditional variance.
Mean
For a random walk $$ Y_t=Y_{t-1}+\varepsilon_t $$ with $\varepsilon_t\sim i.i.d(0,\sigma_\varepsilon^2)$, the condtional mean is $$ \mathbb{E}(Y_{t+h}|Y_{t})=Y_t $$ for $h>0$. This means that given the last observed value $Y_t$, the conditional mean of the process after $h$ periods, $\mathbb{E}(Y_{t+h}|Y_{t})$, is that value, regardless of how much time $h$ has passed. If time starts at $t=0$, then we have the mean conditional on the initial value being $\mathbb{E}(Y_{h}|Y_{0})$. From this we can see that the conditional mean varies with the conditioning information but not the time differential $h$.
Meanwhile, the unconditional mean at any fixed time point $h$ is zero: $$ \mathbb{E}(Y_{h})=\mathbb{E}(\sum_{i=0}^h\varepsilon_i)=\sum_{i=0}^h\mathbb{E}(\varepsilon_i)=\sum_{i=0}^h(0)=0. $$ Since it does not vary with $h$, we could say the mean of the process is zero.
Variance
The conditional variance is $$ \text{Var}(Y_{t+h}|Y_t)=h\sigma_\varepsilon^2. $$ For a fixed time differential $h$, the conditional variance is not increasing (the fluctuations are not getting wilder) over time, but conditional on some fixed time point the unconditional variance grows linearly with the time difference. Thus contrary to the conditional mean, the conditional variance does not vary with the conditioning information but does vary with (namely, grows linearly in) the time differential $h$.
Meanwhile, the unconditional variance at any fixed time point $h$ is the number of the time point $h$ times the variance of the increment term: $$ \text{Var}(Y_h)=\text{Var}(\sum_{i=0}^h\varepsilon_i)=\sum_{i=0}^h\text{Var}(\varepsilon_i)=\sum_{i=0}^h(\sigma_\varepsilon^2)=h \sigma_\varepsilon^2 $$ where the second equality uses the independence of the increments $\varepsilon_i$. Note that we can easily define the variance at a fixed time point but it is not as simple otherwise. Without being very rigorous, one could say the variance is undefined for an undefined time point. (This is in contrast to the mean.)