Here "stationarity" means the first and second moments don't change over time.
From a page of Time Series: Theory and Methods, by Peter J. Brockwell, Richard A. Davis
In this chapter we shall examine the problem of selecting an
appropriate model for a given set of observations $\{X_t t = 1, …, n
> \}$. If the data (a) exhibits no apparent deviations from stationarity
and (b) has a rapidly decreasing autocorrelation function, we
shall seek a suitable ARMA process to represent the mean-corrected
data. If not, then we shall first look fo r a transformation of the
data which generates a new series with the properties (a) and (b).
From a page of Time Series: Theory and Methods, by Peter J. Brockwell, Richard A. Davis
Trend and seasonality are usually detected by inspecting the graph of the
(possibly transformed) series. However they are also characterized by sample
autocorrelation functions which are slowly decaying and nearly periodic
respectively.
From wikipedia
non-stationarity is often indicated by an autocorrelation plot with very slow decay.
My questions:
Why should a stationary ARMA process have a rapidly decreasing autocovariance function?
Should a stationary AR process also have a rapidly decreasing autocovariance function?
Why is non-stationarity often indicated by an autocorrelation plot with very slow decay and values near 1 at small lags?
Why are trend and seasonality characterized by autocorrelation functions that are slowly decaying and nearly periodic, respectively?
Best Answer
It is possible to get a general formula for stationary ARMA(p,q) autocovariance function. Suppose $X_t$ is a (zero mean) stationary solution of an ARMA(p,q) equation:
$$\phi(B)X_t=\theta(B)Z_t$$
Multiply this equation by $X_{t-h}$, $h>q$, take expectations and you will get
$$r(h)-\phi_1r(h-1)-...-\phi_pr(h-p)=0$$
This is a recursive equation, which has a general solution. If all the roots $\lambda_i$ of polynomial $\phi(z)=1-\phi_1z-...-\phi_pz^p$ are different,
$$r(h)=\sum_{i=1}^pC_i\lambda_i^{-h}$$
where $C_i$ are constants which can be derived from the initial conditions. Since $|\lambda_i|>1$ to ensure stationarity it is very clear why the autocorrelation function (which is autocovariance function scaled by a constant) is decaying rapidly (if $\lambda_i$ are not close to one).
I've covered the case of unique real roots of the polynomial $\phi(z)$, all other cases are covered in general theory, but formulas are a bit messier. Nevertheless the terms $\lambda^{-h}$ remain.
Answers to question 2 and 3 more or less follow from this formula. For $AR(1)$ process $r(h)=c\phi_1^h$ and when $\phi_1$ is close to one, i.e. close to non-stationarity, you get the behaviour you describle. The same goes for general formula, if the process is nearly unit-root one of the roots $\lambda_i$ is close to 1 and it dominates other terms, producing the slow decay.