If we have classical linear regression model, and one of the regressors is time series (e.g. GDP), is it necessary for that variable to be stationary? Well i do not think so, because diversity in values yields to better results when we talk about linear regression, but I encounter different opinions.

# Solved – Linear regression, independent variable stationarity

linearregressionstationarity

#### Related Solutions

First of all, it is important to note that stationarity is a property of a process, not of a time series. You consider the ensemble of all time series generated by a process. If the statistical properties¹ of this ensemble (mean, variance, …) are constant over time, the process is called *stationary.* Strictly speaking, it is impossible to say whether a given time series was generated by a stationary process (however, with some assumptions, we can take a good guess).

More intuitively, stationarity means that there are no distinguished points in time for your process (influencing the statistical properties of your observation). Whether this applies to a given process depends crucially on what you consider as fixed or variable for your process, i.e., what is contained in your ensemble.

A typical cause of non-stationarity are time-dependent parameters – which allow to distinguish time points by the values of the parameters. Another cause are fixed initial conditions.

Consider the following examples:

The noise reaching my house from a single car passing

**at a given time**is not a stationary process. E.g., the average amplitude² is highest when the car is directly next to my house.The noise reaching my house from street traffic in general is a stationary process, if we ignore the time dependency of the traffic intensity (e.g., less traffic at night or on weekends). There are no distinguished points in time anymore. While there may be strong fluctuations of individual time series, these vanish when I consider the ensemble of all realisations of the process.

If I we include known impacts on traffic intensity, e.g., that there is less traffic at night, the process is non-stationary again: The average amplitude² varies with a daily rhythm. Every point in time is distinguished by the time of the day.

The position of a single peppercorn in a pot of boiling water is a stationary process (ignoring the loss of water due to evaporation). There are no distinguished points in time.

The position of a single peppercorn in a pot of boiling water dropped in the exact middle at $t=0$ is not a stationary process, as $t=0$ is a distinguished point in time. The average position of the peppercorn is always in the middle (assuming a symmetric pot without distinguished directions), but at $t=ε$ (with $ε$ small), we can be sure that the peppercorn is somewhere near the middle for every realisation of the process, while at a later time, it can also be closer to the border of the pot.

So, the distribution of positions changes over time. To give a specific example, the standard deviation grows. The distribution quickly converges to the respective distributions of the previous example and if we only take a look at this process for $t>T$ with a sufficiently high $T$, we can neglect the non-stationarity and approximate it as a stationary process for all purposes – the impact of the initial condition has faded away.

^{
¹ For practical purposes, this is sometimes reduced to the mean and the variance (weak stationarity), but I do not consider this helpful to understand the concept. Just ignore weak stationarity until you understood stationarity.
² Which is the mean of the volume, but the standard deviation of the actual sound signal (do not worry too much about this here).
}

If the purpose of your model is prediction and forecasting, then the short answer is YES, but the stationarity doesn't need to be on levels.

I'll explain. If you boil down forecasting to its most basic form, it's going to be extraction of the invariant. Consider this: you cannot forecast what's changing. If I tell you tomorrow is going to be different than today in **every imaginable aspect**, you will not be able to produce **any kind of forecast**.

It's only when you're able to extend something from today to tomorrow, you can produce any kind of a prediction. I'll give you a few examples.

- You know that the distribution of the tomorrow's average temperature is going to be
**about the same as today**. In this case, you can take today's temperature as your prediction for tomorrow, the naive forecast $\hat x_{t+1}= x_t$ - You observe a car at mile 10 on a road driving at the rate of speed $v=60 $ mph. In a minute it's probably going to be around mile 11 or 9. If you know that it's driving toward mile 11, then it's going to be around mile 11. Given that its speed and direction are
**constant**. Note, that the location is not stationary here, only the rate of speed is. In this regard it's analogous to a difference model like ARIMA(p,1,q) or a constant trend model like $x_t\sim v t$ - Your neighbor is drunk every Friday. Is he going to be drunk next Friday? Yes, as long as he
**doesn't change his behavior** - and so on

In every case of a reasonable forecast, we first extract something that is constant from the process, and extend it to future. Hence, my answer: yes, the time series need to be stationary if variance and mean are the invariants that you are going to extend into the future from history. Moreover, you want the relationships to regressors to be stable too.

Simply identify what is an invariant in your model, whether it's a mean level, a rate of change or something else. These things need to stay the same in future if you want your model to have any forecasting power.

# Holt Winters Example

Holt Winters filter was mentioned in the comments. It's a popular choice for smoothing and forecasting certain kinds of seasonal series, and it can deal with nonstationary series. Particularly, it can handle series where the mean level grows linearly with time. In other words where the **slope is stable**. In my terminology the slope is one of the invariants that this approach extracts from the series. Let's see how it fails when the slope is unstable.

In this plot I'm showing the deterministic series with exponential growth and additive seasonality. In other words, the slope keeps getting steeper with time:

You can see how filter seems to fit the data very well. The fitted line is red. However, if you attempt to predict with this filter, it fails miserably. The true line is black, and the red if fitted with blue confidence bounds on the next plot:

The reason why it fails is easy to see by examining Holt Winters model equations. It extracts the slope from past, and extends to future. This works very well when the slope is stable, but when it is consistently growing the filter can't keep up, it's one step behind and the effect accumulates into an increasing forecast error.

R code:

```
t=1:150
a = 0.04
x=ts(exp(a*t)+sin(t/5)*sin(t/2),deltat = 1/12,start=0)
xt = window(x,0,99/12)
plot(xt)
(m <- HoltWinters(xt))
plot(m)
plot(fitted(m))
xp = window(x,8.33)
p <- predict(m, 50, prediction.interval = TRUE)
plot(m, p)
lines(xp,col="black")
```

In this example you may be able to improve filter performance by simply taking a log of series. When you take a logarithm of exponentially growing series, you make its slope stable again, and give this filter a chance. Here's example:

R code:

```
t=1:150
a = 0.1
x=ts(exp(a*t)+sin(t/5)*sin(t/2),deltat = 1/12,start=0)
xt = window(log(x),0,99/12)
plot(xt)
(m <- HoltWinters(xt))
plot(m)
plot(fitted(m))
p <- predict(m, 50, prediction.interval = TRUE)
plot(m, exp(p))
xp = window(x,8.33)
lines(xp,col="black")
```

## Best Answer

What you assume in a linear regression model is that the

error termis a white noise process and, therefore, it must be stationary. There is no assumption that either the independent or dependant variables are stationary.However, consider the following simple linear regression model for time series data:

$$Y_t = a + b X_t + \varepsilon_t$$

If $Y_t$ is stationary but $X_t$ is not, then if you rearrange the equation:

$$Y_t - \varepsilon_t = a + bX_t$$

Then, the left-hand side is stationary, but the right-hand side is not, so the model can't be correct.

If, instead, both variables are not stationary, then:

$$Y_t - bX_t = a + \varepsilon_t$$

The right-hand side is stationary, but the left-hand side may or may not be. If it's not, then the model is wrong. It's possible for it to be stationary, as in a cointegration model for example, but it need not be.

Violating the assumption about the stationarity of the error process can lead to all sorts of problems, like spurious regressions where what appears to be a significant coefficient is frequently really not at all significant.