Time-Series Forecasting ARIMA – OLS vs Regression with ARMA Errors: A Comprehensive Comparison

arimaassumptionsforecastingleast squarestime series

I'm working with a time series of macroeconomic data (independent variables) and bank loss rates (dependent variable) to show how bank losses vary based on the state of the economy.

To estimate the relationship between the dependent and independent varaibles, I am considering two regression techniques: standard OLS and Regression with ARMA errors as outlined in this extremely helpful article https://robjhyndman.com/hyndsight/arimax/.

When comparing the coefficients and residuals calculated by the two methodologies, I am struggling to interpret which is preferable. I'll walk you through my thought process with the hope that others can better intuit which is the preferable model.

OLS Model

When I estimate the model using OLS, I get a decently fit model (63% Adj R-squared) with all coefficients showing some degree of significance:

> summary(varLM)

Call:
lm(formula = DepVar ~ modelFactors)

Coefficients:
                                   Estimate Std. Error t value  Pr(>|t|)    
(Intercept)                       -0.260879   0.061043  -4.274  4.73e-05 ***
NAT.5_Year_Treasury_Yield          0.012234   0.005059   2.418   0.01759 *  
NAT.House_Price_Index_Growth      -0.002837   0.001129  -2.514   0.01371 *  
REG.Real_Disposable_Income_Growth -0.011284   0.003674  -3.072   0.00281 ** 
REG.Unemployment_Rate              0.060506   0.006767   8.942  4.18e-14 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.08086 on 91 degrees of freedom
Multiple R-squared:  0.6491,    Adjusted R-squared:  0.6337

The model residuals pass stationarity tests, so I feel fairly confident that my model isn't misspecified. Below are my residuals plotted over time as well as the ADF test on the OLS residuals.

OLS Residuals

> adf.test(varLM$residuals)

Augmented Dickey-Fuller Test

data:  varLM$residuals
Dickey-Fuller = -3.9243, Lag order = 4, p-value = 0.01605
alternative hypothesis: stationary

Now where my models got dinged is that the residuals show autocorrelation. Below is the Durbin-Watson test for this model:

Durbin-Watson test

data:  varLM
DW = 1.0976, p-value = 1.709e-07
alternative hypothesis: true autocorrelation is greater than 0

A model validator could not get over that 1) my models had autocorrelation and 2) that I didn't use an arima structure. In fact they told me that the above model was basically garbage. On the autocorrelation point, I thought I could get around it using Newey-West Standard errors as I've read that serial correlation only affects the model's standard errors…

Regression with ARMA Errors

Based on this feedback, I investigated using a regression with ARMA errors to resolve the serial correlation issue in my models. I used the auto.arima function in R to fit an ARIMA model on my residuals:

> auto.arima(DepVar, xreg = modelFactors)
Series: DepVar 
ARIMA(1,0,0) with non-zero mean 

Coefficients:
     ar1  intercept  NAT.5_Year_Treasury_Yield  NAT.House_Price_Index_Growth
  0.4674    -0.2429                     0.0075                       -0.0029
  0.0985     0.0894                     0.0080                        0.0017
        REG.Real_Disposable_Income_Growth  REG.Unemployment_Rate
                             -0.0078                 0.0586
 s.e.                         0.0051                 0.0101

sigma^2 estimated as 0.005294:  log likelihood=118.34
AIC=-222.67   AICc=-221.4   BIC=-204.72

Comparing the coefficients on this model with the coefficients on the OLS model, I see little change. The obvious difference is now we are modeling the errors using an AR(1) process. The difference can be seen in the residuals which now look like true white noise.
ARMA Error Model Residuals

Conclusion

Am I wrong to say that the OLS model is fine, despite the serial correlation of the residuals? My preference is to stick to a simpler model structure (OLS) because I'd prefer to avoid a more complex model if possible. The other issue with the ARMA model is that I"m building multiple – what I've shown here is a subset. And when use the auto.arima function, I'm getting a different mix of AR() and MA() terms for each model.

I guess in general, is there a good rule of them of when OLS is appropriate for times series and when an ARIMA model is preferrable?

Best Answer

Am I wrong to say that the OLS model is fine?

I'd say no, you're not wrong, but it does depend on what you mean by "fine." An economist introduced me to the book "A Guide to Econometrics" by Peter Kennedy, which is a really nice book for telling you what happens when classical assumptions are violated but you use the classical technique anyways. Usually the sky does not fall, but you do lose something.

In the case of regression with autocorrelated errors, Section 7.4 states:

"With positive first-order autocorrelated errors this implies that several succeeding error terms are likely also to be positive, and once the error term becomes negative it is likely to remain negative for a while...In repeated samples these estimates will average out, since we are as likely to start with a negative error as with a positive one, leaving the OLS estimator unbiased, but the high variation in these estimates will cause the variance of [the OLS estimator] to be greater than it would have been had the errors been distributed randomly."

Additionally, it goes on to mention that the variance of beta_OLS is also underestimated, which of course will mess up your test statistics. But you still have unbiasedness!

Related Question