Solved – Variables importance on regression with ARIMA errors model

time series

One way to calculate importance of a variable in a regression, is to get the decrease in $RSS$ when we calculate a model without the variable in question.

Is this valid for a regression with ARIMA errors?

Suppose I want to get three measures:

Importance of external regressor
Importance of ARIMA coefs
Importance of Seasonal ARIMA coefs

Would the decrease in $RSS$ if I compare the original $RSS$ with the $RSS$ the following models represent the measures I list above?

$ARIMA(p,d,q)(P,D,Q)$ without external regressor
$ARIMA(0,0,0)(P,D,Q)$ with external regressor
$ARIMA(p,d,q)(0,0,0)$ with external regressor

Best Answer

What you want to do is to express your model as a pure right-hand side model where the right hand side contains lags of the output series and the contemporaneous and lag effects of each of the input regressors and a possible constant. Simply add up the effects to be able to partition the forecast into these separate factors. I have programmed this task in a commercially available piece of software I am involved with. If you need help in programming this I would be glad to help you.

Additional comments: In my opinion it makes no sense breakout the effect of ARIMA vs SARIMA as the combination of them is synergistic reflecting unspecified stochastic series. The way I would approach the problem is to build a transfer function including any needed lag structure for the known X's and include any necessary deterministic structure identfied via Intervention Detection AND set the error process to be white noise i.e. no ar or ma structure at all. As a second step I would ignore the external regressors and incorprate both any necessary deterministic structure identfied via Intervention Detection AND a suitable ar and ma process reflecting ARIMA and SARIMA. The difference in RSS should be based upon a fixed number of fitted points as the incorporation of ar and ma structure often changes the degrees of freedom.

Related Solutions

Solved – Regression with ARIMA(0,0,0) errors different from linear regression

As pointed out in the comments, the difference between the models is that auto.arima() has not included an intercept. It selects a model, possibly including the constant, using the AICc. With one covariate, the model is $$y_t = \beta_0 x_t + n_t$$ where $n_t$ is an ARIMA process. Note that the intercept is shifted to the ARIMA process. In this example, the selected model for $n_t$ does not include a constant.

If you know what model you want, why use auto.arima()? Instead, you could use

arima(a,xreg=b)

which gives

Series: a 
ARIMA(0,0,0) with non-zero mean 

Coefficients:
      intercept          b
       48638.40  -26143.23
s.e.   32410.27   27893.41

sigma^2 estimated as 93138232:  log likelihood=-254.25
AIC=514.5   AICc=515.7   BIC=518.03

This is the same as the model obtained using lm(a~b). The estimates are identical, but the standard errors are different because they are estimated in a different way (numerically from the hessian matrix rather than using the inverse of $(X'X)$.)

Solved – How to interpret coefficients in a regression with ARIMA errors

To answer some of my own questions, after additional reading and experimenting:

The trick with the coefficient is that it's in the space of the data after Box-Cox transformation. So invert the transformation to get the beta in the original units.
Yes, the standard approach in intervention analysis is to use a step function for the regressor, unless you have reason to think that the effect will be either an impulse or a slower ramp-up.

One of several resources that I found useful was McLeod et al., Time Series Analysis with R. I figured out the Box-Cox thing by reading the forecast::forecast.Arima() code.

Best Answer

Related Solutions

Solved – Regression with ARIMA(0,0,0) errors different from linear regression

Solved – How to interpret coefficients in a regression with ARIMA errors

Related Question