Solved – Residual analysis of cross-sectional time-series forecasts

cross-sectionforecastingpanel datartime series

I have forecasts and actuals for panel data (i.e. time-series cross-sectional data). The forecasts are already generated and provided by some source outside of R. I'd like to evaluate the quality of the forecasts.

Are there standard tools in R that perform various diagnostics on the residuals? By diagnostics I mean tests such as:

auto-correlation of residuals across the cross-section
auto-correlation of residuals along the time series for a given member
tests for fixed effects vs. random effects
heteroskedasticity, etc.

Or is the best way to perform these diagnostics to perhaps build a panel model using the forecast as the predictor in the panel model?

Best Answer

If by 'auto-correlation of residuals across the cross-section' you mean cross-correlation of residual time series, then the 'ccf' function in R can do cross-correlation between two univariate time series. Related to this look at : Cross-correlation significance in R

Ljung-Box test is implemented in R, and can tell you something the nature of the univariate residuals time series.

(Halbert) White's test helps with testing for heteroscedasticity in residuals.

HTH.

Related Solutions

Solved – Time series cross section forecasting with R

After a bit of research, I can give a partial answer. In his book Wooldridge discusses Poisson and negative binomial regressions for cross-section and panel data. But for regression with lagged variables he only discusses Poisson regression. Maybe negative binomial is discussed in the new edition. The main conclusion is that for random effects Poisson regression with lagged random variable can be estimated by mixed effects Poisson regression model. The detailed description can be found here. The mixed effects Poisson regression in R can be estimated with glmer from package lme4. To adapt it to work with panel data, you will need to create lagged variable explicitly. Then your estimation command should look something like this:

glmer(y~lagy+exo+(1|Country),data,family=quasipoisson)

You should also look into gplm package suggested by @dickoa. But be sure to check, whether it supports lagged variables. Yves Croissant, the creator of gplm and plm packages writes wonderful code, but unfortunately in my personal experience, the code is not tested enough, so bugs crop up more frequently than in standard R packages.

Panel Data – Validity of Pseudo-Panel Data from Repeated Cross-Sectional Data

I do not know whether there are established methods to compare panel data to repeated cross-sectional data. But I want to add that true panel data is not always superior to repeated cross-sectional data in general. Attrition or learning effects for example may be a problem in panel data but not in repeated cross-sectional data although I do not know whether these problems are present in your case. But if this is the case, the second and third years (and so on) of your panel data may be problematic compared to repeated cross-sectional data in some sense. You should keep this in mind.

In general I think what you want to do sounds doable and it could reveal new information in comparison with the analysis of cross-sectional data only (although I do not know your research question).

If the estimations differ between both analyses I would have a look whether what could be the reasons by looking at the advantages and disadvantes of both types of datasets. There are several papers about the this topic which might help you such as

Deaton (1985)

Verbeek & Nijman (1992)

Frees (2004)

Lee & Niemeier (1996)

Hsiao (2007)

Best Answer

Related Solutions

Solved – Time series cross section forecasting with R

Panel Data – Validity of Pseudo-Panel Data from Repeated Cross-Sectional Data

Related Question