Solved – How to down-weight older data in time series regression

estimationforecastingregressiontime seriestransform

In a regression fit of vectors varying with time $t$
$\qquad y \sim [x_t\ x_{t-1}\ x_{t-2}\ …] \cdot [c_t\ c_{t-1}\ c_{t-2} \ …] $ ,
how can one down-weight the older $x_t$ to model "older is less relevant" (than the regression gives) ?
Say that rows in data matrix $X$ are observables (stocks, market segments …)
and columns are times.
Correct me,
Weighted least squares
weights rows; how does one weight columns, times ?

Added:
Down-weighting older data in time series looks complex, much more so than
linear least squares with i.i.d. errors in rows (observations) —
just divide through by $\sigma_i$; or
classification with i.i.d. errors in columns (e.g. features) —
just centre each column.
Can anyone say either

  • yes, I downweight time series; here are some examples on the web
  • no: down-weighting time series is complex, not for novices.

(http://AndrewGelman.com/2005/06/21/timeseries_regr discussed this question 10 years ago,
mentioning Kalman filters and hierarchical Bayes, but with no examples.)

Best Answer

A common method is to use an exponentially weighted cost function: $$ \sum_i \lambda^{i} e(t-i)^2 $$ where $e(t)$ is the residual error, and $\lambda$ is the forgetting rate. If $\lambda=1$, you get back least squares regression.

You can use recursive least squares (RLS) to find a solution efficiently.

Related Question