Solved – Smoothing algorithm for irregular time interval

smoothingtime seriesunevenly-spaced-time-series

I have various sets of irregular interval time series data to which I want to apply some sort of smoothing algorithm to produce a good fit.

I have attempted various methods which all were unsatisfactory.

  1. Loess – Too much of a tendency to overshoot/overreact to outliers
  2. Moving Average – The lag is unacceptable

Example Dataset:

http://i.stack.imgur.com/sFkTe.png

I have read about the "Improved Holt Method for Irregular Time Series", but the paper was too difficult for me to understand and implement in C#.

Can someone point me to a good method / algorithm which produces good smoothing?

The method must be able to calculate the smoothed point at time $t$, without requiring $t+1$, etc., data. It also must be capable of dealing with multiple $y$ values for a given $x$ time.

Best Answer

The simplest algorithm is the median filter. You can find an C++ implementation in the R package robfilter. That implementation also include an 'online' version that only uses past data and implements some algorithmic short-cuts.

Of course you will still have to set the "width" argument yourself, but this is the counter part of looking for a simple algorithm (this package also contains more sophisticated smoothing algorithms).

The median-filter is essentially a rolling window median, so it inherits the good behaviour of the median in terms of insensitivity to outliers and non-parametric interpret-ability.

So, considering the dataset you posted, the median filter would yield:

median filter

and the code:

a1<-read.table("sodat.txt",header=TRUE)
library("robfilter")
d1<-med.filter(a1[,2],width=10,online=TRUE)
plot(d1)
Related Question