Solved – Gaussian Process Regression with wide confidence interval

confidence intervalforecastinggaussian processpredictiontime series

I used Gaussian Process Regression to predict a time series, what I have is sensor's readings that come every hour ( I have data for about 3 years) I chose a periodic kernel function which looks like this

$$
K(x,x′)=\sigma_1^2 \exp(−2\sin^2(π|x−x′|/24)/\mathcal{l_1}^2)+\sigma_2^2 \exp(−2\sin^2(π|x−x′|/8)/\mathcal{l_2}^2)
$$

I got the following predictions
enter image description here
However, the confidence interval looks very weird, (same size along the predictions, and wide even around the data), I can't understand why I am getting this answer, can anyone explain to me what's wrong in my work!?

Briefly, this is what I do :

  1. I get the data (sensor readings) and prepare training ($$ y(N*1) $$) and testing data ($$ y_{*}(M*1) $$) from the readings and preparing two corresponding series of numbers $$ x=[1,2,3,….N] $$ and $$ x_*=[1,2,3,….M] $$
  2. Estimate the parameters using maximum likelihood
  3. Calculate $$ K(x,x) +\sigma_{nois}, K(x_*,x)$$ and $$ K(x_*,x_*)$$
  4. Calculate $$ L=K(x_*,x)*{K(x,x)}^{-1} $$
  5. Calculate $$ Prediction=L*y $$ and $$ Covariance =k(x_*,x_*)-L*K(x,x_*)$$
  6. Calculate the confidence interval like this:

$$ lower=Prediction+0.95*sqrt(Covariance) $$
$$ upper=Prediction-0.95*sqrt(Covariance) $$

  1. Do I do anything wrong?! what is the problem?!
  2. Is it right to
    consider the input (the time) in this way? and for the prediction
    mode, if I am going to predict the coming 24 hours should my input
    be $$ x_{pre}=[1,2,3,….,24] $$

Best Answer

The credible interval gets larger the further you extrapolate from the data. In this case, the data density is approximately the same throughout, so that broadening is not observed, and the credible interval is dominated by the estimate of the assumed additive noise contaminating the observations ($\sigma_\mathrm{nois}$). If you extended the plot, say from -100 to +100, I suspect you would see the broadening.

The width of the band look about right for a 95% credible interval, it has about as many data points lying outside it as would be reasonably expected.

If you expect the noise depends on the attribute/feature, you need a heteroscedastic Gaussian process model.

Related Question