Note: I'm a stats novice, please let me know if any of these terms are unclear or misused and I'll update the question!
I'd like to predict future values of a time series. More precisely, I'm interested in the distribution of the next value.
I've chosen a normal/Gaussian prior. I see that I can calculate my new posterior distribution by solving the conjugate prior (https://en.wikipedia.org/wiki/Conjugate_prior, Continuous Distributions section, normal with known variance).
This seems to treat all observations of my time series equally. What if my time series has an underlying trend upward or down – is there a way to represent and quantify this?
For example, I've taken an existing time series with ~30 data points in the range of 30-50. I now add 10 fake data points that are double the max observed value. This barely moves my distribution's mean by ~4. This is across a range of mean/variance combinations for my prior. With 25% of my observations so far above the prior's mean, and especially because they all occur recently, my intuition is that I should see a large shift in posterior mean and posterior variance.
Is my intuition wrong, or is there something like a "Bayes moving normal" that I can use?
Best Answer
Your question about Bayesian Time Series falls perhaps into Bayesian state-space models (Bayesian Filtering) and the problem could be formulated under the same structure.
Within Bayesian state-space models, under the assumption of linearity, you could structure you dynamics as in a Kalman Filter (the "easiest" you can get).
$$ x_{k+1} = Ax + w_k \\ y_{k+1} = Hx + v_k $$
where, if you assume the latent variables are the weights given to the AR factors (down until $t-n$):
$$x_k = x_{k-1} = x = \begin{bmatrix} a_0 \\ \vdots \\ a_n \end{bmatrix} = a$$
and
$$H = [y_{k} \quad y_{k-1} \quad .. ] \dot \quad a$$
A Bayesian state-space model gives you:
Your questions seemed concerned about fine-tuning how much weight should be given to the latest measurements. If we model the measurement noise $v_k$ where $v_k \sim N(0, e_v)$ "smaller" (*) than the model uncertainty, the filter will rely on the latest measurement and update the AR factors $a$ faster in response to high prediction errors
You can see how some AR, MA and ARMA models can be refactored into state-space models here
(*) lot's to say about this - I am trying to convey the idea here. See "Kalman gain tuning"