Could someone point me to an introductory-level explanation (think Wikipedia entry) of what sequential change point detection is?
Solved – sequential change point detection
change point
Related Solutions
Alexandr,
You need to consider different strategies when building the model and trying to identify outliers. It is an iterative process. Let me just put out some concepts that you need to consider. Differencing, ARMA model structure, parameter changes in the model(ie CHOW test), changes in seasonality, changes in trend and TSAY's focus of outliers, level shifts and variance change. Use WLS when you find a variance change.
We have programmed it. It took us years to perfect as the TSAY work omitted other things as I have listed above.
You could use this free piece of software if you are looking just for level shifts. It does a pretty good job. http://www.beringclimate.noaa.gov/regimes/
Briefly, the package mcp does Bayesian change point regression. As of v0.2, it takes Gaussian, Binomial, Bernoulli, and Poisson. Modeling your data as four intercept-only segments:
model = list(
y ~ 1, # Intercept
~ 1, # etc...
~ 1,
~ 1
)
library(mcp)
df = data.frame(x = seq_along(coverages), y = coverages)
fit = mcp(model, df, par_x = "x")
Let's plot it with a prediction interval, just for fun (green dashed lines). The blue curves are posterior densities for the change point locations. The gray lines are random draws from the posterior.
plot(fit, q_predict = T)
You can use plot_pars() to plot individual parameter estimates. Here are the summaries. where cp_* are the change point estimates:
summary(fit))
Family: gaussian(link = 'identity')
Iterations: 9000 from 3 chains.
Segments:
1: y ~ 1
2: y ~ 1 ~ 1
3: y ~ 1 ~ 1
4: y ~ 1 ~ 1
Population-level parameters:
name mean lower upper Rhat n.eff
cp_1 101.280 99.38 103.0000 1 5627
cp_2 199.562 199.00 200.4314 1 5038
cp_3 299.365 296.85 301.7760 1 2340
int_1 -0.047 -0.11 0.0104 1 5614
int_2 -0.620 -0.68 -0.5592 1 5792
int_3 0.423 0.37 0.4838 1 6463
int_4 -0.018 -0.04 0.0036 1 5382
sigma_1 0.295 0.28 0.3082 1 5963
Read more on the mcp website. Disclaimer: I am the developer of mcp.
Best Answer
In a nutshell, change detection is the problem of determining changes in the distribution of a stochastic process when the decision is made as observations arrive.
The relevant Wikipedia article is this.