Solved – which test can be used for testing if the week over week increase/decrease is significant

statistical significance

I read some posts about this on this forum, but didn't seem to find an easy to understand explanation. so trying to post it here, please help to explain a bit more as i am not an expert on this but would really like to learn.

my question is almost just like the subject, say if I have some week over week stats like volume of traffic, for the current week, how do i know if the difference is significant or not. The purpose is if it's significant, then we would want to alert and take a look at it. If it's within noise, then we can have a nice weekend. also how to take into account seasonality, for example we expect to see the increase for holidays or when promotion happens.

Best Answer

My first try to approach this issue is through simple linear regression, in that case you could just run a regression with independent variable as week num and the dependent variable as traffic count, assuming linear relation, you could try to capture the long term trend, though this attitude assumes that all the OLS assumptions arn't violated, and in time series (which is your case) this could be easily violated.

If youre using R, you could use the auto.arima function, and add external explaining variable as long term trend, let me show you an example with R (i tried to paste the data but got blocked by SO limitations)

modArima <- auto.arima(data$count,xreg=train)
fit2<-forecast(modArima,h=horizon,xreg=predict)
summary(fit2)

and view Results of fit2, notice that the last coefficient of -0.6449 that shows the long term drop is significant

Here you can see that the prediction manages to catch the long term drop:

Data + Prediction with long term drop: