Solved – Trend Analysis: How to tell random fluctuations from actual changes in trends

test-for-trendtime series

I hope somebody in here can help me: I'm looking for some pointers as to how to distinguish random fluctuation from actual changes in trends, e.g.:

  • In a time series with measures taken at monthly (daily) intervals, for how many months (days) does the change have to be consistent (e.g. going from decreasing to increasing) before one can say that it is an actual change in trend and not just a random fluctuation?

  • How large (percentage-wise or in comparison with the variation) does the change have to be in order to be a real change and not just a random fluctuation?

Is there any any rules of thumb or hard rules on this?

Thanks!

EDIT: Example: If I have a time series with 60 observations, and the trend change (from decreasing to increasing, say) at observation 30, when will I be able to detect that it is a real change in the trend and not a random fluctuation? At observation 31? 32? 33? Something else?

r1 <- seq(1,30, by = 1)
r2 <- 1/r1
r3 <- sort(r2)
ex <- append(r2,r3)

   ex
[1] 1.00000000 0.50000000 0.33333333 0.25000000 0.20000000 0.16666667 0.14285714 0.12500000 0.11111111 0.10000000 0.09090909 0.08333333 0.07692308 0.07142857 0.06666667 0.06250000 0.05882353 0.05555556 0.05263158 0.05000000 0.04761905 0.04545455 0.04347826 0.04166667 0.04000000
[26] 0.03846154 0.03703704 0.03571429 0.03448276 0.03333333 0.03333333 0.03448276 0.03571429 0.03703704 0.03846154 0.04000000 0.04166667 0.04347826 0.04545455 0.04761905 0.05000000 0.05263158 0.05555556 0.05882353 0.06250000 0.06666667 0.07142857 0.07692308 0.08333333 0.09090909
[51] 0.10000000 0.11111111 0.12500000 0.14285714 0.16666667 0.20000000 0.25000000 0.33333333 0.50000000 1.00000000

Best Answer

Expanding on @Aleksendr Blekh's answer. You can use the below code for decomposing trend, using the decompose function in R:

births <- scan("http://robjhyndman.com/tsdldata/data/nybirths.dat")
birthstimeseries <- ts(births, frequency=12, start=c(1946,1))
if(!require("TTR")) { install.packages("TTR");  require("TTR") }
birthstimeseriescomponents <- decompose(birthstimeseries)
birthstimeseriescomponents$seasonal # the seasonal component
birthstimeseriescomponents$trend # the trend component
birthstimeseriescomponents$random # random (irregular / remainder) component
# plot the seasonality, trend and random (epsilon) components
plot(birthstimeseriescomponents)

For changes in trends, relevant reading: Hidden Markov Models, Change Point Analysis.

This post may also be helpful to find fluctuations from the underlying trend. The approach here is fitting a polynomial instead of time series decomposition and reporting points that fall outside the confidence intervals.

Related Question