Solved – How to test for presence of trend in time series

Question

Apart from detecting trend from a time series plot, how do you test for its presence before removing the trend using moving average?

I fitted a mathematical trend to the data and the slope was approximately zero, does that mean there is no trend?

Answer 1

You make assumptions about the behaviour of your trend. Without assumptions you cannot hope to ever test data.

Detrending a time series is effectively modelling a time series.

The most common (and often violated) assumptions you first need to make when dealing with time series are the following.

1. $E|X_t|^2 < \infty, \quad \forall t \in \mathbb N$
2. $EX_t = \mu, \quad \forall t \in \mathbb N$
3. $\gamma(s, r) = \gamma(s + t, r + t), \quad \forall r,s,t \in \mathbb N$

The $\gamma$ in item 3 refers to the correlation function taking $s,r$ indices of the time series as arguments.

Time series that comply are referred to as weakly stationary.

Your data sounds like it already complies to 2. If your data doesn't get "noisier" as time go forward then it also complies to 1.

Often the first step in modelling time series is transforming your data to approximately meet these assumptions, where possible. A simple means of meeting items 1,2, that works some of the time, is to difference the data (repeatedly), wherein you subtract each element of the time-series from the element preceding it. Other options include fitting polynomial regression to the data or performing regression on concatenated windows of a cyclical trends.

What you're hopefully left with is a series that approximately meets these assumptions. There's still plenty of trend to be devoured by your model.

Here's a function to compute $\gamma$.

import numpy as np
def autocorrelation(x):
    assert(len(x.shape) == 1)
    n = x.shape[0]
    x -= x.mean()
    trans = np.fft.fft(x, n=n * 2)
    acf = np.fft.ifft(trans * np.conjugate(trans))[:n]
    acf /= acf[0]
    return np.real(acf)

Now let's look at realizations of some stationary data.

import numpy as np
# Make data that isn't autocorrelated
x_not = np.random.normal(0, 1, 1000)

# Make autocorrelated data
z = np.random.normal(0, 1, 1000)
x_auto = z[1:] - .8 * z[:-1]

If you want an explicit test then $\frac{1.96}{\sqrt n}$, where n is the length of the time-series, is the 95% confidence bounds on null autocorrelation.

You can get pretty wild with time series modelling, but this is the linear regression analog of time series. The collection of these tricks form the guts of what is known as (S)ARIMA.