Solved – Highly irregular time series

multilevel-analysistime seriesunevenly-spaced-time-series

I have data for the population of a number of different fish, sampled over a period of about 5 years, but in a very irregular pattern. Sometimes there are months between samples, sometimes there are several samples in one month. There are also many 0 counts

How to deal with such data?

I can graph it easily enough in R, but the graphs are not particularly illuminating, because they are very bumpy.

In terms of modeling – with species modeled as a function of various things – maybe a mixed model (aka multilevel model).

Any references or ideas welcome

Some details in response to comments

There are about 15 species.

I am trying to both get an idea of any trends or seasonality in each fish, and look at how the species are related to each other (my client originally wanted a simple table of correlations)

The goal is descriptive and analytic, not predictive

Further edits: I did find this paper by K. Rehfield et al., which suggests using Gaussian kernels to estimate the ACF for highly irregular time series

http://www.nonlin-processes-geophys.net/18/389/2011/npg-18-389-2011.pdf

Best Answer

I have spent quite some time building a general framework for unevenly-spaced time series: http://www.eckner.com/research.html

In addition, I have written a paper is about trend and seasonality estimation for unevenly-spaced time series.

I hope you will find the results helpful!

Related Solutions

Solved – Moving average of irregular time series data using R

If the problem is just plotting the data (i.e., if you have already identified the faults), you can simply write a function that counts the events in the past hour, as follows.

# Sample data, with two peaks
library(zoo)
seconds <- 60*60*24 # One day
lambda <- function(x) ( 1 + sin(2*pi*x/seconds) ^ 2 ) / 2
n <- 100
x <- seconds*sort(runif(n))
x <- x[runif(n) < lambda(x)]
x <- zoo( 0*x+1, x )
plot( index(x), x, type="h" ) 

# Moving average over the past hour
f <- function(u) sum(u - 3600 < index(x) & index(x) <= u )
f <- Vectorize(f)
curve(f(x), xlim=c(0,seconds))

# Exponentially-weighted average, half-life = 1 hour
tau <- 3600
f <- function(u) sum( ifelse(index(x)<=u,exp((index(x)-u)/tau),0) )
f <- Vectorize(f)
curve(f(x), xlim=c(0,seconds))

But this will be inefficient if there is a lot of data. Instead, you can consider two types of events: "a fault occurred" (i.e., enters our window of observation) and "a fault occurred one hour ago" (i.e., leaves our window of observation). Associate the values +1 and -1 to those events, and compute the cumulated sum: that is the number of faults in the previous hour.

# More efficient moving average
y <- merge(x, zoo(-x, index(x) + 3600), fill=0)
plot( cumsum(y[,1] + y[,2]) )

Solved – Asynchronous (irregular) Time Series Analysis

I know of one possible solution, but it is sufficiently complicated that I'm going to take the easy option and link you to the relevant academic paper (a critically under-rated paper in my opinion):

Frank de Jong, Theo Nijman (1997) "High Frequency Analysis of Lead-Lag Relationships Between Financial Markets"

I'm sure more work must have been done on this problem since then. A good way to find it is to use the "citations" page on ideas.repec. A link to the relevant page for the above-mentioned paper is here. A few titles look quite relevant.

Related Question